Bernoulli experiment (or trial)

The Bernoulli experiment consists in a random experiment with the following features:

• Only two events: \(A\) and \(A^c\), for example success and failure. \(\Omega=\{A,A^c\}\)
• The success happens with probability \(p\) and failure with probability \(q=1-p\): \[P(A)=p\quad P(A^c)=q=1-p\]

Bernoulli Process

This process has even further these three characteristics:

• Only two possible outcomes in each draw
• \(P(A)\) remains constant thorough the experiment
• Repeated trials are independent, i.e. what happened before does not affect what will happen in the future.

Bernoulli Distribution

The random variable \(X\) follows a Bernoulli distribution with parameter \(p \in [0,1]\), \(X\sim Ber(1,p)\) if its probability function is given by: \[P(X=x)=f_X(x)=\begin{cases}p^x(1-p)^{1-x} & x\in\{0,1\}\\ 0 & otherwise\end{cases}\]

It is trivial to check that \(P(X=1)=p\) and \(P(X=0)=1-p\).

Bernoulli Distribution

The first two moments of the distribution (mean and variance) are given by:

• \(E[X]=p\)
• \(V[X]=p(1-p)\)

In summary, Bernoulli’s tries to model what is the probability of success or failure in a single independent trial.

Binomial Distribution

This distribution, is a generalization of Bernoulli’s.

In this case, it might be better to start from the summary to stress the difference with Bernoulli’s:

The Binomial distribution answers what is the probability that, after \(n\) independent fail-success trials, in any order, for example, you succeeded \(m\leq n\) times.

Binomial Distribution

A random variable \(X\) follows a Binomial distribution, \(X\sim Bin(n,p)\) if its probability function \(P(X=x)\) is: \[ f_X(x)=\begin{cases}\binom{n}{x}p^x(1-p)^{n-x} & x=0,1,...,n\\ 0 & otherwise\end{cases} \]

Note that if we change \(n\) or \(p\) we would get another binomial distribution, i.e. \(B(n,p)\neq B(m,r)\) if either \(n\neq m\) or \(p\neq r\).

Bernoulli is a Binomial distribution with \(n=1\).

Side note about binomials

Binomial distribution

Computing this probability can become very laborious, even for small values for \(n\). This can be solved thanks to:

1. Computers and advanced calculators
2. Tables for values for \(n\) from 1 to 20, and for \(p\) from 0.05 to 0.5. Note that if \(p>0.5\) you can just \[X\sim Bin(x,n,p)\ \leftrightarrow\ \hat{X}\sim Bin(n-x,n,1-p)\]

Binomial distribution

The first two moments of the distribution are found as:

1. \(E[X]=\mu_X=\sum x\binom{n}{x}p^x(1-p)^{n-x}=np\)
2. \(V[X]=E[(X-\mu)^2]=\\=\sum x^2\binom{n}{x}p^x(1-p)^{n-x}-\mu^2=np(1-p)\)

Additivity Theorem of the Binomial Distribution

Let \(k\) independent random variables \(X_i\) with \(i=1,2,..., k\), where \(X_i\sim Bin(x_i,n_i,p)\), then

\[ \begin{aligned} \text{if }S_k&=X_1+X_2+...+X_k=\sum_{i=1}^k X_i \\ S_k&\sim Bin\left(s_k, n=\sum_{i=1}^k n_i, p\right) \end{aligned} \]

Example

Using the tables for the binomial distribution, or the calculator, find the value of the following probabilities:

1. \(P(X\leq 3)\) if \(X\sim Bin(9,0.4)\)
2. \(P(2\leq X\leq 6)\) if \(X\sim Bin(7,0.3)\)
3. If \(X\sim Bin(10,p)\) and \(P(X\leq 5)=0.8338\), what is \(p\)?

Example

The SNS (Portugal’s National Health Service) has said that the probability of an individual carrying a specific gene, that triggers a certain disease is 0.1. In the city of Porto, 10 individuals were randomly selected for a study, while 8 individuals were randomly chosen in Lisbon for the same purpose. Assess the veracity of the following sentence:

❓ Bernoulli and Binomial · Question 1

If \(X\sim Bin(n,p)\), then \(E[X]\) and \(V[X]\) are:

A. \(np\) and \(np(1-p)\)

B. \(np\) and \(np\)

C. \(p\) and \(p(1-p)\)

D. \(np^2\) and \(np\)

❓ Bernoulli and Binomial · Question 2

A Binomial model requires:

A. draws without replacement

B. a continuous support

C. \(n\) independent trials with a constant \(p\)

D. a known \(\lambda\)

✏️ Bernoulli and Binomial · Question 3

A machine produces defective items with probability \(p=0.1\), independently.

In a sample of \(n=10\) items, compute \(P(X=0)\), \(P(X\leq 1)\), \(E[X]\) and \(V[X]\).

Hypergeometric Distribution

Suppose you have a population with size \(N\), from where \(M\) elements have some characteristic, and therefore \(N-M\) do not. From that sample, we draw randomly a sample of size \(n\), without replacement, and we would like to know, from this sample, how many have the characteristic we mentioned.

Hypergeometric Distribution

In this case the r.v. \(X\) would be how many elements from our sample have the referred characteristic. \(X\sim Hypergeometric (N,M,n)\) if its probability distribution function \(P(X=x)\) is:

\[ f_X(x)=\frac{\binom{M}{x}\binom{N-M}{n-x}}{\binom{N}{n}} \]

where \(max(0,M+n-N)\leq x \leq min(M,n)\)

Hypergeometric Distribution

The first two moments of the distribution are found as:

• \(E[X]=\mu_X=n\frac{M}{N}=np\)
• \(V[X]=npq\frac{N-n}{N-1}\) note here that because we do not replace elements, we get a smaller variance.

Hypergeometric vs Binomial

Hypergeometric \(\rightarrow\) Binomial

❓ Hypergeometric · Question 1

The Hypergeometric differs from the Binomial because the draws are:

A. independent

B. infinite in number

C. continuous

D. without replacement, so \(p\) changes at each draw

❓ Hypergeometric · Question 2

Approximating a Hypergeometric by a Binomial is safe when:

A. \(N\) is large

B. \(\frac{n}{N}\leq 0.1\)

C. \(M=N\)

D. \(n>\frac{N}{2}\)

✏️ Hypergeometric · Question 3

A box holds \(N=20\) items, of which \(M=5\) are defective. You draw \(n=4\) without replacement.

Compute \(P(X=1)\), and say whether the Binomial approximation would be appropriate here.

Practice

The exercises go from easy to hard:

• 🟢 warm-up: one formula, one step
• 🟡 standard: two or three steps
• 🟠 exam level: this is what you will find in the midterm
• 🔴 stretch: you need to combine several things

🟢 Practice · Exercise 1

Let \(X\sim Bin(8,0.35)\). Using the table, find:

(a) \(P(X\leq 2)\)

(b) \(P(X=4)\)

(c) \(P(X\geq 6)\)

🟢 Practice · Exercise 2

For a given company, the probability of a receipt 🧾 being filled incorrectly is 0.2. You pick randomly 10 receipts.

Find the probability that:

(a) exactly 5 receipts contain irregularities;

(b) at most 3 receipts contain irregularities;

(c) at least one receipt was filled incorrectly.

🟡 Practice · Exercise 3

Let \(X\sim Bin(10,0.7)\). Find \(P(X\geq 8)\).

🟠 Practice · Exercise 5

Machine 1 produces 6 pieces per hour and machine 2 produces 4. Each piece is defective with probability 0.1, independently of everything else.

Find the probability that in one hour, between both machines, at most one defective piece is produced.

🟠 Practice · Exercise 6

An auditor 🔍 receives a folder with 50 invoices, 6 of which contain errors. She checks 5 invoices, chosen randomly, without replacement.

(a) Find the probability that she finds at least one invoice with errors.

(b) Compute the same probability with the Binomial approximation. Is the approximation good here?

✅ Practice · Exercise 6 · Solution

🔴 Practice · Exercise 7

Each receipt is filled incorrectly with probability 0.2.

How many receipts must you inspect, at least, to have a probability of at least 0.9 of finding one or more incorrect receipts?

🔴 Practice · Exercise 8

Let \(X\sim Bin(n,p)\) with \(E[X]=4\) and \(V[X]=2.4\).

(a) Find \(n\) and \(p\), and then \(P(X\leq 3)\).

(b) True or false: if \(X\sim Bin(n,p)\) and \(Y\sim Bin(n,\theta)\) are independent, then \(X+Y\sim Bin(n,p+\theta)\).

📝 Homework

Problem set 3.1, Questions 1 to 5.

Bibliography

• Murteira, B.; Silva Ribeiro, C.; Andrade e Silva, J. & Pimenta, C.,Introdução à Estatística,Escolar Editora,McGraw-Hill, 2010
• Paulino, C. D. & Branco, J. A. (2005). Exercícios de Probabilidade e Estatística. Escolar Editora
• Pimenta, F., Andrade e Silva, J.; Silva Ribeiro, C. & Murteira, B., Introdução à Estatística, 3ª Edição, Escolar Editora, 2015

Binomials

Back to Binomial Distribution