Probability

Andrey Kolmogorov defined a set of characteristics that any probability \(P\) measure should have, these are called the Kolmogorov’s axioms (1933):

1. \(P(A)\in\mathbb{R}\) and \(P(A)\geq 0\), for any event \(A\subseteq\Omega\).
2. \(P(\Omega)=1\)
3. For any \(A\) and \(B\) disjoint, \(P(A)+P(B)=P(A\cup B)\)

Corollary

Let \(A\) and \(B\) be some events in \(\Omega\)

1. \(P(\emptyset)=0\)
2. \(P(A^c)=1-P(A)\)
3. \(A\subset B\Rightarrow P(A)\leq P(B)\)
4. \(0\leq P(A)\leq 1,\ \forall A\)
5. \(P(A\setminus B)=P(A)-P(A\cap B)\)

Corollary

6. \(P(A\cup B)= P(A)+P(B)-P(A\cap B)\)
7. \[P\left(\bigcup^n_{i=1}A_i\right)=\sum_{i=1}^n P(A_i)-\sum_{i\neq j} P(A_i\cap A_j)+\\ \sum_{i\neq j\neq k} P(A_i\cap A_j\cap A_k)+...+(-1)^{n-1}P\left(\bigcap_{i=1}^n A_i\right)\]

Conditional Probability

Conditional probability, as the wording implies, means the probability of something happening given something else has happened. Now, note “something” here makes reference to an event.

\[P(A|B)\]

It reads the probability of \(A\), given \(B\).

Conditional Probability

Note that if we think on sets, saying given \(B\) we are immediately excluding everything that could have happened if \(B\) did not happen, and therefore our Universal set is no longer \(\Omega\), but \(B\).

What we are looking for are, among the events that live in \(B\), how many of those live in \(A\) (because those would trigger event \(A\)). Actually, we are interested on the relative measure of those outcomes, compared to the whole size of \(B\): \[P(A|B)=\frac{P(A\cap B)}{P(B)}\]

Example

Consider a factory that makes 10 wrenches 🔧. Among those, we know that 2 have imperfections. Suppose you intend to remove, randomly, 2 🔧 from the lot (of 10). Consider the following events:

\(A = \{\text{The first :wrench: is faulty}\}\) \(B = \{\text{The second :wrench: is faulty}\}\)

What if we want to compute \(P(B)\)? For a correct assessment for \(B\), we would better have some information on the realization of \(A\)!

Example

If the fist 🔧 was faulty, then \(A\) happened. If the first 🔧 was ok, then \(A^c\) happened, and therefore we can compute \(P(B|A)\) and \(P(B|A^c)\). We are assuming that we are removing these 🔧 without replacing them.

Let’s see why this last detail (replacing the 🔧) is so relevant before going on.

Example: With replacement

Example: with replacement

• With \(A\), we know that when we pick the second wrench (\(B\)), in the box there are 2 💥 and 8 🔧.

• With \(A^c\), we know that when we pick the second wrench (\(B\)), in the box there are 2 💥 and 8 🔧.

Example: no replacement

Example: no replacement

• With \(A\), we know that when we pick the second wrench (\(B\)), in the box there are 1 💥 and 9 🔧.

• With \(A^c\), we know that when we pick the second wrench (\(B\)), in the box there are 2 💥 and 9 🔧.

Conclusion Conditional Probability

So formally

Corollary

It follows that the identity \[P(B|A)=\frac{P(A\cap B)}{P(A)}\] or \[P(A\cap B)=P(B|A)P(A)\] with \(P(A)\neq 0\) also holds true.

Corollary

Now let’s think on \(P(A\cap B \cap C)\):

• \(P(A\cap B \cap C)=P(A\cap (B \cap C))\)
• \(P(A\cap B \cap C)=P(A|B\cap C)P(B\cap C)\)
• \(P(A\cap B \cap C)=P(A|B\cap C)P(B|C) P(C)\)

Corollary

Note that given the commutativity of the intersection, we could have obtained also:

• \(P(A\cap B \cap C)=P(A|B\cap C)P(C|B) P(B)\)
• \(P(A\cap B \cap C)=P(B|A\cap C)P(A|C) P(C)\)
• \(P(A\cap B \cap C)=P(B|A\cap C)P(C|A) P(A)\)
• \(P(A\cap B \cap C)=P(C|A\cap B)P(A|B) P(B)\)
• \(P(A\cap B \cap C)=P(C|A\cap B)P(B|A) P(A)\)

Example

Let’s define the events:

\(W=\{Woman\}\), \(M=\{Man\}\), \(U=\{Unemployed\}\)

1. Woman: \(P(W)=\frac{525}{1000}=0.525\)
2. Unemployed: \(P(U)=\frac{100}{1000} = 0.1\)
3. Unemployed woman: \(P(W\cap U)=\frac{55}{1000}=0.055\)

Example

2. A citizen is randomly chosen from the population, and it happens to be a woman. What is the probability she is unemployed?

Independent Events

Independent Events

From the definition of independence, we can obtain several properties. Let \(A\) and \(B\) independent events with \(P(A)P(B)>0\):

1. \(P(A|B)=P(A)\) and \(P(B|A)=P(B)\) (remember conditional example with replacement)
2. \(A^c\) and \(B\) are independent, as well as \(A\) and \(B^c\), and even \(A^c\) and \(B^c\).
3. If \(A\) and \(B\) are incompatible, they cannot be independent. \(P(A\cap B)=P(\emptyset)=0\neq P(A)P(B)\)
4. Any event is independent of \(\Omega\) and \(\emptyset\).

Example

Are \(W\) and \(U\) from the previous example independent?

Example

Consider a die 🎲 that is thrown twice. Consider the following two events:

\(A=\{\text{The die shows an odd number the first time}\}\) \(B=\{\text{The die shows a number }>4\text{ the second time}\}\)

Are \(A\) and \(B\) independent events?

Example

In this case, \(\Omega=\{(x,y)\in \mathbb{N}^2| x,y \leq 6\}\) with \(\# \Omega = 6^2=36\)

• \(P(A)=\frac{18}{36}=\frac{1}{2}\)

• \(P(B)=\frac{12}{36}=\frac{1}{3}\)

• \(P(A\cap B)=\frac{1}{6}=P(A)P(B)\)

• \(P(A|B)=\frac{P(A\cap B)}{P(B)}=\frac{1}{2}=P(A)\)

• \(P(B|A)=\frac{P(B\cap A)}{P(A)}=\frac{1}{3}=P(B)\)

• They are independent events!

Remark

Two events being independent is not the same that they being incompatible:

Example

Let \(A\) and \(B\) be two events such that: \(P(A)=0.6\), \(P(B)=t\), and \(P(A\cup B)=0.8\)

Find \(t\) such that \(A\) and \(B\) are:

1. Mutually exclusive or incompatible.
2. Independent.

Example

1. In this case, what we need is that \(P(A\cap B)=0\).

Example

2. To make \(A\) and \(B\) independent, we need that \(P(A\cap B)=P(A)P(B)=0.6t\):

Law of Total Probability

Example

Consider a financial institution that sells two products, \(\alpha\) and \(\beta\), with very high yields. It is known that, among its clients, 10% invest a share of their wealth in \(\alpha\) and the rest in \(\beta\). From those who invest in \(\alpha\), 70% manage to get returns above the market. From among those who do not invest in \(\alpha\), 55% get returns above the market. Randomly choosing a client of this firm, find the probability this customer gets a return above the market.

Example

Let’s define the events:

• \(A_1\) the client invest in \(\alpha\)
• \(A_2\) the client invest in \(\beta\)
• \(B\) has returns above the market.

Example

\[P(B)=\sum_{i=1}^2 P(A_i\cap B)\] \[P(B)=P(B|A_1)P(A_1)+P(B|A_2)P(A_2)\] \[P(B)=0.7\times 0.1 + 0.55\times 0.9 = 0.565\]

Bayes Theorem

Note that this is a consequence of the Law of Total Probability.

Bayes Theorem

On the other side, \(\sum_i P(A_i)=1\) and \(\sum_{i} P(A_i|B)=1\)

Bayes Theorem has been widely used in economics, in biomedical sciences, and social sciences when looking for causality.

If event \(B\) represents consequences and event \(A_i\) probable cause, Bayes Theorem allows to assess the probability of this cause \((P(A_i))\).

Example

Let’s go back to the previous example, about our investors.

Example

If the customer invested in \(\beta\), then the event we are trying to is \(A_2\), but conditional on event \(B\), \(P(A_2|B)\):

\[P(A_2|B)=\frac{P(A_2\cap B)}{P(B)}=\frac{P(A_2)\times P(B|A_2)}{\sum_i P(A_i)\times P(B|A_i)}\]

We knew from the previous exercise that \(P(B)=0.565\), and therefore we obtain:

\[P(A_2|B)=\frac{0.9\times 0.55}{0.565}=0.876\]

Example

How do we interpret this?

Example

All these computations can be very easy with the help of the following table:

Example

Let’s verify now if the event \(A_1\) and \(B^c\) are independent or not!

Practice

The exercises go from easy to hard:

• 🟢 warm-up: one formula, one step
• 🟡 standard: two or three steps
• 🟠 exam level: this is what you will find in the midterm
• 🔴 stretch: you need to combine several things

🟢 Practice · Exercise 1

Let \(A\) and \(B\) be two events with \(0<P(A)<1\) and \(0<P(B)<1\). You know that \(A\subset B\).

What is \(P[(A\cup B)\cap B^c]\)?

A. \(0\)

B. \(P(A)\)

C. \(P(B)\)

D. \(1\)

🟢 Practice · Exercise 2

Let \(A\) and \(B\) be two events such that \(P(A)=0.75\), \(P(B)=0.5\) and \(P(A\cup B)=1\).

Find \(P(A|B)\).

🟡 Practice · Exercise 3

Let \(P(A)=a\) and \(P(B)=b\), with \(0<a<1\) and \(0<b<1\). True or false, the probability that neither \(A\) nor \(B\) happen is:

(a) \(1-a-b+ab\), if \(A\) and \(B\) are independent.

(b) \(1\), if \(A\) and \(B\) are complementary.

🟡 Practice · Exercise 4

Let \(A,B,C\) be events such that:

\(A\cup B\cup C=\Omega\), \(P(A)=0.3\), \(P(C)=0.5\), \(P(B^c)=0.7\), \(A\cap B=\emptyset\) and \(B\cap C=\emptyset\).

Find \(P(A\cap C)\).

🟠 Practice · Exercise 5

\(A\) and \(B\) are independent, and \(A\) has a probability twice as large as \(B\). The probability that at least one of them happens is \(0.5\).

Find \(P(B)\).

🟠 Practice · Exercise 6

5% of the students are excluded from continuous assessment 📚, and of these, 98% end up with a final grade above 13. Of those not excluded, 10% end up with a final grade below 13.

Pick a student randomly at the end of the semester.

(a) What is the probability of a grade above 13, given the student was in continuous assessment?

(b) Knowing the student got a grade above 13, what is the probability that the student was excluded?

✅ Practice · Exercise 6 · Solution

📝 Homework

Problem set 2.1, Questions 3 to 10. Questions 8 to 10 are Bayes, like Exercise 6, and we will come back to them before the midterm.

Bibliography

• Murteira, B.; Ribeiro C.; Silva, J. and Pimenta, C. (2010) Introdução à Estatística (2^a Edição). McGraw-hill.
• Paulino C.D.; Branco J.A. (2005). Exercícios de Probabilidade e Estatística. Escolar Editora.
• Pedrosa, A.; Gama, S. (2004). Introdução Computacional à Probabilidade e Estatística. Porto Editora.