Sets, Events and the Concept of Probability
Statistics I · Lecture 4
Today
- The concept of probability
- A refresh on set theory
- Sample space and events
- Laplace and frequency interpretations
Disclaimer
These slides are a free translation and adaptation from the slide deck for Estatística I by Prof. Sandra Custódio and Prof. Teresa Ferreira from the Lisbon Accounting and Business School, Polytechnic University of Lisbon.
Probability, background
The concept
Consider the following scenario:
:briefcase: Investor: What’s the probability this startup will succeed?
:bar_chart: Analyst: Hard to say, every startup is different.
:briefcase: Investor: But if you had to guess, based on similar cases?
:bar_chart: Analyst: Maybe 1 in 3 succeed under these conditions.
:briefcase: Investor: So, would you bet on it?
:bar_chart: Analyst: Yes, I would.
:briefcase: Investor: Even if the odds aren’t great?
:bar_chart: Analyst: I believe this one has what it takes.
The concept
Here we can define probability in terms of frequency of occurrence, i.e. as a percentage of successes in a moderately large number of similar situations.
This is the most natural and traditional way of thinking about probability.
- Regarding a fair coin :coin: we could say: “with probability 50% the coin lands on heads” meaning “roughly half of the time.”
But, what if this company belongs to a completely novel market sector?
The concept
There might be situations where the frequency concept is not adequate, because it might refer to a one-time event. These are subjective beliefs.
A company is recruiting a new CEO, and a board member says:
“I believe there’s a 90% chance that our chosen candidate will be an effective CEO.”
The concept
It might seem easy to disregard the second case as unscientific or useless. However, many times people need to make decisions under uncertainty with not enough data (or no data at all!) about previous realizations of the specific event.
Beliefs allow the decision maker to, well, make some decision, at least consistently.
- What’s the difference between both situations?
- What do they have in common?
Uncertainty
A refresh on Set Theory
Sets and elements
A set is a collection of objects, which are elements of the set.
How to write down a Set
There are several ways to specify a set.
By extension, or as a list:
If a set \(S\) has a finite number of elements (\(x_i\in S\)) we can write it like this: \[S=\{x_1, x_2, ..., x_n\}\]
If a set \(S\) has an infinite (but countable) elements (\(x_i\in S\)) we can write it like: \[S=\{x_1, x_2, ...\}\]
How to write down a Set
By describing the property (\(P\)) that \(x\) must satisfy to be included in \(S\): \[S= \{x|x \text{ satisfies } P\}\] in this case \(|\) reads as such that. For example \[S=\{x\in\mathbb{R}\mid x\geq 0\}\] to describe the non-negative real numbers.
This example is special, as the positive real numbers cannot be written down as a a list. In this case the interval \([0,\infty)\) is an uncountable set.
More definitions
More definitions
The universal set is important because it defines the scope of our analysis. Say we are studying the performance of students of Statistics I in 2026.
The cars parked outside our institution do not belong to the universal set, because they are not relevant for our purpose. Only students of Statistics I in 2026 belong to the universal set.
Set Operations
Set Operations
Note that \(\vee\) stands for or, and \(\wedge\) stands for and.
Set Operations
Sometimes we might need to consider the union or intersection of many sets, and for that we can use a notation similar to the one we used for summations:
\[\bigcup_{n=1}^\infty S_n = S_1 \cup S_2 \cup ... = \{x\in\Omega | x \in S_n \text{ for some } n\}\]
\[\bigcap_{n=1}^\infty S_n = S_1 \cap S_2 \cap ... = \{x\in\Omega | x \in S_n \text{ for every } n\}\]
Set Operations
Set Definition
Set definition
❓ Set Theory · Question 1
Two sets \(S\) and \(T\) are disjoint when:
A. \(S\cup T=\emptyset\)
B. \(S=T\)
C. \(S\cap T=\emptyset\)
D. \(S\subseteq T\)
✅ C. Disjoint means they share no element at all, so their intersection is empty. Their union is empty only if both sets are.
❓ Set Theory · Question 2
The set \(S\setminus T\) contains exactly the elements that:
A. belong to both \(S\) and \(T\)
B. belong to \(T\) but not to \(S\)
C. belong to neither
D. belong to \(S\) but not to \(T\)
✅ D. \(S\setminus T=\{x\in S\,|\,x\notin T\}\). Note the operation is not symmetric: \(S\setminus T\) and \(T\setminus S\) are different sets.
✏️ Set Theory · Question 3
Let \(\Omega=\{1,2,3,4,5,6\}\), \(A=\{1\}\), \(B=\{3,6\}\) and \(C=\{2,4,6\}\).
Write down \(A\cup B\), \(B\cap C\), \((B\cup C)^c\) and \(C\setminus B\).
✅ Set Theory · Solution
\(A\cup B=\{1,3,6\}\)
\(B\cap C=\{6\}\)
\(B\cup C=\{2,3,4,6\}\), so \((B\cup C)^c=\{1,5\}\)
\(C\setminus B=\{2,4\}\)
Back to Probability
In probability, \(\Omega\), the universal set, is a non-empty set that contains all possible outcomes of an experiment. Each outcome is represented by \(\omega\), and obviously \(\omega\in\Omega\).
Back to Probability
The sample space (\(\Omega\)) can be:
- Discrete, when \(\#\Omega\) is finite, or countable infinite.
- Continuous, when \(\#\Omega\) is uncountable.
Back to Probability
Consider the experiment of throwing a die :game_die: and noting the number shown on side facing upwards.
- The sample space is \(\Omega=\{1,2,3,4,5,6\}\)
- In this case \(\# \Omega = 6\)
- \(\Omega\) is discrete.
Back to Probability
Consider now the random experiment of measuring the life expectancy of a lamp :bulb:, measured in hours.
- The sample space is \(\Omega=\{x\in\mathbb{R}|x\geq 0\}\)
- In this case, \(\Omega\) is all non-negative real numbers.
- \(\Omega\) is continuous.
Remember, \(\Omega\) must include all possible outcomes from your experiment! Even then ones that seem ludicrous.
Back to Probability
Example
Let’s go back to our experiment with the :game_die:
The sample space is: \(\Omega=\{1,2,3,4,5,6\}\)
Within this space, we can define the following events:
- \(A=\{1,3,5\}\), i.e. the number is odd.
- \(B=\{3,4,5,6\}\), i.e. the number is at least 3.
- \(C=\{1,2,3\}\) , i.e. the number is lower than 4.
- \(D=\{6\}\), i.e. the number is larger than 5.
Example
Now let’s revisit the example of our :bulb:
The sample space is: \(\Omega=\{x\in\mathbb{R}|x\geq 0\}\)
In this space, we can define the following events:
- \(A=\{x\in\mathbb{R}|75<x<95\}\), i.e. the :bulb: lasts between 75 and 95 hours.
- \(B=\{x\in\mathbb{R}|x\leq 100\}\), i.e. the :bulb: lasts no longer than 100 hours.
- \(C=\{x\in\mathbb{R}|x\geq 60\}\), i.e. the :bulb: lasts at least 60 hours.
Events
Mixing up Sets and Probability
Consider two events \(A\) and \(B\) both subsets of \(\Omega\)
\(A^c\) contains all the outcomes that are not in \(A\). \(A^c\) is the event of not \(A\).
If \(A\subseteq B\), then an outcome that realizes event \(A\) (\(\omega\in A\)), also realizes \(B\), as \(A\subseteq B\Rightarrow \omega\in B\) as well. \(A\Rightarrow B\)
For \(A\cup B\) to happen, we need \(\omega \in A\) or \(\omega \in B\), which means that \(A\) happens, or \(B\) happens, or both happen simultaneously.
Mixing up Sets and Probability
For \(A\cap B\) to happen, we need \(\omega \in A\) and \(\omega \in B\), which means that \(A\) and \(B\) happen simultaneously.
\(A\) and \(B\) are incompatible if \(A\cap B=\emptyset\), i.e. if an outcome is in one set, it cannot be in another, for example it cannot be that \(A\) and \(A^c\) happen simultaneously!
Inherited set properties for events
Consider two events \(A\) and \(B\) both subsets of \(\Omega\)
- Commutativity: \(A\cup B = B\cup A\); \(A\cap B=B\cap A\)
- Associativity: \((A\cup B)\cup C=A\cup(B\cup C)\); \((A\cap B)\cap C=A\cap(B\cap C)\)
- Distributivity: \(A\cup(B\cap C)=(A\cup B)\cap(A\cup C)\); \(A\cap (B\cup C)=(A\cap B)\cup(A\cap C)\)
- De Morgan’s Laws: \((A\cap B)^c=A^c\cup B^c\); \((A\cup B)^c=A^c\cap B^c\)
Inherited set properties for events
- \(\left(A^c\right)^c=A\)
- Complement law: \(A\cup A^c=\Omega\); \(A\cap A^c=\emptyset\)
- Identity element: \(A\cup\emptyset = A\); \(A\cap\Omega = A\)
- Absorbing element: \(A\cup\Omega = \Omega\); \(A\cap\emptyset = \emptyset\)
- Idempotent law: \(A\cup A=A\) ; \(A\cap A=A\)
- \(A\subset B\Rightarrow A\cap B=A\); \(A\subset B\Rightarrow A\cup B = B\)
Example
Consider the sample space \(\Omega =\{1,2,3,4,5,6\}\), from our :game_die: case.
Define the events: \[A=\{1\},\ B=\{3,6\},\ C=\{2,4,6\},\ D=\{4,5,6\}\]
Example
Let’s define the following events in \(\Omega\)
- \(A\cup B\)
- \(A\cap B\)
- \(A^c\)
- \((A\cup B)^c\)
- \((B\cap C)^c\)
- \(B\setminus C\)
- \(C\setminus D\)
Concept of probability
Besides the concepts we already saw of frequency and subjectivity for probability, there was an older, called “classic” one. This one was introduced by Pierre-Simon Laplace in 1812.
Example
Consider now an experiment throwing two dice :game_die: :game_die:
- How many outcomes are in \(\Omega\)? \(6^2=36\), \(\#\Omega=36\).
- Let \(A\) be the event where both dice show the same number: \[A=\{(1,1), (2,2),...,(6,6)\}\] here \(\# A=6\)
- The probability that both dice show the same number is \[P(A)=\frac{\# A}{\# \Omega}=\frac{6}{36}=\frac{1}{6}\]
Laplace or Classic interpretation of probability
The problem with this interpretation, is that we cannot use it, or it becomes meaningless, it when \(\Omega\) is uncountable or infinite. Also, what if the outcomes are not equally likely? (i.e. if the dice are not fair?)
Frequency interpretation
This is today still the dominant interpretation of probability.
In this case, what we want is to observe several independent repetitions of the experiment. After a while, some statistical regularity begins to emerge.
Frequency interpretation
Logically, if you run an experiment, and are interested in the probability of event \(A\), then the events you are registering are \(A\) and \(A^c\) or not \(A\).
Every time you run your experiment, you count when you get an \(A\) and when you observe an \(A^c\) event. Obviously, the total number of experiments is how many times you observed \(A\) and how many times you observed \(A^c\).
Example:
Experiment: Draw a random number in the interval \([0,1]\). \(A\) denotes \(x<0.4\).
| \(A\) | \(A^c\) | \(N\) | \(P(A)\) |
|---|---|---|---|
| 0 | 1 | 1 | 0 |
| 2 | 8 | 10 | 0.2 |
| 17 | 33 | 50 | 0.34 |
| 39 | 61 | 100 | 0.39 |
| 217 | 283 | 500 | 0.434 |
| 802 | 1198 | 2000 | 0.401 |
Example:
Frequency interpretation
As you can see, the more experiments we run, the more stabilized the ratio of occurrences for \(A\) over the total number of experiments. More generally:
\[P(A)=\lim_{N\rightarrow \infty}\frac{A\text{ occurrences}}{N \text{- Number of Experiments}}\]
This is the relative frequency of \(A\) in \(N\) experiments: \(f_A\)
Not always possible to repeat that many times the experiment in the same conditions.
About our previous example
It seems that the probability that the random number between 0 and 1 is below 0.4 is approximately 40%. The more experiments we run, the closer our relative frequency is to that number.
\[P(A)\underset{N \rightarrow \infty}{\rightarrow} 0.4\]
By the way, we will see later that theoretically, indeed \(P(A)=0.4\)
❓ The Concept of Probability · Question 1
You roll two fair dice. The probability that the sum equals 7 is:
A. \(\frac{1}{12}\)
B. \(\frac{1}{6}\)
C. \(\frac{7}{36}\)
D. \(\frac{1}{9}\)
✅ B. There are 6 favourable outcomes out of 36: \((1,6),(2,5),(3,4),(4,3),(5,2),(6,1)\), so \(P=\frac{6}{36}=\frac{1}{6}\).
❓ The Concept of Probability · Question 2
Under the frequency interpretation, \(P(A)\) is:
A. a subjective degree of belief
B. always \(\frac{\# A}{\#\Omega}\)
C. the limit of the relative frequency of \(A\) as the number of experiments grows
D. always \(0.5\)
✅ C. The frequency interpretation repeats the experiment many times and takes \(P(A)=\lim_{N\to\infty}\frac{\text{occurrences of }A}{N}\). The \(\frac{\# A}{\#\Omega}\) formula is the classical definition instead, and it needs equally likely outcomes.
✏️ The Concept of Probability · Question 3
Roll two fair dice. Let \(A\) be “both dice show the same number” and \(B\) be “the sum is at least 10”.
Compute \(P(A)\), \(P(B)\) and \(P(A\cap B)\).
✅ The Concept of Probability · Solution
\(\#\Omega=36\).
\(A=\{(1,1),...,(6,6)\}\), so \(P(A)=\frac{6}{36}=\frac{1}{6}\).
\(B=\{(4,6),(5,5),(6,4),(5,6),(6,5),(6,6)\}\), so \(P(B)=\frac{6}{36}=\frac{1}{6}\).
\(A\cap B=\{(5,5),(6,6)\}\), so \(P(A\cap B)=\frac{2}{36}=\frac{1}{18}\).
Practice
Practice
If there is time left, two quick ones. Difficulty goes 🟢 warm-up, 🟡 standard, 🟠 exam level, 🔴 stretch.
🟢 Practice · Exercise 1
Consider the following events:
A. “get a 7 when rolling a cubic die 🎲”
B. “Spain wins the next World Cup ⚽”
C. “rain in London ☔”
Which is correct?
- \(A\) is impossible, while \(C\) is unlikely
- \(B\) is certain, and \(C\) is very likely
- \(P(B)\) and \(P(C)\) are computed using the subjective notion of probability
- \(A\) and \(C\) are impossible
✅ 3. Nobody can repeat the next World Cup many times, nor list equally likely outcomes. What is left is a belief.
🟢 Practice · Exercise 2
When computing a probability from the analysis of all possible (equally likely) outcomes, we are using:
A. the frequentist definition of probability
B. the classic definition of probability
C. the subjective definition of probability
D. the axiomatic definition of probability
✅ B. This is Laplace: favourable cases over possible cases.
📝 Homework
Problem set 2.1, Questions 1 and 2, to check the ideas of today.