The concept

Consider the following scenario:

The concept

Here we can define probability in terms of frequency of occurrence, i.e. as a percentage of successes in a moderately large number of similar situations.

This is the most natural and traditional way of thinking about probability.

The concept

There might be situations where the frequency concept is not adequate, because it might refer to a one-time event. These are subjective beliefs.

• A company is recruiting a new CEO, and a board member says:

  “I believe there’s a 90% chance that our chosen candidate will be an effective CEO.”

The concept

It might seem easy to disregard the second case as unscientific or useless. However, many times people need to make decisions under uncertainty with not enough data (or no data at all!) about previous realizations of the specific event.

Beliefs allow the decision maker to, well, make some decision, at least consistently.

• What’s the difference between both situations?
• What do they have in common?

Sets and elements

A set is a collection of objects, which are elements of the set.

How to write down a Set

There are several ways to specify a set.

By extension, or as a list:

• If a set \(S\) has a finite number of elements (\(x_i\in S\)) we can write it like this: \[S=\{x_1, x_2, ..., x_n\}\]

• If a set \(S\) has an infinite (but countable) elements (\(x_i\in S\)) we can write it like: \[S=\{x_1, x_2, ...\}\]

How to write down a Set

By describing the property (\(P\)) that \(x\) must satisfy to be included in \(S\): \[S= \{x|x \text{ satisfies } P\}\] in this case \(|\) reads as such that. For example \[S=\{x\in\mathbb{R}\mid x\geq 0\}\] to describe the non-negative real numbers.

This example is special, as the positive real numbers cannot be written down as a a list. In this case the interval \([0,\infty)\) is an uncountable set.

More definitions

More definitions

The universal set is important because it defines the scope of our analysis. Say we are studying the performance of students of Statistics I in 2026.

The cars parked outside our institution do not belong to the universal set, because they are not relevant for our purpose. Only students of Statistics I in 2026 belong to the universal set.

Set Operations

Set Operations

Note that \(\vee\) stands for or, and \(\wedge\) stands for and.

Set Operations

Sometimes we might need to consider the union or intersection of many sets, and for that we can use a notation similar to the one we used for summations:

• \[\bigcup_{n=1}^\infty S_n = S_1 \cup S_2 \cup ... = \{x\in\Omega | x \in S_n \text{ for some } n\}\]

• \[\bigcap_{n=1}^\infty S_n = S_1 \cap S_2 \cap ... = \{x\in\Omega | x \in S_n \text{ for every } n\}\]

Set Operations

Set Definition

Set definition

❓ Set Theory · Question 1

Two sets \(S\) and \(T\) are disjoint when:

A. \(S\cup T=\emptyset\)

B. \(S=T\)

C. \(S\cap T=\emptyset\)

D. \(S\subseteq T\)

❓ Set Theory · Question 2

The set \(S\setminus T\) contains exactly the elements that:

A. belong to both \(S\) and \(T\)

B. belong to \(T\) but not to \(S\)

C. belong to neither

D. belong to \(S\) but not to \(T\)

✏️ Set Theory · Question 3

Let \(\Omega=\{1,2,3,4,5,6\}\), \(A=\{1\}\), \(B=\{3,6\}\) and \(C=\{2,4,6\}\).

Write down \(A\cup B\), \(B\cap C\), \((B\cup C)^c\) and \(C\setminus B\).

Back to Probability

The sample space (\(\Omega\)) can be:

• Discrete, when \(\#\Omega\) is finite, or countable infinite.
• Continuous, when \(\#\Omega\) is uncountable.

Back to Probability

Consider the experiment of throwing a die 🎲 and noting the number shown on side facing upwards.

1. The sample space is \(\Omega=\{1,2,3,4,5,6\}\)
2. In this case \(\# \Omega = 6\)
3. \(\Omega\) is discrete.

Back to Probability

Consider now the random experiment of measuring the life expectancy of a lamp 💡, measured in hours.

1. The sample space is \(\Omega=\{x\in\mathbb{R}|x\geq 0\}\)
2. In this case, \(\Omega\) is all non-negative real numbers.
3. \(\Omega\) is continuous.

Back to Probability

Example

Let’s go back to our experiment with the 🎲

The sample space is: \(\Omega=\{1,2,3,4,5,6\}\)

Within this space, we can define the following events:

1. \(A=\{1,3,5\}\), i.e. the number is odd.
2. \(B=\{3,4,5,6\}\), i.e. the number is at least 3.
3. \(C=\{1,2,3\}\) , i.e. the number is lower than 4.
4. \(D=\{6\}\), i.e. the number is larger than 5.

Example

Now let’s revisit the example of our 💡

The sample space is: \(\Omega=\{x\in\mathbb{R}|x\geq 0\}\)

In this space, we can define the following events:

1. \(A=\{x\in\mathbb{R}|75<x<95\}\), i.e. the 💡 lasts between 75 and 95 hours.
2. \(B=\{x\in\mathbb{R}|x\leq 100\}\), i.e. the 💡 lasts no longer than 100 hours.
3. \(C=\{x\in\mathbb{R}|x\geq 60\}\), i.e. the 💡 lasts at least 60 hours.

Events

Mixing up Sets and Probability

Consider two events \(A\) and \(B\) both subsets of \(\Omega\)

1. \(A^c\) contains all the outcomes that are not in \(A\). \(A^c\) is the event of not \(A\).

2. If \(A\subseteq B\), then an outcome that realizes event \(A\) (\(\omega\in A\)), also realizes \(B\), as \(A\subseteq B\Rightarrow \omega\in B\) as well. \(A\Rightarrow B\)

3. For \(A\cup B\) to happen, we need \(\omega \in A\) or \(\omega \in B\), which means that \(A\) happens, or \(B\) happens, or both happen simultaneously.

Mixing up Sets and Probability

4. For \(A\cap B\) to happen, we need \(\omega \in A\) and \(\omega \in B\), which means that \(A\) and \(B\) happen simultaneously.

5. \(A\) and \(B\) are incompatible if \(A\cap B=\emptyset\), i.e. if an outcome is in one set, it cannot be in another, for example it cannot be that \(A\) and \(A^c\) happen simultaneously!

Inherited set properties for events

Consider two events \(A\) and \(B\) both subsets of \(\Omega\)

1. Commutativity: \(A\cup B = B\cup A\); \(A\cap B=B\cap A\)
2. Associativity: \((A\cup B)\cup C=A\cup(B\cup C)\); \((A\cap B)\cap C=A\cap(B\cap C)\)
3. Distributivity: \(A\cup(B\cap C)=(A\cup B)\cap(A\cup C)\); \(A\cap (B\cup C)=(A\cap B)\cup(A\cap C)\)
4. De Morgan’s Laws: \((A\cap B)^c=A^c\cup B^c\); \((A\cup B)^c=A^c\cap B^c\)

Inherited set properties for events

5. \(\left(A^c\right)^c=A\)
6. Complement law: \(A\cup A^c=\Omega\); \(A\cap A^c=\emptyset\)
7. Identity element: \(A\cup\emptyset = A\); \(A\cap\Omega = A\)
8. Absorbing element: \(A\cup\Omega = \Omega\); \(A\cap\emptyset = \emptyset\)
9. Idempotent law: \(A\cup A=A\) ; \(A\cap A=A\)
10. \(A\subset B\Rightarrow A\cap B=A\); \(A\subset B\Rightarrow A\cup B = B\)

Example

Consider the sample space \(\Omega =\{1,2,3,4,5,6\}\), from our 🎲 case.

Define the events: \[A=\{1\},\ B=\{3,6\},\ C=\{2,4,6\},\ D=\{4,5,6\}\]

Example

Let’s define the following events in \(\Omega\)

Concept of probability

Besides the concepts we already saw of frequency and subjectivity for probability, there was an older, called “classic” one. This one was introduced by Pierre-Simon Laplace in 1812.

Example

Consider now an experiment throwing two dice 🎲 🎲

1. How many outcomes are in \(\Omega\)? \(6^2=36\), \(\#\Omega=36\).
2. Let \(A\) be the event where both dice show the same number: \[A=\{(1,1), (2,2),...,(6,6)\}\] here \(\# A=6\)
3. The probability that both dice show the same number is \[P(A)=\frac{\# A}{\# \Omega}=\frac{6}{36}=\frac{1}{6}\]

Laplace or Classic interpretation of probability

The problem with this interpretation, is that we cannot use it, or it becomes meaningless, it when \(\Omega\) is uncountable or infinite. Also, what if the outcomes are not equally likely? (i.e. if the dice are not fair?)

Frequency interpretation

This is today still the dominant interpretation of probability.

In this case, what we want is to observe several independent repetitions of the experiment. After a while, some statistical regularity begins to emerge.

Frequency interpretation

Logically, if you run an experiment, and are interested in the probability of event \(A\), then the events you are registering are \(A\) and \(A^c\) or not \(A\).

Every time you run your experiment, you count when you get an \(A\) and when you observe an \(A^c\) event. Obviously, the total number of experiments is how many times you observed \(A\) and how many times you observed \(A^c\).

Example:

Experiment: Draw a random number in the interval \([0,1]\). \(A\) denotes \(x<0.4\).

Example:

Frequency interpretation

As you can see, the more experiments we run, the more stabilized the ratio of occurrences for \(A\) over the total number of experiments. More generally:

\[P(A)=\lim_{N\rightarrow \infty}\frac{A\text{ occurrences}}{N \text{- Number of Experiments}}\]

This is the relative frequency of \(A\) in \(N\) experiments: \(f_A\)

Not always possible to repeat that many times the experiment in the same conditions.

About our previous example

It seems that the probability that the random number between 0 and 1 is below 0.4 is approximately 40%. The more experiments we run, the closer our relative frequency is to that number.

\[P(A)\underset{N \rightarrow \infty}{\rightarrow} 0.4\]

By the way, we will see later that theoretically, indeed \(P(A)=0.4\)

❓ The Concept of Probability · Question 1

You roll two fair dice. The probability that the sum equals 7 is:

A. \(\frac{1}{12}\)

B. \(\frac{1}{6}\)

C. \(\frac{7}{36}\)

D. \(\frac{1}{9}\)

✏️ The Concept of Probability · Question 3

Roll two fair dice. Let \(A\) be “both dice show the same number” and \(B\) be “the sum is at least 10”.

Compute \(P(A)\), \(P(B)\) and \(P(A\cap B)\).

Practice

If there is time left, two quick ones. Difficulty goes 🟢 warm-up, 🟡 standard, 🟠 exam level, 🔴 stretch.

🟢 Practice · Exercise 2

When computing a probability from the analysis of all possible (equally likely) outcomes, we are using:

A. the frequentist definition of probability

B. the classic definition of probability

C. the subjective definition of probability

D. the axiomatic definition of probability

📝 Homework

Problem set 2.1, Questions 1 and 2, to check the ideas of today.