Random Pairs and Midterm 1 Review
Statistics I · Lecture 7
Today
- Random pairs: joint and marginal densities
- Covariance and correlation
- Midterm 1 review, from 🟢 to 🔴
Random Pairs
Random pair
When running an experiment, it could be interesting to study the relationship between two numeric features associated to each of the outcomes.
Discrete random pair
A random pair \((X,Y)\) is discrete when:
- The support \((X,Y)\), \((\Omega_X,\Omega_Y)\), is a finite or countable infinite of pairs.
- \(P\left(\Omega_X,\Omega_y\right)=1\)
Joint density function \(f_{X,Y}\)
Let \((X,Y)\) a discrete random pair, the joint density function \(f_{X,Y}(x,y)\) is a function \(f_{X,Y}:\mathbb{R}^2\rightarrow\mathbb{R}\) defined as:
\[ f_{X,Y}(x,y)=\left\{ \begin{array}{cl} P(X=x,Y=y) & , (x,y)\in(\Omega_X,\Omega_Y)\\ 0 & , (x,y)\in\mathbb{R}^2\setminus(\Omega_X,\Omega_Y) \end{array} \right. \]
Joint density function \(f_{X,Y}\)
\(f_{X,Y}\) satisfies the following properties:
- \(f_{X,Y}(x,y)\geq 0\forall(x,y)\in\mathbb{R}^2\)
- \(\sum_{x_i\in\Omega_X}\sum_{y_j\in\Omega_Y}P\left(X=x_i,Y=y_j\right)=1\) \(\forall i,j=1,2,...\)
A possible notation for \(P(X=x_i,Y=y_j)\) is \(p_{i,j}\)
Joint density function \(f_{X,Y}\)
| \(y_1\) | \(y_2\) | \(\dots\) | \(y_j\) | \(\dots\) | ||
|---|---|---|---|---|---|---|
| \(x_1\) | \(p_{11}\) | \(p_{12}\) | \(\dots\) | \(p_{1j}\) | \(\dots\) | \(\sum_{j=1}^\infty p_{1j}\) |
| \(x_2\) | \(p_{21}\) | \(p_{22}\) | \(\dots\) | \(p_{2j}\) | \(\dots\) | \(\sum_{j=1}^\infty p_{2j}\) |
| \(\vdots\) | \(\vdots\) | \(\vdots\) | \(\ddots\) | \(\vdots\) | \(\ddots\) | \(\vdots\) |
| \(x_i\) | \(p_{i1}\) | \(p_{i2}\) | \(\dots\) | \(p_{ij}\) | \(\dots\) | \(\sum_{j=1}^\infty p_{ij}\) |
| \(\vdots\) | \(\vdots\) | \(\vdots\) | \(\ddots\) | \(\vdots\) | \(\ddots\) | \(\vdots\) |
| \(\sum_{i=1}^\infty p_{i1}\) | \(\sum_{i=1}^\infty p_{i2}\) | \(\dots\) | \(\sum_{i=1}^\infty p_{ij}\) | \(\dots\) | 1 |
Marginal probability function
Given a random pair \((X,Y)\), the marginal probability function of \(X\) and \(Y\) is respectively:
- \(f_X(x_i) = P(X=x_i)=\) \[\sum_{j=1}^\infty P(X=x_i, Y=y_j) = \sum_{j=1}^\infty p_{ij}\]
- \(f_Y(y_j) = P(Y=x_j)=\) \[\sum_{i=1}^\infty P(X=x_i, Y=y_j) = \sum_{i=1}^\infty p_{ij}\]
For \(i=1,2,...\) and \(j=1,2,...\). Note that these functions have one dimension only.
Example
At SuperStore :convenience_store:, three trained employees are qualified to operate the checkout counters, restock products on the shelves, and perform some administrative tasks. SuperStore has three checkout counters, and at least one of them must always be operating.
At any given day and moment when SuperStore is open to customers, consider the following random variables:
- \(X\) N of employees in the checkout counters :shopping_cart: :credit_card: .
- \(Y\) N of employees restocking products on the shelves :package:.
Example
The r.v. \(X\) has \(\Omega_X=\{1,2,3\}\) and the following pdf:
| \(x\) | 1 | 2 | 3 |
|---|---|---|---|
| \(f_X(x)\) | 0.17 | 0.8 | 0.03 |
Consider the following table for the joint probability of \((X,Y)\)
| \(X\setminus Y\) | 0 | 1 | 2 | |
|---|---|---|---|---|
| 1 | \(a\) | \(2b\) | \(b\) | |
| 2 | 0.1 | \(c\) | 0 | |
| 3 | 0.03 | 0 | 0 | |
| 1 |
Example
If \(P(X=1, Y=0)=0.02\), find \(b\) and \(c\)
- Directly we get \(a=0.02\) as it is \(P(X=1, Y=0)\)
- Note that the top row is \(f_Y(y)\) and the left column is \(f_X(x)\)
- Fill directly \(f_X(x)\) with \(0.17\), \(0.8\), and \(0.03\).
- Fill \(f_Y(y)\) with the summation of each column
Example
| \(X\setminus Y\) | 0 | 1 | 2 | \(\color{red}{f_X(x)}\) |
|---|---|---|---|---|
| 1 | \(\color{red}{0.02}\) | \(2b\) | \(b\) | \(\color{red}{0.17}\) |
| 2 | 0.1 | \(c\) | 0 | \(\color{red}{0.8}\) |
| 3 | 0.03 | 0 | 0 | \(\color{red}{0.03}\) |
| \(\color{red}{f_Y(y)}\) | \(\color{red}{0.15}\) | \(\color{red}{2b+c}\) | \(\color{red}{b}\) | 1 |
- From first row: \(0.02 + 2b + b = 0.17\) \(\Rightarrow\) \(b=\frac{0.15}{3}=0.05\)
- From second row: \(0.1 + c = 0.8\) \(\Rightarrow\) \(c=0.7\)
Example
\(P(X=2|Y\geq 1)\) is approximately…?
- \[P(X=2|Y\geq 1)=\frac{P(X=2, Y\geq 1)}{P(Y\geq 1)}\]
- \[= \frac{P(X=2, Y=1) + P(X=2, Y=2)}{P(Y=1)+P(Y=2)}\]
- \[\frac{0.7+0}{0.8+0.05}=\frac{0.7}{0.85}\approx 0.8235\]
Independence of random variables (revisited)
Let \((X,Y)\) a discrete random pair, \(X,Y\) are independent if, and only if: \[P(X=x, Y=y)=P(X=x)P(Y=y)\quad \forall(x,y)\in\mathbb{R}^2\]
Equivalently: the joint pdf is the product of the marginal pdfs, and the joint cdf is the product of the marginal cdfs.
Example
… (continued exercise) Are \(X\) and \(Y\) independent?
- \(P(X=2, Y=2)=0\)
- \(P(X=2)P(Y=2)= 0.8 \times 0.05=0.04\)
- \(0\neq 0.04\)
- \(X\) and \(Y\) are not independent.
Moments of a random pair
If \(g(x,y)=xy\), then \(E[g(x,y)]=E[XY]\) and that equals \[\sum_{i=1}^\infty\sum_{j=1}^\infty x_iy_jP(x=x_i,Y=y_j)\]
Moments of a random pair
Note that this is equivalent to \(cov(X,Y)=E[XY]-E[X]E[Y]\)
Properties of the covariance
The covariance tries to capture how the two r.v. move together. If it is positive, it means that both tend to go in the same direction more often than not (both above or below their means at the same time). Being negative means that more often than not when one is above its mean, the other is below.
- \(cov(X,Y)=cov(Y,X)\)
- \(cov(X,X)=V[X]\) and \(cov(Y,Y)=V[Y]\) if \(V[X]\) and \(V[Y]\) exist.
- \(cov(a+bX, c+dY)=bd cov(X,Y)\) with \(a,b,c,e\in\mathbb{R}\)
Properties of the covariance
If \(X\) and \(Y\) are independent r.v. then \(cov(X,Y)=0\). Note that the opposite is not necessarily true, i.e. \(cov(X,Y)=0\) does not imply that \(X\) and \(Y\) are independent.
Another important identity with the covariance is the following:
\[V[X\pm Y] = V[X]+V[Y]\pm 2 cov(X,Y)\]
Example
Knowing that \(E[Y]=0.9\), \(cov(X,Y)\) is equal to? …From the first table: \(E[X]=0.17\times 1 + 0.8 \times 2 + 0.03 \times 3 = 1.86\)
\[E[XY]=\sum_{x}\sum_{y}xyP(X=x,Y=y)=\] \[= 1 \times 0 \times 0.02 + 1\times 1 \times 0.1 + 1\times 2 \times 0.05 +\] \[+ 3\times 0 \times 0.1 + 2 \times 1 \times 0.7 + 2\times 2 \times 0 + \] \[+ 3 \times 0 \times 0.03 + 3 \times 1 \times 0 + 3 \times 2 \times 0 = 1.6\]
\(cov(X,Y)=E[XY]-E[X]E[Y]=1.6-1.86\times 0.9=-0.074\)
Correlation coefficient
A caveat of the covariance is that its units depends directly on the units of \(X\) and \(Y\). The correlation coefficient allow us to express this relationship, between \(X\) and \(Y\) without being affected by the units in which these r.v. are measured.
\[\rho_{XY} = \frac{cov(X,Y)}{\sqrt{V[X]V[Y]}}=\frac{cov(X,Y)}{\sigma_X\sigma_Y}\]
Clearly \(\rho\in[-1,1]\). Note also that \(|\rho|=1\) if and only if \(P(Y=a+bX)=1\) with \(a,b\in\mathbb{R}\). If \(X\) and \(Y\) are independent r.v. then \(\rho=0\).
Correlation coefficient
| Correlation coefficient | Correlation |
|---|---|
| \(|\rho| = 1\) | Perfect |
| \(0.8 \leq |\rho| < 1\) | Strong |
| \(0.5 \leq |\rho| < 0.8\) | Moderate |
| \(0.1 \leq |\rho| < 0.5\) | Weak |
| \(0 < |\rho| < 0.1\) | Very weak |
| \(\rho= 0\) | None |
Write positive or negative in front of correlation if \(\rho>0\) or \(\rho<0\) respectively.
What does \(\rho\) actually look like?
Same 160 observations every time. Only \(\rho\) changes. Match what you see against the table on the previous slide.
What the chart showed
At \(\rho=0\) the cloud is a shapeless blob; as \(|\rho|\) grows it tightens onto a line, and at \(|\rho|=1\) every point sits exactly on it. The sign sets the tilt, the magnitude sets the tightness.
Because both variables here have \(\sigma=1\), covariance and correlation coincide. Rescale \(X\) into cents and the covariance changes by a factor of 100 while \(\rho\) does not move: that is exactly the unit problem \(\rho\) was built to solve.
\(\rho=0\) means no linear relationship, not “no relationship”. Points on a perfect parabola \(Y=X^2\) have \(\rho=0\) and are entirely dependent.
Example
- Based on the previous question, find the correlation coefficient between \(X\) and \(Y\).
From the marginal probability function we obtain \(V[X]\) and \(V[Y]\): \[V[X]=0.1804\text{ and }V[Y]=0.19\]
Therefore, \[\rho = \frac{cov(X,Y)}{\sigma_X\sigma_Y}=\frac{-0.074}{\sqrt{0.1804}\sqrt{0.19}}=-0.3997\]
We observe a weak negative linear correlation between \(X\) and \(Y\).
❓ Random Pairs · Question 1
If \(X\) and \(Y\) are independent, then:
A. \(cov(X,Y)=1\)
B. \(V[X+Y]=V[X]-V[Y]\)
C. \(\rho_{XY}=1\)
D. \(cov(X,Y)=0\)
✅ D. Independence implies \(E[XY]=E[X]E[Y]\), so the covariance vanishes. Note also \(V[X+Y]=V[X]+V[Y]\) in that case.
❓ Random Pairs · Question 2
“If \(cov(X,Y)=0\) then \(X\) and \(Y\) are independent.” This statement is:
A. always true
B. false, zero covariance does not imply independence
C. true only for discrete r.v.
D. true only when \(\rho=1\)
✅ B. Covariance only captures the linear association. Independence is strictly stronger, so the implication runs one way only.
✏️ Random Pairs · Question 3
The pair \((X,Y)\) has \(f_{X,Y}(0,0)=0.3\), \(f_{X,Y}(0,1)=0.2\), \(f_{X,Y}(1,0)=0.2\), \(f_{X,Y}(1,1)=0.3\).
Find the marginals, \(cov(X,Y)\) and \(\rho_{XY}\). Are \(X\) and \(Y\) independent?
✅ Random Pairs · Solution
Marginals: \(P(X=0)=P(X=1)=0.5\) and \(P(Y=0)=P(Y=1)=0.5\), so \(E[X]=E[Y]=0.5\) and \(V[X]=V[Y]=0.5-0.25=0.25\).
\(E[XY]=1\times 1\times 0.3=0.3\), so \(cov(X,Y)=0.3-0.25=0.05\) and \(\rho_{XY}=\frac{0.05}{0.5\times 0.5}=0.2\).
Not independent: \(f_{X,Y}(0,0)=0.3\neq P(X=0)P(Y=0)=0.25\).
Midterm 1 Review
Midterm 1 Review
Next session is the midterm: Topics 1 and 2, 80 minutes. Today we practice Topic 2, from easy to hard:
- 🟢 warm-up: one formula, one step
- 🟡 standard: two or three steps
- 🟠 exam level: this is what you will find in the midterm
- 🔴 stretch: you need to combine several things
For Topic 1, go back to the workshop of Lecture 3 and to problem set 1.
🟢 Review · Exercise 1
The number of work absences, in days, of the workers of a company is a r.v. with pdf:
| \(x\) | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|
| \(f(x)\) | \(a\) | 0.15 | 0.3 | 0.10 | \(b\) |
60% of the workers did not skip more than 3 days.
(a) Find \(a\) and \(b\).
(b) Find the probability that a worker skipped 5 days, given that it skipped more than 3.
(c) Find \(E[X]\). After a strike, every worker skips 1 more day. What happens to \(E[X]\)?
✅ Review · Exercise 1 · Solution
(a) \(F(3)=a+0.15+0.3=0.6\), so \(a=0.15\). The pdf adds to one: \(0.15+0.15+0.3+0.1+b=1\), so \(b=0.3\)
(b) \(P(X=5|X>3)=\frac{P(X=5)}{P(X>3)}=\frac{0.3}{1-0.6}=0.75\)
(c) \(E[X]=1(0.15)+2(0.15)+3(0.3)+4(0.1)+5(0.3)=3.25\)
After the strike the r.v. is \(X+1\), and \(E[X+1]=E[X]+1=4.25\). It increases.
🟡 Review · Exercise 2
A financial institution offers two assets, \(A\) and \(B\). 10% of the clients invest in \(A\) and the rest in \(B\). Of those who invest in \(A\), 70% get above market returns 📈. Of those who invest in \(B\), only 55%.
Pick a client randomly. True or false:
(a) The probability of getting a return above the market is 0.565.
(b) Knowing that a client got an above market return, the probability that the client invested in \(B\) is 0.495.
(c) “Investing in \(A\)” and “getting a return below the market” are independent.
✅ Review · Exercise 2 · Solution
Let \(M\) be “above the market”. \(P(A)=0.1\), \(P(M|A)=0.7\), \(P(M|B)=0.55\).
(a) ✅ \(P(M)=0.1\times 0.7+0.9\times 0.55=0.07+0.495=0.565\)
(b) ❌ \(0.495\) is \(P(M\cap B)\). The conditional is \(P(B|M)=\frac{0.495}{0.565}\approx 0.876\)
(c) ❌ \(P(M^c|A)=0.3\) but \(P(M^c)=0.435\). Knowing the client is in \(A\) changes the probability, so they are not independent.
🟡 Review · Exercise 3
In a Management class, 70% of the students are male. For 60% of the males this course was their first choice, and for 75% of the females.
Pick a student randomly, and let \(p\) be the probability that this course was the student’s first choice.
(a) Find \(p\).
(b) Find the probability of being female, knowing that the course was the first choice.
(c) Find the probability of being male, or of having chosen the course as first choice.
(d) Are “male” and “not first choice” independent?
✅ Review · Exercise 3 · Solution
(a) \(p=0.7\times 0.6+0.3\times 0.75=0.42+0.225=0.645\)
(b) \(P(F|FC)=\frac{0.3\times 0.75}{p}=\frac{0.225}{0.645}\approx 0.349\)
(c) \(P(M\cup FC)=P(M)+P(FC)-P(M\cap FC)=0.7+p-0.42=0.28+p=0.925\)
(d) No. \(P(FC^c|M)=0.4\) while \(P(FC^c)=0.355\).
🟠 Review · Exercise 4
Consider the random pair \((X,Y)\) with:
- \(\Omega_X=\{0,1\}\) and \(\Omega_Y=\{-1,1\}\)
- \(P(X=0)=0.5\) and \(P(Y=1)=0.6\)
- \(P(X=1,Y=1)=p\), with \(0<p<1\)
(a) Find \(p\) such that \(E[XY]=0.1\).
(b) Let \(p=0.4\). Find \(P(X+Y=0)\).
(c) Let \(p=0.4\). Find \(Cov(X,Y)\) and \(V[2Y-X]\).
✅ Review · Exercise 4 · Solution
(a) \(XY\neq 0\) only when \(X=1\). \(P(X=1,Y=-1)=0.5-p\), so \(E[XY]=p-(0.5-p)=2p-0.5=0.1\Rightarrow p=0.3\)
(b) \(X+Y=0\) only if \(X=1\) and \(Y=-1\): \(P(X=1,Y=-1)=0.5-0.4=0.1\)
(c) \(E[XY]=2(0.4)-0.5=0.3\), \(E[X]=0.5\), \(E[Y]=0.6-0.4=0.2\), so \(Cov(X,Y)=0.3-0.1=0.2\)
\(V[X]=0.25\) and \(V[Y]=1-0.2^2=0.96\):
\[V[2Y-X]=4V[Y]+V[X]-4Cov(X,Y)=3.84+0.25-0.8=3.29\]
🔴 Review · Exercise 5
At SuperStore 🏪, \(X\) is the number of employees at the checkout counters 🛒 and \(Y\) the number restocking shelves 📦. The joint probability is:
| \(X\setminus Y\) | 0 | 1 | 2 |
|---|---|---|---|
| 1 | \(a\) | \(2b\) | \(b\) |
| 2 | 0.1 | \(c\) | 0 |
| 3 | 0.03 | 0 | 0 |
You know that \(f_X(1)=0.17\), \(f_X(2)=0.8\) and \(P(X=1|Y=0)=0.1333\).
(a) Find \(a\), \(b\) and \(c\).
(b) Find \(P(X=2|Y\geq 1)\). Are \(X\) and \(Y\) independent?
(c) Find \(Cov(X,Y)\).
✅ Review · Exercise 5 · Solution
(a) \(0.1+c=0.8\Rightarrow c=0.7\). \(P(X=1|Y=0)=\frac{a}{a+0.13}=0.1333\Rightarrow a\approx 0.02\). Then \(a+3b=0.17\Rightarrow b=0.05\)
(b) \(P(Y\geq 1)=0.1+0.05+0.7=0.85\), so \(P(X=2|Y\geq 1)=\frac{0.7}{0.85}\approx 0.824\)
Not independent: \(f_{X,Y}(3,1)=0\), but \(f_X(3)f_Y(1)=0.03\times 0.8\neq 0\)
(c) \(E[X]=0.17+1.6+0.09=1.86\), \(E[Y]=0.8+2(0.05)=0.9\), \(E[XY]=1(0.1)+2(0.05)+2(0.7)=1.6\)
\[Cov(X,Y)=1.6-1.86\times 0.9=-0.074\]
📝 Homework
Problem set 2.1, Questions 8 to 10, and problem set 2.2, Questions 10 to 13.
Good luck in the midterm 🍀
Bibliography
- Figueiredo, F., Figueiredo, A., Ramos, A. & Teles, R. (2009). Estatística Descritiva e Probabilidades (2a Edição). Escolar Editora.
- Murteira, B., Ribeiro, C.R., Silva, J.R. & Pimenta, C. (2007). Introdução à Estatística (2a Edição). McGraw-Hill.
- Pestana, D. & Velosa, S.F. (2008). Introdução à Probabilidade e à Estatística (3a Edição). Fundação Calouste Gulbenkian.
- Paulino, C.D. & Branco, J.A. (2005). Exercícios de Probabilidade e Estatística. Escolar Editora.