Random Variables

A random variable is a function that will allow us to quantify (transform into a number) each outcome.

Summarizing, a r.v. is a function that associates a real number to each outcome from \(\Omega\).

Random Variables

• \(\{X=x\}\), \(\{X\leq x\}\), \(\{X>x\}\) are events
• \(\{X=x\}\) happens when, from our experiment, we obtain \(\omega\) such that \(X(\omega)=x\). \(\{X=x\}=\{\omega\in\Omega | X(\omega)=x\}\)
• The probability of \(X\leq x\) for example, is then \[P(X\leq x)=P\left(\{\omega\in \Omega| X(\omega)\leq x\}\right)\]

Discrete Random Variable

\(X\) is a discrete r.v. when:

• The support of \(X\), \(\Omega_X\), is finite or countable infinite.
• \(P(\Omega_X)=1\)

Probability density function (pdf)

Let \(X\) be a discrete r.v. The pdf of \(X\) is a function \(f_X:\mathbb{R}\rightarrow\mathbb{R}\) such that:

\[f_X(x)=\left\{\begin{array}{cc}P(X=x) & ,\text{ if } x\in\Omega_X\\ 0 & ,\text{ if } x\in\mathbb{R}\setminus\Omega_X\end{array}\right.\]

Naturally, by construction the pdf satisfies the following properties:

Probability density function (pdf)

The pdf gives the probability in a single point. The total probability is distributed among single points, \(x_i\). A reasonable representation of a pdf of a discrete r.v. could be:

Where \(p_i=P(X=x_i)\)

Example

Consider the discrete r.v. \(X\) with the following pdf:

We could define:

• Support of \(X\): \(\Omega_X=\{0,1,2,3,4\}\)
• \(\sum_{x\in\Omega_X} f(x)=1 \Rightarrow 0.05 + a + 0.35 + 0.25 + 0.05 = 1\) that is \(a=\)
• 0.3

Example

Our table is now:

What is \(P(X=2|X\leq 3)\)?

Continuous random variables

\(X\) is a continuous r.v. if:

• The support of \(X\), \(\Omega_X\) is uncountable infinite.
• \(P(\Omega_X)=1\)
• \(P(X=x)=0\) \(\forall x\in\mathbb{R}\).

Probability density function (pdf)

Let \(X\) a continuous r.v.

There is a function \(f_X:\mathbb{R}\rightarrow\mathbb{R}\), the pdf of \(X\) such that:

• \(f_X(x)\geq 0\), \(\forall x\in\mathbb{R}\)
• \(\int_{-\infty}^{\infty}f_X(x)dx=1\)

Technically, from Measure Theory, we need an absolutely continuous r.v. to ensure the existence of a pdf. These issues are beyond the scope of this course. Just know that when we say continuous r.v. we mean absolutely continuous r.v.

Probability density function (pdf)

Note that this pdf allows to compute the probability of events \(x\in(a,b]\):

\[P(a<X\leq b)=\int_a^b f_X(x)dx\]

Observe that if you would do \(X=a\) you would get the integral from \(a\) to \(a\), which makes \(dx=0\) and therefore the integral (and the probability) becomes 0.

Example

Let \(X\) be a continuous r.v. with the following pdf:

\[ f(x)=\left\{\begin{array}{cc} \theta x^2 & , 0\leq x< 1\\ 0 & ,\mathbb{R}\setminus [0,1) \end{array}\right. \]

• Support for \(X\): \(\Omega_X=[0,1)\)

• \[\int_{-\infty}^{\infty}f(x)dx=1\Leftrightarrow\int_0^1\theta x^2dx=\left[\theta\frac{x^3}{3}\right]_{0}^1\]

• \[\theta\frac{1}{3}-\theta\frac{0}{3}=1\Leftrightarrow \theta=3\]

Cumulative distribution function (cdf)

Let \(X\) be a r.v. The distribution function \(F_X:\mathbb{R}\rightarrow[0,1]\), defined as:

\[F_X(x)=P(X\leq x)\]

\(F_X\) is unique.

Cumulative distribution function (cdf)

With a discrete r.v.

\[F(x)=P(X\leq x)=\left\{ \begin{array}{cc} 0 & x<0 \\ 0.05 & 0\leq x < 1 \\ 0.05 + 0.3 = 0.35 & 1 \leq x < 2 \\ 0.35 + 0.35 = .7 & 2 \leq x < 3 \\ 0.7 + 0.25 = .95 & 3 \leq x < 4 \\ 1 & x\geq 4 \end{array} \right.\]

Cumulative distribution function (cdf)

Let’s revisit our previous example:

\[P(X=2|X\leq 3)= \frac{P(X=2)}{P(X\leq 3)}=\] \[\frac{F(2)-F(2^-)}{F(3)}= \frac{0.7-0.35}{0.95}=0.368 \]

Cumulative distribution function (cdf)

With a continuous r.v.:

\[F_X(x)=P(X\leq x)=\int_{-\infty}^{x} f_X(x)dx\]

The distribution function, \(F_X\) allows to compute the probability of \(\{X\in(a,b]\}\)

\[P(a<X\leq b)=\int_a^b f_X(x)dx=F_X(b)-F_X(a)\]

Example

Example

Consider the continuous r.v. defined previously, with the pdf:

\[ f(x)=\left\{ \begin{array}{cc} 3x^2 & , 0\leq x< 1\\ 0 & , \mathbb{R}\setminus [0,1) \end{array} \right. \]

Support for \(X\): \(\Omega_X=[0,1)\)

Distribution function (cdf):

\[ F(x)=P(X\leq x) = \int_{-\infty}^x f(t)dt = \left\{ \begin{array}{cc} 0 & , x<0\\ x^3 & ,0\leq x<1 \\ 1& ,x\geq 1 \end{array} \right. \]

Properties of the cdf

Nonetheless the r.v. is discrete or continuous, \(F_X\) has the following properties:

• \(F_X:\mathbb{R}\rightarrow[0,1]\)
• \(F_X\) is continuous from the right: \(\lim_{x\rightarrow a^+}F_X(x)=F(a)\)
• \(F_X\) is monotone non-decreasing.
• \(\lim_{x\rightarrow -\infty} F_X(x)=0\)
• \(\lim_{x\rightarrow \infty} F_X(x)=1\)

Properties of the cdf

\(F_X\) for \(X\) r.v. discrete

• Single points, discontinuous support \(\Omega_X\)
• It is continuous from the right
• \(P(X<x)=F(X^-)\)
• \(P(X=x)=F(x)-F(x^-)\)
• \(P(a<X\leq b)=F(b)-F(a)\)
• \(P(a\leq X\leq b)=F(b)-F(a^-)\)
• \(P(a<X<b)=F(b^-)-F(a^-)\)
• \(P(a\leq X < b) = F(b^-)-F(a^-)\)

Properties of the cdf

\(F_X\) for \(X\) r.v. continuous

• Continuous support \(\Omega_X\)
• It is continuous in \(\mathbb{R}\)
• \(P(X<x)=P(X\leq x)=F(x)\)
• \(P(a\square X\square b)=F(b)-F(a)\), [replace \(\square\) for \(<\) or \(\leq\)]

pdf and cdf

For a discrete r.v.

\[P(X=x)=F_X(x)-F_X(x^-)\] \[+\downarrow \uparrow -\] \[F_X(x)=\sum_{x_i\leq x}P(X=x_i)\]

pdf and cdf

For a continuous r.v.

\[ f_X(x)=\left\{ \begin{array}{cc} F_X'(x) & ,x\in\mathbb{R} \text{ if }F_X'\text{ exists} \\ 0 & \text{, otherwise} \end{array} \right. \]

\[ Derivative \downarrow \uparrow Primitive\]

\[ F_X(x)=\int_{-\infty}^x f_X(t)dt\]

The pdf of \(X\), a continuous r.v. is not unique.

❓ Discrete and Continuous r.v. · Question 1

If \(X\) is a continuous random variable, then \(P(X=a)\) equals:

A. \(f_X(a)\)

B. \(F_X(a)\)

C. \(0\)

D. \(1\)

❓ Discrete and Continuous r.v. · Question 2

The cumulative distribution function \(F_X\) is:

A. decreasing

B. always continuous

C. non decreasing, going from 0 to 1

D. the same as the pdf

✏️ Discrete and Continuous r.v. · Question 3

Let \(X\) be continuous with \(f(x)=2x\) for \(0\leq x<1\) and \(0\) otherwise.

Check that this is a valid pdf, find \(F_X(x)\), then compute \(P(X\leq 0.5)\) and \(P(0.2<X\leq 0.8)\).

Describing a population

We could describe the range of \(X\), a r.v., as a population, in the statistical sense, because it describes all the possible values it can take.

We can use numerical values to do so, which can represent dispersion or centrality of the data.

Expected Value or mean

The expected value or mean, is a location parameter for our r.v.

Example discrete

Let \(X\) be a discrete r.v. as in the previous example:

Let \(g(X)=2(X-1)^2+3(X-1)-5\), find \(E[g(X)]\).

Example discrete

\[g(X)=2(X-1)^2+3(X-1)-5\] \[=2(X^2-2X+1)+3X-3-5\] \[=2X^2-4X+2+3X-8\] \[=2X^2-X-6\]

\[E[Y]=E[2X^2-X-6]\]

\[=2E[X^2]-E[X]-6\]

Example discrete

We only need to find \(E[X]\) and \(E[X^2]\) to obtain \(E[g(X)]\).

\[E[X]=\sum xP(X=x)\] \[ = 0\times .05 + 1 \times .3 + 2 \times .35 + 3 \times .25 + 4\times .05 = 1.95\]

\[E[X^2]=\sum x^2 P(X=x)\]

\[ = 0\times .05 + 1 \times .3 + 4 \times .35 + 9 \times .25 + 16\times .05 = 4.75\]

\[E[g(X)]=2\times 4.75 - 1.95 - 6 = 1.55\]

Example continuous

Recall our example for continuous r.v.s. \(X\): \[ f_X(x)=\left\{ \begin{array}{cc} 3x^2 & ,0\leq x < 1 \\ 0 & , \mathbb{R}\setminus[0,1) \end{array} \right. \]

Find \(E[g(X)]\) when \(g(X)=2(X-1)^2+3(X-1)-5\) We know already \(g(X)=2X^2-X-6\). Let’s focus on \(E[x]\) and \(E[X^2]\).

Example continuous

\[E[X]=\int_{-\infty}^{\infty} xf_X(x)dx = \int_0^1 x\times 3x^2 dx\] \[= \int_0^1 3x^3dx=\left[3\frac{x^4}{4}\right]_{0}^1=\frac{3}{4}=0.75\]

\[E[X^2]=\int_{-\infty}^{\infty} x^2f_X(x)dx = \int_0^1 x^2\times 3x^2 dx\] \[= \int_0^1 3x^4dx=\left[3\frac{x^5}{5}\right]_{0}^1=\frac{3}{5}=0.6\]

Example continuous

Finally,

\(E[g(x)]=2\times 0.6 - 0.75 - 6 = -5.55\)

p-quantile

The p-quantile, \(x_p\), of a r.v. \(X\) is a location parameter, with fixed value.

p-quantile for discrete r.v.

\(x_p\) is the value for \(x\in\Omega_X\) such that:

1. \(P(X\leq x)\geq p\)
2. \(P(X\geq x)\geq 1-p\)

p-quantile for continuous r.v.

\(x_p\) is an \(x\in\Omega_X\) such that \(F_X(x)=p\)

Let’s apply this for the examples we just used for the expected value.

Example - Discrete

Find the median (0.5-quantile) for \(X\)

\[ F(x)=P(X\leq x)=\left\{ \begin{array}{cc} 0 &, x\leq 0\\ 0.05 &, 0\leq x <1 \\ 0.35 &, 1\leq x < 2 \\ 0.7 &, 2 \leq x < 3 \\ 0.95 &, 3 \leq x < 4 \\ 1 &, x\geq 4 \end{array} \right. \]

For example \(F(2^-)=0.35\leq 0.5 \leq 0.7=F(2)\) and therefore \(X_{0.5}=Me = 2\). Given that \(E[X]=1.95<Me(X)=2\) the distribution is slightly negatively (or left) skewed.

Example - Continuous

\[F_X(x)=P(X\leq x)=\left\{ \begin{array}{cc} 0 & x < 0 \\ x^3 & 0\leq x <1 \\ 1 & x\geq 1 \end{array} \right. \]

Let’s find \(x\) such that \(F(x)=0.5\)

\(F(x)=0.5\Leftrightarrow x^3=0.5\Leftrightarrow x=\sqrt[3]{0.5}\approx0.7937\)

And therefore, \(x_{0.5}=Me=.7937\)

Variance

Let \(X\) be a r.v. The variance of \(X\), if it exists, is defined as:

\[V[X]=E\left[\left(X-E[X]\right)^2\right]\]

It can be shown, very easily, with some algebraic manipulation that \(V[X]=E\left[X^2\right]-\left(E[X]\right)^2\)

Variance

Remember that \(E[X]\equiv\mu_X\)

1. For a discrete r.v.: \[V[X]=\sum_{x\in\Omega_X}(x-\mu_X)^2P(X=x)\]
2. For a continuous r.v.: \[V[X]=\int_{-\infty}^{\infty}(x-\mu_X)^2f_X(x)dx\]

Variance

Some properties for the variance:

1. \(V[a]=0\) for \(a\in\mathbb{R}\)
2. \(V[aX+b] = a^2V[X]\) for \(a,b\in \mathbb{R}\) and \(X\) a r.v.
3. If \(X\) and \(Y\) are independent r.v. with finite variance, then \[V[X\pm Y]=V[X]+V[Y]\]

Standard deviation \(\sigma_X\)

If \(\sigma^2_X\) is the variance of \(X\), then the standard deviation is known as: \[\sigma_X=\sqrt{V[X]}\]

One characteristic of the standard deviation is that its units are the same as those of the random variable.

Coefficient of variation

While the variance and standard deviation allow us to measure the dispersion of the data, we might want to have it relative to the mean (a \(\sigma_X=1\) can be a lot for \(X\) taking relatively low values, but negligible if we are talking in millions!)

For that we use the coefficient of variation:

\[C.V._X =\frac{\sigma_X}{\mu_X}\times 100\]

Coefficient of variation

Some properties of the \(CV_X\)

1. It is not defined for \(\mu_X=0\)
2. Lower values for \(CV_X\) means less dispersion around \(\mu_X\), therefore, more precision.
3. \(CV_X\) could be a measure for risk, if we are looking at returns for some asset.
4. \(CV_X\) is the relative weight of deviations from the mean, over the mean itself.

Example

Let’s compute \(\sigma^2\), \(\sigma\), and \(CV\) for our previous examples:

1. \(V[X]=E[X^2]-\mu_X^2=4.75-1.95^2=0.9475\)
2. \(\sigma=\sqrt{V[X]}=\sqrt{0.9475}=0.97\)
3. \(CV=\frac{\sigma}{\mu}\times 100 = \frac{0.97}{1.95}\times 100 = 49.7\%\)

Example

For the continuous r.v. case:

1. \(V[X]=E[X^2]-\mu_X^2=0.6-0.75^2=0.0375\)
2. \(\sigma=\sqrt{V[X]}=\sqrt{0.0375}=0.1936\)
3. \(CV=\frac{\sigma}{\mu}\times 100 = \frac{0.1936}{0.75}\times 100=25.81\% < 50\%\)

❓ Moments of a Population · Question 1

For constants \(a\) and \(b\), \(V[aX+b]\) equals:

A. \(a^2V[X]\)

B. \(aV[X]+b\)

C. \(a^2V[X]+b^2\)

D. \(aV[X]\)

❓ Moments of a Population · Question 2

The coefficient of variation is useful because it:

A. is a pure number, measuring dispersion relative to the mean

B. keeps the units of \(X\)

C. is always below 1

D. equals the variance

✏️ Moments of a Population · Question 3

The number of faulty items \(X\) in a batch has \(f(0)=0.5\), \(f(1)=0.3\), \(f(2)=0.15\), \(f(3)=0.05\).

Compute \(E[X]\), \(V[X]\), \(\sigma_X\) and the coefficient of variation.

📝 Homework

Problem set 2.2, Questions 1 to 9. They are all single random variables: pdf, cdf, quantiles and moments.

Next lecture we finish Topic 2 with random pairs, and then we review for the midterm.