Date: January 24, 2021

Topic: Expectation

Recall

How is the expectation of a discrete random variable calculated?

How is the expectation of a function $g(X)$ of a discrete random variable calculated?

How is the expectation of a continuous random variable calculated?

How is the expectation of a function $g(X)$ of a continuous random variable calculated?

Explain what is means when we state the expectation is a linear operator?

How do you estimate the expectation of the random variable?

Notes

Discrete random variables

The expectation of the discrete random variable, $X$, is given by:

<aside> 💡 $\mathbb{E}(X) = \sum_{x=0}^\infty x P(X=x)$

</aside>

where $P(X=x)$ is the probability mass function for the random variable.

Similarly, the expectation of the function $g(X)$ of the discrete random variable is given by:

<aside> 💡 $\mathbb{E}[g(X)] = \sum_{x=0}^\infty g(x) P(X=x)$

</aside>

The expectation of discrete random variables is discussed in the following video:

https://www.youtube.com/embed/4l6N4mD1n6I

Continuous random variables

The expectation of the continuous random variable, $X$, is given by:

<aside> 💡 $\mathbb{E}(X) = \int_{-\infty}^\infty x f(x) \textrm{d}x$

</aside>

where $f(x)$ is the probability density function for the random variable.

Similarly, the expectation of the function $g(X)$ of the continuous random variable is given by:

<aside> 💡 $\mathbb{E}[g(X)] = \int_{-\infty}^\infty g(x) f(x) \textrm{d}x$

</aside>

Linearity

The expectation is a linear operator and as such:

<aside> 💡 $\mathbb{E}\left( \sum_{i=1}^N a_i X_i \right) = \sum_{i=1}^N a_i \mathbb{E}(X_i)$

</aside>

where the $X_i$ are random variables and the $a_i$ are all non-random coefficients. If we have only two random variables the above reduces to:

$$ \mathbb{E}(aX + bY + c) = a\mathbb{E}(X) + b\mathbb{E}(Y) + c $$

Estimating the expectation

You can estimate the expectation for a random variable by calculating the sample mean using:

$$ \mu = \frac{1}{N} \sum_{i=1}^N X_i $$

where the $X_i$ are identically-distributed random variables. The sample mean is an accurate estimator for the expectation of the random variable. It is, in other words, unbiased.

<aside> 📌 SUMMARY: The expectation of a random variable is a number that can be calculated from the probability density function of a continuous random variable or the probability mass function for a discrete random variable. The expectation operator is a linear operator. You can estimate the expectation of a random variable by calculating a sample mean.

</aside>