Date: January 24, 2021

Topic: Variance

Recall

How is the variance defined?

Which moment of the distribution is the variance equivalent too?

What is the expression for estimating the variance from a set of random variables?

If we use the expression on the right to estimate the variance what must we be able to say about the random variables that were inserted into the equation?

Definition

The variance of a random variable, $X$, is calculated using:

<aside> 💡 $\textrm{var}(X) = \mathbb{E}(X^2) - \mathbb{E}(X)^2$

</aside>

It is straightforward to show that calculating the variance using the expression above is equivalent to calculating the following expectation:

$$ \textrm{var}(X) = \mathbb{E}\left\{[X-\mathbb{E}(X)]^2\right\} $$

This expectation is also referred to as the 2nd central moment of the distribution.

Estimating the variance

If you have a sample of $N$ identical random variables, $X_i$, that are also all independent you can estimate the variance of the distribution they were sampled from using:

<aside> 💡 $S^2 = \frac{N}{N-1} \left[ \frac{1}{N} \sum_{i=1}^N X_i^2 - \left( \frac{1}{N} \sum_{i=1}^N X_i \right)^2 \right]$

</aside>

This expression is equivalent to:

$$ S^2 = \frac{1}{N-1} \sum_{i=1}^N \left[ X_i - \left( \frac{1}{N} \sum_{i=1}^N X_i \right) \right]^2 $$

If the $N-1$ in the expression above is replaced by $N$ then the resulting estimator is biased because the sample mean in these expressions is an estimate for the true expectation as is discussed in the video below:

https://www.youtube.com/watch?v=luoyGyDQ2sk

<aside> 📌 SUMMARY: The variance is defined using the expression at the top of the page. It can be estimated by using the expression that appears lower down the page.

</aside>