In the previous two chapters, we focused on studying a single random variable. However, we can define more than one random variable on the same sample space.

<aside> ❓ Let’s consider the following motivating example. Suppose we roll two six-sided fair dice. The sample space contains 36 possible sample points. Define

$$ \begin{aligned} &X_1 \text{: the number of die 1} \\ &X_2 \text{: the number of die 2} \\ &X_3 \text{: the sum of the two dice} \\ &X_4 \text{: the product of the two dice} \end{aligned} $$

</aside>

Sometimes, we may want to assign a probability to an event that involves two or more random variables. For example,

$$ \begin{aligned} P\{X_1 = 1, X_2 = 2\} &= \frac{|E|}{|S|} = \frac{|E_1 \cap E_2|}{|S|} \\ &= \frac{|(1, 2)|}{|S|} = \frac{1}{36} \end{aligned} $$

This is called the joint probability of $X_1$ and $X_2$.

Two discrete random variables

Let $X_1$ and $X_2$ be two discrete random variables defined on the same sample space. The joint probability mass function for $X_1$ and $X_2$ is given by

$$ p(x_1, x_2) = P\{X_1 = x_1, X_2 = x_2\} $$

The joint probability mass function has the following properties:

1. $p(x_1, x_2) \geq 0$ for all $x_1$ and $x_2$.
2. $\sum\limits_{X_2} \sum\limits_{X_1} p(x_1, x_2) = 1$.

Given the joint PMF, we can also define the joint cumulative distribution function (also called the joint probability distribution function) as

$$ \begin{aligned} F(a, b) &= P\{X_1 \leq a, X_2 \leq b\} \\ &= \sum_{X_1 \leq a, X_2 \leq b} p(x_1, x_2) \\ &= \sum_{X_1 \leq a} \sum_{X_2 \leq b} p(x_1, x_2) \end{aligned} $$

To distinguish with the joint probability mass function and the joint probability distribution function, we call the mass and distribution functions of a single random variable as marginal mass and distribution functions.

Suppose we have two discrete random variables $X$ and $Y$,

$$ \begin{aligned} p_X(x) &= P\{X = x\} \\ &= P\left\{ (X = x) \cap S \right\} \\ &= P\left\{ \{X = x\} \cap \{Y \leq \infty\} \right\} \\ &= \sum_{y: p(x, y) > 0} p(x, y) \end{aligned} $$

The marginal probability mass function of $X$ can be obtained by summing the joint probability mass function over possible values of $Y$. The marginal distribution function can be derived from the joint distribution function:

$$ F_X(a) = P\{X \leq a\} = P\{X \leq a, Y < \infty\} = F_{X, Y}(a, \infty) $$

<aside> ❓ Suppose we have a box that contains 3 red balls and 4 blue balls. Let

$$ \begin{aligned} &X \text{: number of red balls,} \\ &Y \text{: number of blue balls} \end{aligned} $$

If we randomly draw 3 balls out of the 7 balls, find the joint probability mass function $p(x, y)$.

</aside>

We can list out all possible configurations of this random experiment: