In this chapter, we introduce the concept of the probability of an event. Then we show how probabilities can be computed in certain situations. As a preliminary, however, we need to discuss the concept of the sample space and the events of an experiment.
An experiment is the process by which an observation is made. In particular, we are interested in a random experiment whose outcome is not predictable with certainty. The set of all possible outcomes of an experiment is known as the sample space of the experiment, and is often denoted S. An event (denoted $E$) is a set that contains some possible outcomes of the random experiment.
By definition, any event is a subset of the sample space. For a given random experiment, its sample space is unique. Let’s see some examples.
| Experiment | Possible outcome | Sample space | Event |
|---|---|---|---|
| Test of a certain disease on a patient | Positive or negative | $S = \{p, n\}$ | $E = \{n\}$ |
| Rolling a six-sided die | $1, \cdots, 6$ | $S = \{1, 2, 3, 4, 5, 6\}$ | Outcome $\geq 3: E = \{4, 5, 6\}$ |
| Tossing two coins | Each coin is either head or tail | $S = \{(H,H), (H,T), (T,H), (T,T)\}$ | First toss is a head: $E = \{ (H, H), (H, T) \}$ |
| Life time of a computer | All non-negative real numbers | $S = \{X: 0 \leq X < \infty \}$ | Survives for more than 10 hours: $E = \{ X: 10 < X < \infty \}$ |
For each experiment, we may define more than one event. Take the die-rolling example, we can define events like
$$ \begin{aligned} &E_3 = \{3\} &E_4 = \{4\} \\ &E_5 = \{5\} &E_6 = \{6\} \end{aligned} $$
If we observed the event $E = \{4, 5, 6\}$, it means we observed one of the three events $E_4$, $E_5$ or $E_6$. We say $E$ can be decomposed into $E_4$, $E_5$ and $E_6$. If an event can be further decomposed, it is called a compound event. Otherwise, it’s called a simple event. Each simple event contains one and only one outcome.
Finally, events with no outcome is called the null event, and is denoted $\empty$. For example, an event of the outcome is greater than 7 in the die-rolling example.
Suppose we have a sample space $S$ and two events $E$ and $F$.
If all of the outcomes in $E$ are also in $F$, then we say that is contained in $F$, or $E$ is a subset of $F$. We write it as $E \subset F$. Subsets have several properties:
We denote $E \cup F$ as the union of the two events. It is a new event which consists of all outcomes in $E$, and all the outcomes in $F$. In other words, $E \cup F=\{\text{either in }E\text{ or in }F\}$. The union operation also has a few properties: