Conditional Probability and Independence

Overview

When an event has occurred, the probabilities of some random events change as more information becomes known. For example, when drawing from a mobile game's gacha, we might regard getting a six-star item and not getting one in a single draw as equally likely. However, after making $50$ consecutive draws without getting a six-star item, it is no longer particularly sensible to insist that “getting a six-star item and not getting one are equally likely.”

In short, it is necessary to study the probability that an event occurs under certain known conditions.

Conditional Probability

Definition

If event $A$ is known to have occurred, the probability that event $B$ occurs under this condition is called conditional probability, denoted by $P(B|A)$.

In a probability space $(\Omega, \mathcal{F}, P)$, if an event $A \in \mathcal{F}$ satisfies $P(A) > 0$, then the conditional probability $P(\cdot|A)$ is defined by

$$P(B|A) = \frac{P(AB)}{P(A)} \quad \forall B \in \mathcal{F}$$

It can be verified that the $P(\cdot|A)$ defined above is a probability function on $(\Omega, \mathcal{F})$.

The following two formulas follow directly from the definition of conditional probability:

• Multiplication rule for probabilities: In a probability space $(\Omega, \mathcal{F}, P)$, if $P(A) > 0$, then for any event $B$,

$$P(AB) = P(A)P(B|A)$$

• Law of total probability: In a probability space $(\Omega, \mathcal{F}, P)$, if events $A_1, \cdots, A_n$ are pairwise disjoint and their union is $\Omega$, then for any event $B$,

$$P(B) = \sum_{i=1}^{n} P(A_i)P(B|A_i)$$

Bayes' Formula

In general, let $A_1, A_2, \cdots, A_n$ be the possible causes of event $B$. If $P(A_i)$ and $P(B|A_i)$ are known, the probability of $B$ can be calculated using the law of total probability. In many situations, however, we need to infer the probabilities of the possible causes from the result that event $B$ has occurred. Thus,

$$P(A_i|B) = \frac{P(A_iB)}{P(B)} = \frac{P(A_i)P(B|A_i)}{\sum_{j=1}^{n} P(A_j)P(B|A_j)}$$

This is Bayes' formula.

Independence of Events

When studying conditional probability, we may encounter $P(B|A) = P(B)$. Intuitively, whether event $B$ occurs tells us nothing about event $A$; that is, events $B$ and $A$ are “unrelated.” This leads to the following definition.

Definition

If events $A$ and $B$ in the same probability space satisfy

$$P(AB) = P(A)P(B)$$

then $A$ and $B$ are called independent. For multiple events $A_1, A_2, \cdots, A_n$, we call them independent if and only if, for every set of events $\{ A_{i_k} : 1 \leq i_1 < i_2 < \cdots < i_k \leq n \}$,

$$P( A_{i_1}A_{i_2} \cdots A_{i_r} ) = \prod_{k=1}^{r} P(A_{i_k})$$

Independence of Multiple Events

For multiple events, pairwise independence generally does not imply that the events are independent. Consider the following counterexample:

Suppose we have a regular tetrahedral die. Three faces are colored red, green, and blue respectively, while the remaining face contains all three colors. Roll the die once, and let events $A$, $B$, and $C$ denote that the face touching the table contains red, green, and blue, respectively.

It is easy to calculate that $P(A) = P(B) = P(C) = \frac{1}{2}$, while $P(AB) = P(BC) = P(CA) = P(ABC) = \frac{1}{4}$.

Clearly, $A, B, C$ are pairwise independent. However, since $P(ABC) \neq P(A)P(B)P(C)$, $A, B, C$ are not mutually independent.