Basic concepts

Overview

When studying a particular random phenomenon, we usually focus on the following elements:

• The sample space $\Omega$, which specifies all possible outcomes of the random phenomenon.
• The event field $\mathcal{F}$, which represents all events of interest.
• Probability $P$, which describes how likely each event is to occur.

Sample spaces and random events

Definition

An indivisible outcome that may occur in a random phenomenon is called a sample point. The set of all sample points is called the sample space, usually denoted by $\Omega$.

A random event is a subset of the sample space $\Omega$, composed of several sample points and denoted by the uppercase letters $A, B, C, \cdots$.

For an outcome $\omega$ of a random phenomenon and a random event $A$, we say that event $A$ occurs if and only if $\omega \in A$.

For example, the result of rolling a die once is a random phenomenon, whose sample space can be written as $\Omega=\{1,2,3,4,5,6\}$. Let random event $A$ be “the result is greater than $4$”; then $A = \{ 5, 6 \}$. If a roll produces $\omega = 3$, then $\omega \notin A$, so event $A$ does not occur.

Operations on events

Because random events are defined as subsets of the sample space $\Omega$, set operations such as intersection, union, and complement can be transferred to random events. The notation remains the same as for set operations.

In particular, the union of events $A \cup B$ may also be written as $A + B$, and the intersection of events $A \cap B$ may also be written as $AB$; these may then be called the sum event and the product event, respectively.

Event fields

When studying a particular random phenomenon, we need to specify which events interest us. By the definition of a random event, it is clear that $\mathcal{F} \subset 2^{\Omega}$ (where $2^{\Omega}$ denotes the power set of $\Omega$), but $\mathcal{F} = 2^{\Omega}$ is not required. This may be difficult to appreciate when the sample space $\Omega$ is finite: although $2^{\Omega}$ is larger, it is still finite. When $\Omega$ is infinite, however, the cardinality of $2^{\Omega}$ becomes much larger, and it inevitably contains events with “not very good” properties that we do not care about. Giving up some properties merely to accommodate those events would then be counterproductive.

Although $\mathcal{F} = 2^{\Omega}$ is not required, this does not mean that every subset of $2^{\Omega}$ can be an event field. We are usually interested in the probabilities of events obtained by operating on other events, so we want the event field $\mathcal{F}$ to satisfy the following conditions:

• $\varnothing \in \mathcal{F}$；
• If $A \in \mathcal{F}$, then the complementary event $\bar{A} \in \mathcal{F}$;
• If there is a sequence of events $A_n \in \mathcal{F}, n = 1, 2, 3\dots$, then $\bigcup A_n \in \mathcal{F}$.

In short, the event field $\mathcal{F}$ is closed under complementation and countable unions, and contains the element $\varnothing$.

It can be proved that an event field $\mathcal{F}$ satisfying the three conditions above is also closed under countable intersections.

For a die roll, with sample space $\Omega=\{1,2,3,4,5,6\}$, the following two sets can be event fields:

• $\mathcal{F}_1 = \{ \varnothing, \Omega \}$
• $\mathcal{F}_2 = \{ \varnothing, \{1, 3, 5\}, \{2, 4, 6\}, \Omega \}$

The following two sets cannot:

• $\mathcal{F}_3 = \{ \varnothing, \{1\}, \Omega \}$ (not closed under complementation)
• $\mathcal{F}_4 = \{ \{1, 3, 5\}, \{2, 4, 6\} \}$ (does not contain $\varnothing$ and is not closed under unions)

Probability

Definition

Classical definition

In the early development of probability theory, the random phenomena under consideration were relatively simple: the sample space $\Omega$ was finite, and it was intuitively assumed that all sample points were equally likely. This led to the following definition:

If a random phenomenon satisfies:

• It has only finitely many elementary outcomes;
• Every elementary outcome is equally likely;

then for every event $A$, define its probability as

$$P(A)=\frac{\#(A)}{\#(\Omega)}$$

where $\#(\cdot)$ denotes a measure of the size of the random event (a set).

It was later discovered that this definition can be directly extended to some situations where $\Omega$ is infinite, giving rise to the so-called geometric probability.

Axiomatic definition

The intuitive definition above has a major logical flaw: it uses the notion of “likelihood” while defining “probability”, creating a circular definition. Moreover, “equally likely” becomes ambiguous when the sample space is infinite, leading to a series of problems including the Bertrand paradox.

After extensive exploration, the Soviet mathematician Kolmogorov gave the first axiomatic definition of probability in 1933, in his book Foundations of the Theory of Probability:

A probability function $P$ is a mapping from the event field $\mathcal{F}$ to the closed interval $[0, 1]$ that satisfies:

• Normalization: The probability of event $\Omega$ is $1$, that is, $P(\Omega)=1$.
• Countable additivity: If a sequence of events $A_1, A_2, \cdots$ is pairwise disjoint, then $P\left( \bigcup_{i \geq 1} A_i \right) = \sum_{i \geq 1} P(A_i)$.

Properties of probability functions

For any random events $A, B \in \mathcal{F}$, we have

• Monotonicity: If $A \subset B$, then $P(A) \leq P(B)$.
• Inclusion–exclusion principle: $P(A+B) = P(A) + P(B) - P(AB)$.
• $P(A - B) = P(A) - P(AB)$, where $A - B$ denotes the set difference.

Probability spaces

As mentioned at the beginning, when studying a particular random phenomenon we usually focus on the sample space $\Omega$, the event field $\mathcal{F}$, and the probability function $P$. We call the triple $(\Omega, \mathcal{F}, P)$ a probability space.

Probability is meaningful only when discussed within a specified probability space. Ultimately, the Bertrand paradox mentioned above arises from failing to define the sample space $\Omega$ unambiguously.

References and notes

• Probability (mathematical branch) _ Baidu Encyclopedia
• Probability - Wikipedia