Axiomatic Approach to Probability

A.N. Kolmogorov's axiomatic system (1933). The limitations of the classical definition led to the modern definition of probability, built on the language of SETS rather than counting -- the axiomatic approach. Andrey Nikolayevich Kolmogorov, a Russian mathematician, combined Richard von Mises's notion of a sample space with measure theory to present this axiomatic system in 1933. It gives a logically complete structure for probability theory, of which the classical theory turns out to be one particular case; the axioms are the rules from which every later theorem of probability is proved.

The three axioms. Let $S$ be a finite sample space, let $\mathcal P(S)$ be the class of all events (subsets of $S$), and let $P$ be a real-valued function defined on $\mathcal P(S)$. $P(A)$ is called a probability function of the event $A$ when the following axioms hold:

• $[P_1]$ Non-negativity axiom. For any event $A$: $P(A)\ge 0$.
• $[P_2]$ Additivity axiom. For any two mutually exclusive events $A,B$: $P(A\cup B)=P(A)+P(B)$.
• $[P_3]$ Normalisation axiom. For the certain (sure) event: $P(S)=1$.

Note 12.1. (i) From the axioms it follows that $0\le P(A)\le 1$ for every event $A$. (ii) If $A_1,A_2,\ldots,A_n$ are mutually exclusive events in $S$, then additivity extends to any finite collection: $P(A_1\cup A_2\cup\cdots\cup A_n)=P(A_1)+P(A_2)+\cdots+P(A_n)$.

Theorem 12.1. When the outcomes are equally likely, let $S$ be a sample space and, for any subset $A$ of $S$, define $P(A)=\dfrac{n(A)}{n(S)}$ as before. Then $P(A)$ satisfies the axioms $[P_1],[P_2]$, and $[P_3]$ -- the classical formula is fully compatible with, and a special case of, the axiomatic system.

Theorem 12.2. When the outcomes are NOT equally likely, a finite probability space is built differently: given $S=\{a_1,a_2,\ldots,a_n\}$, assign each point $a_i$ a real number $p_i$ (its probability), satisfying (i) each $p_i\ge 0$, and (ii) $\sum p_i = p_1+p_2+\cdots+p_n=1$. Define, for any event $A$, $P(A)$ as the sum of the $p_i$ for points inside $A$. Then $P(A)$ again satisfies $[P_1],[P_2],[P_3]$ -- this is how probabilities are assigned when the sample points genuinely have DIFFERENT chances. Such assignments are sometimes displayed as a table of outcomes against their probabilities.

Note 12.2. Even irrational numbers can act as valid probabilities, as long as they are non-negative and the full set sums to 1.

Illustration 12.6. For $S=\{1,2,3\}$: (1) with $P(A)=n(A)/n(S)$, $P(\{1\})=P(\{2\})=P(\{3\})=\tfrac13$ -- all outcomes equally likely, and the axioms hold. (2) Assign $P(\{1\})=\tfrac14, P(\{2\})=\tfrac14, P(\{3\})=\tfrac12$ -- outcomes NOT equally likely, but the axioms still hold. (3) Assign $P(\{1\})=0, P(\{2\})=\tfrac12$, and $P(\{3\})=\tfrac12$ -- again not equally likely, axioms still hold (a probability of exactly 0 for a point that CAN occur is allowed by the axioms). …