Total Probability Theorem

Partition of a Sample Space

A collection of events $E_1,E_2,\dots,E_n$ is called a partition of the sample space $S$ if:

1. $E_i\cap E_j=\varnothing$ for all $i\ne j$ (the events are pairwise mutually exclusive), and
2. $E_1\cup E_2\cup\cdots\cup E_n=S$ (the events are exhaustive -- together they cover every possible outcome).

Every outcome of the experiment therefore belongs to exactly one of $E_1,\dots,E_n$.

Statement of the Theorem

Let $E_1,E_2,\dots,E_n$ be a partition of $S$ with $P(E_i)>0$ for every $i$, and let $A$ be any event of $S$. Then

Proof

Since $E_1,\dots,E_n$ partition $S$, the event $A$ can be written as the union of its pieces lying in each $E_i$:

Because the $E_i$ are pairwise disjoint, the sets $A\cap E_i$ are also pairwise disjoint, so by the addition rule for mutually exclusive events,

Applying the multiplication theorem (Section 2) to each term, $P(A\cap E_i)=P(E_i)P(A\mid E_i)$, gives exactly the stated formula. $\blacksquare$

Why This Is Useful

The theorem is the standard tool whenever an event $A$ can occur through several distinct, non-overlapping "routes" or "causes" $E_1,\dots,E_n$ -- e.g. an item being manufactured by one of several machines, or a ball being drawn from one of several bags/urns chosen at random -- and only the conditional probability of $A$ given each individual route is easy to find directly.

Worked Illustration

Bag I has $4$ white and $3$ black balls; Bag II has $3$ white and $5$ black balls. A bag is selected at random (so $P(\text{I})=P(\text{II})=\tfrac12$) and a ball is drawn from it. Here $\{\text{I},\text{II}\}$ is a partition of the underlying sample space (exactly one bag is chosen). With $P(W\mid \text{I})=\tfrac47$ and $P(W\mid \text{II})=\tfrac38$, …