The Theorem of Total Probability: From Intuition to Formula
Imagine you want to know the probability that a randomly selected student owns a bicycle. You don't have a direct number, but you do know the school is divided into three houses — Red, Blue, and Green — and you know the bicycle ownership rate within each house. How would you combine that information?
You'd reason: first pick a house at random, then within that house check for bicycle ownership. The overall probability is a weighted average of the house-specific probabilities, where the weights are the probabilities of being in each house. That's the core idea of the Theorem of Total Probability.
The Setup
We have a sample space S (all possible outcomes). Suppose S is partitioned into n mutually exclusive and exhaustive events A1,A2,…,An. "Mutually exclusive" means no two overlap — a student belongs to exactly one house. "Exhaustive" means together they cover the whole space — every student is in some house.
Now let B be any other event in the same sample space — say, "owns a bicycle." We want P(B).
The Intuition
Since the Ai's cover everything, B can be split into pieces: B happens partly when A1 happens, partly when A2 happens, and so on. Formally:
B=(B∩A1)∪(B∩A2)∪⋯∪(B∩An)
Because the Ai's are disjoint, these intersections are also disjoint. So the probability of B is just the sum of the probabilities of these pieces:
P(B)=P(B∩A1)+P(B∩A2)+⋯+P(B∩An)
Now recall the multiplication rule: P(B∩Ai)=P(Ai)⋅P(B∣Ai). Substituting gives:
P(B)=P(A1)P(B∣A1)+P(A2)P(B∣A2)+⋯+P(An)P(B∣An)
That's the theorem.
Theorem of Total Probability
P(B)=∑i=1nP(Ai)P(B∣Ai)
where {A1,A2,…,An} is a partition of the sample space.
Why It Matters
The theorem lets you compute an "overall" probability when you only know probabilities conditioned on different scenarios. It's the bridge between conditional probabilities and unconditional ones.
Think of it as a weighted average: each P(B∣Ai) is weighted by P(Ai), the chance of being in that scenario.
A Concrete Example
A factory has two machines. Machine I produces 60% of the output, Machine II produces 40%. Machine I has a 2% defect rate, Machine II has a 5% defect rate. What's the overall defect rate? …