Q.A box contains three coins: two fair coins and one fake two-headed coin. A coin is picked randomly from the box and tossed. a) What is the probability that it lands head up? b) If it happens to be head, what is the probability that it is the two-headed coin?

Bayes' Theorem: Learning from Evidence

Imagine you have a bag with 3 red marbles and 7 blue marbles. If you pick one at random, the chance it's red is 3 out of 10 — that's straightforward. But now suppose someone picks a marble, doesn't show it to you, but tells you it's not blue. Suddenly, the only possibilities left are the red marbles. The probability that the hidden marble is red jumps to 1 (certainty). You just updated your belief based on new evidence.

That's the core idea of Bayes' Theorem: how to revise a probability when you get new information. It answers the question: Given that I now know B happened, how should I change my belief about A?

The Intuition in One Sentence

Bayes' Theorem says: The probability that A is true, given that B is true, equals the probability that B would happen if A were true, times the original probability of A, divided by the overall probability of B.

In other words: Your updated belief = (likelihood of the evidence under your hypothesis) × (your prior belief) / (total probability of the evidence).

The Precise Statement

Let $A$ and $B$ be two events. Then:

Where:

• $P(A \mid B)$ is the posterior probability — what you want: the probability of $A$ given that $B$ occurred.
• $P(B \mid A)$ is the likelihood — how probable the evidence $B$ is if $A$ is true.
• $P(A)$ is the prior probability — your initial belief about $A$ before seeing any evidence.
• $P(B)$ is the marginal probability of $B$ — the total chance that $B$ happens, regardless of $A$.

Why It Works: A Simple Example

Suppose 1% of a population has a disease. A test for the disease is 99% accurate: it correctly identifies 99% of those who have it (true positive) and correctly says 99% of those who don't have it are negative (true negative). You take the test and get a positive result. What is the probability you actually have the disease?

Many people guess 99%. But that's wrong — and Bayes' Theorem shows why.

Let $D$ = "has the disease", $T^+$ = "tests positive". We know:

• $P(D) = 0.01$ (prior)
• $P(T^+ \mid D) = 0.99$ (likelihood)
• $P(T^+ \mid \text{not } D) = 0.01$ (false positive rate)

First, find $P(T^+)$, the total probability of a positive test:

Now apply Bayes:

So even with a positive test, there's only a 50% chance you have the disease. The test is good, but the disease is rare — most positive results come from the large number of healthy people who get false positives. …