Mean and Variance of a Discrete Probability Distribution

Once a probability distribution is written down, two summary numbers describe it well: where it is centred, and how spread out it is. These play exactly the role that the mean and variance play for an ordinary frequency distribution in statistics — the only difference is that probabilities replace relative frequencies.

Mean. The mean (also called the expected value) of a discrete random variable $X$ with values $x_i$ and probabilities $P(x_i)$ is

$\mu = \sum_i x_i \, P(X = x_i)$.

It is a weighted average of the possible values, each value weighted by how likely it is. It represents the long-run average value of $X$ if the experiment were repeated a very large number of times.

Variance and standard deviation. The variance measures how far, on average, the values of $X$ spread out from the mean:

$\sigma^2 = \sum_i (x_i - \mu)^2 \, P(X = x_i)$.

Expanding the square and using $\sum_i P(x_i) = 1$ and $\sum_i x_i P(x_i) = \mu$ gives a much easier computational form:

$\sigma^2 = \sum_i x_i^2 \, P(X = x_i) - \mu^2$.

In words: the variance is the mean of the squares minus the square of the mean. This shortcut is what is actually used in every worked problem, since it avoids recomputing $(x_i - \mu)$ for every value. The non-negative square root $\sigma$ is called the standard deviation of $X$, and is in the same units as $X$ itself, which makes it easier to interpret than the variance.

Worked Example. A random variable $X$ takes the values $1, 2, 3, 4$ with probabilities proportional to the value itself: $P(X = x) = kx$ for $x = 1,2,3,4$, where $k$ is a constant to be found.

Step 1 — find $k$. Since the probabilities must add to $1$: $k(1+2+3+4) = 1 \Rightarrow 10k = 1 \Rightarrow k = \frac{1}{10}$. So $P(1) = 0.1,\; P(2)=0.2,\; P(3)=0.3,\; P(4)=0.4$.

Step 2 — mean. $\mu = 1(0.1) + 2(0.2) + 3(0.3) + 4(0.4) = 0.1+0.4+0.9+1.6 = 3.0$. …