Transition Probability Matrices (Markov Chains)

Modelling probabilities that change over time

The transition probabilities between two states, A and B, can be read directly off the transition matrix as a from/to table:

Each row sums to $1$ (a Brand-A customer either stays with A or switches to B; a Brand-B customer either switches to A or stays with B), which is exactly the defining property of a transition probability matrix.

A transition probability matrix (or stochastic matrix) describes a system that moves between a fixed set of states from one period to the next, where the probability of moving to each state depends only on the CURRENT state. Each row lists the probabilities of moving FROM that row's state TO every state (including staying put) — every row must sum to exactly $1$.

Example matrix

Suppose customers repeatedly choose between Brand A and Brand B, switching according to

read as: a Brand-A customer stays with A with probability $0.7$ and switches to B with probability $0.3$; a Brand-B customer switches to A with probability $0.4$ and stays with B with probability $0.6$. Both rows sum to $1$, confirming $P$ is a valid transition matrix.

Finding the distribution after one or more steps

If $\pi_0=(p_A,p_B)$ is the probability distribution across states now (a row vector), the distribution one period later is

and after $n$ periods, $\pi_n=\pi_0 P^n$ (apply $P$ repeatedly, once per period). …