Pearson's Correlation Coefficient

Covariance tells us the direction in which two variables move together, but its size is tied to whatever units $x$ and $y$ happen to be measured in, so a covariance of, say, 1.2 carries no meaning on its own — is that a strong relationship or a weak one? Karl Pearson's correlation coefficient fixes this by dividing the covariance by the product of the two variables' own standard deviations, which cancels out the units and squeezes the result onto a fixed scale.

• $r$ close to $+1$: a strong positive linear relationship (as $x$ increases, $y$ tends to increase too).
• $r$ close to $-1$: a strong negative linear relationship (as $x$ increases, $y$ tends to decrease).
• $r$ close to $0$: little or no linear relationship (the variables may still be related in a non-linear way that $r$ does not detect).

The geometric point of view. Plotting every pair $(x_i, y_i)$ as a point on a graph gives a scatter diagram, and Pearson's $r$ has a direct visual meaning on it: the closer the plotted points cluster around a single straight line, the closer $|r|$ is to 1; an upward-sloping cluster gives a positive $r$, a downward-sloping cluster gives a negative $r$, and a shapeless cloud of points with no visible line gives an $r$ close to $0$. This geometric picture is exactly why $r$ is described as measuring the strength of linear association — it is, in effect, measuring how tightly the data hugs its own best-fitting straight line.

…