Regression Coefficients and Their Properties

The two numbers $b_{yx}$ and $b_{xy}$ are the regression coefficients. They carry all the information the regression lines need, and they obey a set of properties that are frequently tested and are also useful checks on your arithmetic.

Property 1 — The two coefficients have the same sign as $r$. Since $b_{yx}=r\dfrac{\sigma_y}{\sigma_x}$ and $b_{xy}=r\dfrac{\sigma_x}{\sigma_y}$, and the standard deviations $\sigma_x,\sigma_y$ are always positive, the sign of each coefficient equals the sign of $r$. Hence both regression coefficients are positive, or both are negative — they can never have opposite signs.

Property 2 — $r$ is the geometric mean of the two coefficients.

The sign of $r$ is the common sign of the two coefficients (Property 1).

Property 3 — Both coefficients cannot exceed $1$ together. Because $b_{yx}\cdot b_{xy}=r^{2}$ and $r^{2}\le 1$, the product of the two coefficients can never exceed $1$. So if one coefficient is greater than $1$, the other must be less than $1$ (indeed less than the reciprocal of the first).

Property 4 — Arithmetic mean of the coefficients is not less than $|r|$. By the AM–GM inequality, $\dfrac{b_{yx}+b_{xy}}{2}\ge \sqrt{b_{yx}b_{xy}}=|r|$ (for positive coefficients).

Property 5 — Independent of change of origin, dependent on change of scale. Shifting the data by a constant (change of origin) leaves the regression coefficients unchanged, but rescaling (change of scale) alters them. Precisely, if $u=\dfrac{x-a}{h}$ and $v=\dfrac{y-b}{k}$, then

$b_{yx} = \frac{k}{h}\,b_{vu}, \qquad b_{xy} = \frac{h}{k}\,b_{uv}.$ …