Mathematics and Statistics · Ch 11 — Linear Regression

The Two Lines of Regression and Their Equations

For a linear relationship there are two lines of regression, one for each direction of estimation. Both are straight lines and both pass through the point of means $(\bar x,\ \bar y)$ .

(1) Regression line of $Y$ on $X$ — used to estimate $Y$ for a given $X$ :

y - \bar y = b_{yx}\,(x - \bar x),

where $b_{yx}$ is the regression coefficient of $Y$ on $X$ , given by

b_{yx} = r\,\frac{\sigma_y}{\sigma_x} = \frac{\operatorname{Cov}(x,y)}{\sigma_x^{2}} = \frac{\sum (x-\bar x)(y-\bar y)}{\sum (x-\bar x)^{2}} = \frac{n\sum xy - \sum x\sum y}{n\sum x^{2} - (\sum x)^{2}}.

(2) Regression line of $X$ on $Y$ — used to estimate $X$ for a given $Y$ :

x - \bar x = b_{xy}\,(y - \bar y),

where $b_{xy}$ is the regression coefficient of $X$ on $Y$ , given by

b_{xy} = r\,\frac{\sigma_x}{\sigma_y} = \frac{\operatorname{Cov}(x,y)}{\sigma_y^{2}} = \frac{\sum (x-\bar x)(y-\bar y)}{\sum (y-\bar y)^{2}} = \frac{n\sum xy - \sum x\sum y}{n\sum y^{2} - (\sum y)^{2}}.

Choosing the right line. To estimate $y$ from a given $x$ , use the line of $Y$ on $X$ ; to estimate $x$ from a given $y$ , use the line of $X$ on $Y$ . Using the wrong line gives a poorer estimate, so always match the line to the variable you are predicting.

Both lines pass through $(\bar x, \bar y)$ . Substituting $x=\bar x$ in the first line gives $y=\bar y$ , and similarly for the second — so the two lines intersect exactly at the point of means. This fact lets us recover $\bar x$ and $\bar y$ by solving the two regression equations simultaneously (§4).

Tip

A quick way to write the equations …

Definition 1Regression coefficient of $Y$ on $X$ ($b_{yx}$)

The slope of the line of $Y$ on $X$ : $b_{yx}=r\dfrac{\sigma_y}{\sigma_x}=\dfrac{\sum(x-\bar x)(y-\bar y)}{\sum(x-\bar x)^2}$ . It gives the average change in …

Definition 2Regression coefficient of $X$ on $Y$ ($b_{xy}$)

The slope (measured against $Y$ ) of the line of $X$ on $Y$ : $b_{xy}=r\dfrac{\sigma_x}{\sigma_y}=\dfrac{\sum(x-\bar x)(y-\bar y)}{\sum(y-\bar y)^2}$ . It gives the average chang …

Definition 3Point of means

The point $(\bar x,\bar y)$ through which both regression lines pass; it is their point …

Start your 14-day free trial to unlock the full solution →

← Previous · 1Meaning of Regression Next · 3 →Regression Coefficients and Their Properties