STAT5002 Chap.5 The Regression Line, Residuals and R Squared
The Regression Line, Residuals and R Squared
Once you can measure how tightly a scatter plot clusters with the correlation coefficient, the next step is to summarise the cloud with a line you can actually predict from. The regression line of y on x needs only five numbers: the two means, the two SDs and r. The slope is b = r × SD(y) ÷ SD(x) and the intercept is a = ȳ − b x̄, so the line always passes through the point of averages.
You will see why the tempting SD line, which ignores r, overshoots at both ends of the cloud, and why the regression line climbs only r SDs of y for every SD of x. We then use the line for prediction and compare it with the baseline prediction, which simply uses the mean of y for every case.
A short section explains why you cannot rearrange the y on x line to predict x from y: unless r is exactly plus or minus one you must refit with lm(x ~ y), which gives a different slope. The second half is about how well the line fits. A residual is an observed value minus its fitted value, and least squares residuals always average zero.
Comparing SSE, the squared residuals left by the line, with SST, the squared deviations left by the baseline, gives the coefficient of determination: one minus SSE over SST, which equals r squared and is read as the share of variation in y explained by the line.
Finally you learn to read a residual plot, slice it into vertical strips, and decide whether the data are homoscedastic, so the line is safe for prediction, or heteroscedastic, so it is not. Every example uses data we built ourselves, with the matching R commands.
What this chapter covers
- 01
Five summaries that define a scatter plot
- 02
Slope b = r SDy/SDx and intercept a = ȳ − b x̄
- 03
Why the SD line overshoots at the extremes
- 04
Fitting the line in R with lm(y ~ x)
- 05
Prediction from the line versus the baseline mean
- 06
Predicting x from y means refitting, not rearranging
- 07
Residuals as observed minus fitted values
- 08
Average residual is always zero
- 09
SSE, SST and r squared as explained variation
- 10
Residual plots and vertical strips
- 11
Homoscedastic versus heteroscedastic data
Worked example · free
Sleep and reaction time: line, prediction, r squared and one residual
- +1Slope: b = r × SDy / SDx = −0.55 × 28 / 1.1 = −14.0 ms per extra hour of sleep. The sign matches r, so reaction time falls as sleep rises.
- +1Intercept: a = ȳ − b x̄ = 312 − (−14.0)(6.8) = 312 + 95.2 = 407.2, so the line is ŷ = 407.2 − 14.0x.
- +1Prediction at x = 5.5: ŷ = 407.2 − 14.0 × 5.5 = 330.2 ms. Check: 5.5 hours is 1.18 SDs below the mean sleep, and the prediction sits 0.65 SDs of reaction time above 312, which is r times as many SDs in the opposite direction.
- +1Explained variation: r² = (−0.55)² = 0.3025, so the line explains about 30% of the variation in reaction time; the other 70% is left in the residuals.
- +1Residual: e = y − ŷ = 340 − 330.2 = 9.8 ms. It is positive, so this participant was slower than the line predicts.
Key terms
- Regression line
- The straight line ŷ = a + bx with slope r × SDy/SDx and intercept ȳ − b x̄; it passes through the point of averages and minimises the sum of squared residuals.
- Baseline prediction
- The prediction that ignores x and uses the mean of y for every case; it is the reference the regression line is compared against.
- Residual
- The vertical distance between an observed point and the regression line, e = y − ŷ; positive when the point lies above the line.
- Coefficient of determination
- The quantity 1 − SSE/SST, equal to r squared for a simple linear regression; it is the proportion of variation in y explained by the line.
- Residual plot
- A scatter plot of residuals against x, used to check whether a linear model is appropriate; a good fit shows random scatter around zero.
- Heteroscedasticity
- A pattern in which the vertical spread of the points or residuals changes across the vertical strips of x; the regression line should not then be used for prediction.
The Regression Line, Residuals and R Squared FAQ
How do I get the regression slope from the correlation and the two standard deviations?
Multiply r by the SD of the response and divide by the SD of the predictor: b = r × SDy/SDx. Then the intercept is the mean of y minus b times the mean of x. Use the same SD version, sample or population, for both, since only their ratio matters.
Can I rearrange y = a + bx to predict x from y?
Not unless every point lies on the line, that is r equals plus or minus one. Otherwise the line for predicting x is a different fit with slope r × SDx/SDy, so refit it with lm(x ~ y) and use those coefficients instead of solving the original equation.
What does r squared actually mean in linear regression?
It is the share of the variation in y, measured by squared deviations from the mean, that the regression line removes. An r squared of 0.47 says the residuals keep 53% of that variation, so predictions improve on the plain average but remain far from exact.
Why do the residuals from a regression line add up to zero?
Because the intercept is set to ȳ − b x̄, the line passes through the point of averages, and the positive and negative gaps cancel exactly. A nonzero sum in your own working is therefore a sign of an arithmetic slip, often in the fitted values.
How can I tell from a residual plot whether the data are homoscedastic?
Divide the plot into a few vertical strips along x and compare how widely the residuals spread in each. Roughly equal spread is homoscedastic; spread that widens or narrows, such as a fan shape, is heteroscedastic and means the line should not be used for prediction.
Exam move
Build one small data set of your own with eight to ten points and do the whole chain by hand once: means, SDs, r, slope, intercept, one prediction, every residual, SSE, SST and r squared. Then repeat it in R with lm(), cor() and model$residuals and confirm every number matches, including the residuals summing to zero.
Next, practise the reverse question on the same data: refit with lm(x ~ y) and write down how the two slopes differ, so the rearranging trap becomes concrete. For prediction questions, train yourself to state the baseline mean beside the regression prediction and to say how many SDs the prediction moved.
Finally, sketch two residual plots from memory, one even band and one fan, draw three vertical strips on each, and write the one sentence about whether the line may be used for prediction.
Working through The Regression Line, Residuals and R Squared in STAT5002? Sia is AskSia’s AI Statistics tutor — ask any STAT5002 The Regression Line, Residuals and R Squared question and get a clear, step-by-step explanation grounded in how STAT5002 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
STAT5003 Computational Statistical Methods · DATA2002 Data Analytics: Learning from Data · QBUS5001 Foundation in Data Analytics for Business
