DATA1001 Chap.4 Regression
Regression
Regression describes and predicts a quantitative response from a quantitative predictor with a fitted straight line. Correlation summarises the direction and strength of linear association. It is unitless, symmetric in the two variables, unchanged by positive shifts or rescaling, sensitive to influential points and unable to capture every curved relationship.
The regression slope equals correlation times response SD divided by predictor SD and is measured in response units per predictor unit. The fitted line passes through the point of averages. A fitted value is the model prediction at a chosen predictor value. A residual is observed minus fitted, so its sign records whether the point lies above or below the line.
Canvas Topic 5, Linear Model, asks students to read coefficient output, calculate fitted values and residuals, distinguish the regression line from the SD line and judge model adequacy. Regression to the mean follows because the fitted response in standard units is correlation times the predictor's standard units. When absolute correlation is below one, an extreme predictor is paired with a less extreme fitted response.
This is a prediction property, not a causal mechanism. R squared describes the fraction of observed response variation explained by the fitted linear relationship in the sample. Residual standard error describes a typical vertical prediction gap in response units. A residual plot should show an unstructured band around zero.
Curvature challenges the straight line, a funnel challenges constant spread and an isolated point may dominate the fit. Avoid extrapolation outside the observed predictor range. In output questions, read the model call first, attach units to the slope, then combine estimate, uncertainty, diagnostics and study design.
What this chapter covers
- 01
Topic 5: Linear Model
- 02
Interpret correlation and its limitations
- 03
Calculate slope, intercept and fitted values
- 04
Read observed minus fitted residuals
- 05
Interpret R squared and residual standard error
- 06
Use residual plots and avoid extrapolation
Worked example · free
Read a fitted line and its residual
- 1Substitute 8: fitted use is 12 plus 24, which equals 36 units.
- 1Residual is observed minus fitted: 40 minus 36 equals 4 units.
- 1The positive residual places the observation above the line.
- 1The slope says each extra opening hour is associated with 3 additional units on average.
Key terms
- Correlation
- A unitless measure of the direction and strength of a linear association.
- Regression slope
- The predicted response change for a one unit predictor increase.
- Fitted value
- The response predicted by the fitted line at a specified predictor value.
- Residual
- The observed response minus its fitted value.
- R squared
- The fraction of observed response variation explained by the fitted linear relationship in the sample.
- Extrapolation
- Prediction outside the observed predictor range, where the fitted relationship may not continue.
Regression FAQ
Does correlation establish causal effects?
Correlation describes linear association and does not identify a causal effect. A strong value can arise from confounding, selection or reverse direction. Use the study design to decide whether causal language is available, even when the slope p value is very small.
What does the slope mean?
State the average predicted change in the response for a one unit increase in the predictor, with both variables and units named. If the predictor value zero has no useful meaning, the intercept can be mathematically necessary without a practical interpretation.
How is a residual calculated?
Subtract fitted response from observed response. A positive residual is above the line and a negative residual is below it. Residuals are in response units, and their plot is used to check whether structure remains after fitting the line.
What does R squared tell me?
R squared is the fraction of observed response variation explained by the fitted linear relationship in the sample. It does not give the slope, prove causation or guarantee good predictions. Inspect residual patterns and prediction context as well.
When is prediction unsafe?
Prediction is unsafe when it extrapolates far beyond observed predictor values, when residual plots show curvature or changing spread, or when an influential point controls the line. Name the specific issue rather than declaring every model unusable from one imperfect feature.
Exam move
Read regression output in a fixed order. Start with the model call to identify response and predictor. State the coefficient units, use the intercept and slope for a fitted value, and compute residual as observed minus fitted. Next read slope uncertainty and its p value, then residual standard error and R squared. Keep their jobs separate.
Practise standard unit predictions so regression to the mean becomes visual: multiply the predictor's standard units by correlation. For plots, cover any caption and decide whether the pattern is straight, whether spread is stable and whether one point is influential. Write one corrective sentence for each failed condition.
Redo several questions with x and y swapped so you remember that correlation stays the same while the regression equation changes. End every output response with a design sentence, especially when a tempting causal claim appears. The chapter exam lens provides two quick items and one residual interpretation for timed revision.
