ECON7880 Chap.4 Linear Regression and Residual Diagnostics
Linear Regression and Residual Diagnostics
Interpret the fitted equation in units
Linear regression predicts a numeric target from an intercept and weighted predictors.
This chapter separates the model equation, fitted value and residual, then links RSS, MSE and R-squared to the errors shown in a residual plot.
Diagnose what one score hides
Curvature, changing spread and influential observations can invalidate a simple linear story even when R-squared looks strong.
Multiple regression and interactions also require held-fixed language and context-specific slope interpretation.
Keep equation, prediction and error distinct
The equation describes how predictors combine. A fitted value appears after one observation is substituted. A residual requires the observed target and equals observed minus predicted.
Writing those objects on separate lines protects the residual sign and units. A positive residual means the observation lies above the line and the model under-predicted; a negative residual means the model over-predicted.
Connect the objective to reported metrics
Ordinary least squares selects coefficients that minimise the sum of squared residuals, written as ∑(y - ŷ)².
MSE averages squared errors and keeps the squared target unit. R-squared compares residual variation with the variation remaining under a mean-only benchmark. A high training R-squared can coexist with weak test prediction, so always separate fit to known observations from generalisation.
Use residual plots to test the mean function
A well-behaved residual plot has no systematic curve or funnel around zero.
Curvature suggests a missing nonlinear form, changing spread suggests unequal error variance and isolated large points need investigation. Add terms only when their interpretation and held-out performance justify them.
Polynomial flexibility can lower training error while creating unstable predictions near or beyond the observed range.
Interpret multiple regression conditionally
Each slope describes a predictor change while included alternatives are held fixed. An interaction makes one slope depend on another predictor, so the main coefficient alone is incomplete.
Evaluate the combined slope at meaningful reference values and state those values. Avoid causal language unless the data design supports intervention claims; a reliable predictive association is still not proof that changing the predictor changes the outcome.
What this chapter covers
- 01
Intercept and slope units
- 02
Fitted values
- 03
Observed-minus-predicted residuals
- 04
RSS and MSE
- 05
R-squared
- 06
Interactions and residual patterns
Worked example · free
Compute one fitted value and residual
- 2Substitute predictor units.
- 3Compute observed minus predicted.
- 3Interpret slope and error direction.
Key terms
- Fitted Value
- The target value predicted by substituting an observation’s predictors into the estimated regression equation.
- Residual Sum Squares
- The sum of squared observed-minus-predicted residuals across observations.
- Interaction Term
- A product of predictors allowing the effect of one predictor to depend on another.
Linear Regression and Residual Diagnostics FAQ
How should a regression slope be interpreted?
It is the predicted change in the target for a one-unit predictor increase, holding other included predictors fixed. Give the units, identify the fitted specification and stay within the observed data range.
Why are residuals squared during fitting?
Squaring prevents positive and negative misses from cancelling and places larger weight on larger errors. Ordinary least squares chooses coefficients that minimise their sum across all training observations.
What does a curved residual pattern suggest?
It suggests the straight-line mean function is missing structure. Test a justified nonlinear form or transformation using validation rather than adding complexity automatically to the fitted equation, and compare the revised residual pattern and held-out MSE with the original line. Retain simplicity when curvature does not generalise to later or otherwise untouched observations.
Exam move
For each regression example, write the equation, substitute one case, compute the signed residual and interpret the slope in units. Recalculate RSS and MSE from a tiny set of observations so their relationship is visible. Sketch a residual curve, funnel and isolated point, then name the different concern each pattern raises.
For an interaction, compute the conditional slope at two reference values instead of quoting the main coefficient alone. Finish by comparing train and test error and writing a bounded conclusion that names range, held-fixed variables and the strongest diagnostic limitation. Create one deliberately implausible extrapolation and mark the observed predictor range beside it.
Compare a mean-only baseline, a linear model and a flexible polynomial on the same holdout. Practise rejecting the most complex model when its training fit improves but test MSE and edge behaviour deteriorate. Translate the selected equation into one ordinary prediction and one warning for a case beyond the supported range.
Working through Linear Regression and Residual Diagnostics in ECON7880? Sia is AskSia’s AI Economics tutor — ask any ECON7880 Linear Regression and Residual Diagnostics question and get a clear, step-by-step explanation grounded in how ECON7880 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
