QBUS2820 Chap.2 Least Squares and Linear Regression
Least Squares and Linear Regression
Linear regression expresses a conditional mean through an intercept and slopes, then chooses coefficients by minimising squared residuals. This chapter develops fitted values, residual signs, simple and multiple regression interpretation, categorical predictors, interactions, hierarchy and quadratic terms.
The main discipline is to keep units and conditioning language visible: a multiple-regression slope describes a fitted change in the response for a one-unit predictor change while other included predictors are held fixed. Regression interpretation begins with the scale of every variable.
Changing a predictor from dollars to thousands of dollars changes the numerical slope without changing the fitted relationship, so units belong in every coefficient sentence. Centring a predictor can give the intercept a meaningful reference while leaving fitted values unchanged. Model diagnostics ask whether residual patterns reveal curvature, changing spread, unusual observations or dependence that the equation ignores.
These checks do not automatically dictate one repair; they identify which assumption or prediction region needs investigation. Categorical indicators compare each displayed level with a named reference, and changing that reference reorganises coefficients without changing the underlying fitted group means. Interaction terms require conditional sentences because the slope at one value differs from the slope elsewhere.
Quadratic terms also make marginal effects depend on x, so interpreting only the linear coefficient is incomplete. Extrapolation remains risky even when residual diagnostics look tidy inside the observed range.
What this chapter covers
- 01
Simple linear regression equations
- 02
Least-squares coefficient fitting
- 03
Fitted values and residual signs
- 04
Multiple regression interpretation
- 05
Categorical indicator variables
- 06
Interaction effects and hierarchy
- 07
Quadratic and nonlinear terms
- 08
Training fit compared with future value
Worked example · free
Compute a fitted order count
- +1The marks attached to these steps support self-checking and are not a University published assessment scheme. Translate seven hundred contacts to x = 7 because the predictor unit is hundreds.
- +1Substitute to obtain 18.4 + 2.6 × 7 = 36.6 fitted orders.
- +1Compute residual as observed minus fitted: 41 − 36.6 = 4.4.
- +1Interpret the positive residual: the model underpredicted this observation by 4.4 orders.
- +1Interpret the slope as 2.6 additional fitted orders per extra hundred contacts without making an unsupported causal claim.
Key terms
- Fitted Value
- The response predicted by the estimated regression equation for specified predictors.
- Residual
- The observed response minus its fitted value for one case.
- Least Squares
- The coefficient rule that minimises the sum of squared residuals.
- Indicator Variable
- A numerical representation comparing a categorical level with a reference level.
- Interaction Effect
- A term allowing one predictor slope to depend on another predictor.
- Hierarchy Principle
- The practice of retaining relevant main effects when their interaction is included.
- Reference Level
- The categorical baseline against which displayed indicator coefficients compare fitted response differences.
Least Squares and Linear Regression FAQ
How should I interpret a slope?
State the response change, predictor unit and conditioning phrase. In multiple regression, the slope is the fitted response change for one predictor unit while the other included predictors are held fixed.
What does a positive residual mean?
Because residual equals observed minus fitted, a positive residual means the observed response exceeds the model prediction. The model underpredicted that case by the residual amount in response units.
Why keep main effects with an interaction?
An interaction changes a slope according to another variable. Keeping the corresponding main effects preserves the lower-order structure and makes the resulting conditional effects interpretable across reference values.
Why might I centre a predictor?
Centring can make the intercept describe the fitted response at a typical predictor value rather than at an irrelevant zero. It does not alter fitted values when the model is re-expressed consistently.
What should a residual plot reveal?
Look for systematic curvature, changing vertical spread, clusters or influential observations. A pattern suggests the fitted mean or error structure misses something, while a clean plot supports but does not prove the model assumptions.
Exam move
Practise regression as a four-line routine: write the fitted equation, substitute predictors with their units, calculate the fitted value, and compute observed minus fitted. Draw a scatterplot with one fitted line and several residual segments, then use the picture to explain why residuals have signs and why least squares squares them.
Create original examples with one categorical indicator, one interaction and one quadratic term. For each, write a full sentence interpreting the relevant coefficient or combined effect. Do not memorise a main-effect sentence that ignores an interaction. Use the solved problem-set style to practise confidence intervals and benchmark tests later, but keep this chapter focused on fit and interpretation.
End by comparing two models that have different training residual sums and explain why the smaller training number does not prove better future prediction.
Working through Least Squares and Linear Regression in QBUS2820? Sia is AskSia’s AI Statistics tutor — ask any QBUS2820 Least Squares and Linear Regression question and get a clear, step-by-step explanation grounded in how QBUS2820 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
