MATH2089 Chap.13 Linear Regression
Linear Regression
Simple linear regression models a mean response as a line plus random error. Least squares estimates intercept and slope by minimising squared residuals, while inference uses standard errors and a t distribution under the error assumptions.
This chapter interprets coefficients with units, separates fitted values from observations, diagnoses curvature and changing spread through residual plots, distinguishes mean response from new observation intervals and warns against causal language and extrapolation. The model says that the conditional mean of the response changes linearly with the predictor.
The intercept is the fitted mean response at predictor zero, and the slope is the fitted change in mean response for a one-unit increase in the predictor. Both need units. The intercept may lack practical meaning when zero is outside the observed range, even though it remains part of the algebraic line. Least squares chooses coefficients that minimise the sum of squared vertical residuals.
A fitted value lies on the line and a residual is observed response minus fitted value. Residual signs show whether observations lie above or below the fitted relation. With an intercept, residuals sum to zero, but that identity alone does not show that the linear model is suitable. Inference for the slope uses its standard error and a t reference distribution under the regression error conditions.
A slope test against zero asks about a linear association, while a confidence interval shows plausible changes in mean response per predictor unit. A small p-value is not a measure of the association's size and does not establish causation. Causal interpretation requires a design that supports it, not only a fitted equation. Residual diagnostics examine model structure.
Curvature in residuals suggests a nonlinear mean relation. A funnel shape suggests changing variance. Sequential patterns may indicate dependence. A normal quantile plot helps assess the normal-error condition for small-sample inference. High-leverage observations have unusual predictor values and can strongly influence the slope even when their residuals are not the largest.
Investigate them rather than removing them automatically. Prediction has two targets. A confidence interval for the mean response estimates the average outcome at a chosen predictor value. A prediction interval for one new observation also includes individual error variation and is therefore wider. Both are most defensible inside the observed predictor range.
Extrapolation assumes the linear pattern continues where no data checked it and must be labelled. R squared summarises a proportion of response variation accounted for by the fitted line in the sample, but does not validate assumptions, prove causation or replace effect size.
A complete report gives the fitted equation, coefficient interpretation, uncertainty, residual checks, prediction target and a scope statement tied to the design and predictor range.
What this chapter covers
- 01
Simple linear model
- 02
Least squares coefficients
- 03
Fitted values and residuals
- 04
Slope inference
- 05
Residual diagnostics
- 06
Confidence and prediction intervals
Worked example · free
Read a fitted line
- +1The fitted value is 3+1.5 times 4, which equals 9.
- +1Residual is observed minus fitted, 10-9=1.
- +1The observation lies one unit above the fitted mean line.
- +1The slope estimates a 1.5 unit mean response increase per one unit of x within the observed range.
Key terms
- Regression slope
- The expected change in mean response for a one unit predictor increase.
- Residual plot
- A diagnostic graph of residuals used to assess mean form and variance behaviour.
- Prediction interval
- An interval for one new response that includes individual variation and mean uncertainty.
Linear Regression FAQ
What does R squared prove?
It measures the proportion of sample response variation explained by the fitted linear relation. It does not prove causation, validate assumptions or guarantee accurate extrapolation.
What does a curved residual plot mean?
Systematic curvature challenges the linear mean form. A nonlinear term, transformation or different model may be needed before the fitted coefficient inference can be trusted.
Why is a prediction interval wider?
A mean response interval covers uncertainty about the fitted average. A prediction interval also includes the natural variation of one new observation, so it must be wider.
Exam move
Work one small regression by hand to keep fitted values and residuals concrete, then practise reading coefficient estimates, standard errors and intervals from software output. Pair every regression output with a residual plot and a normal quantile plot, and flag any prediction outside the observed x range. Write slope units as response units per predictor unit and decide whether the intercept has a useful contextual meaning.
For one fitted point, calculate the residual and locate it vertically on the scatterplot. Create diagnostic sketches showing curvature, a funnel and an influential high-leverage observation, then name the model concern in each. Compare mean-response and new-observation intervals at the same predictor value.
Finish by stating association unless the design independently supports causation, and treat R squared as a summary rather than an assumption check.
Working through Linear Regression in MATH2089? Sia is AskSia’s AI Mathematics tutor — ask any MATH2089 Linear Regression question and get a clear, step-by-step explanation grounded in how MATH2089 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.