STAT7038 Regression Modelling
The Complete Exam Bible · S1 2026STAT7038 Overview
- Australian National University
- S1 2026
- 7 chapters
- Science
Regression Modelling teaches the classical linear model end to end — from fitting one straight line through a cloud of points to building, testing and choosing a multiple-regression model with several predictors.
- Assessed by Final examination 70% · Assignment 15% · In-tutorial quiz 10% · Online quiz 5%
- Key terms Least-squares estimator, ANOVA decomposition, Confidence interval vs prediction interval, Leverage (hat value)
- How to prepare Build your one A4 sheet around the recurring chains, because every exam item is a procedure on supplied numbers.
- Most asked What is allowed in the STAT7038 exam?
How STAT7038 is assessed
| Component | Weight | Format |
|---|---|---|
| Final examination | 70% | MCQ + short calculation + short written · covers Weeks 1–12 · open to one A4 double-sided typed/printed sheet; calculator, R outputs & statistical tables supplied |
| Assignment | 15% | R-based, non-redeemable — submitted across the semester |
| In-tutorial quiz | 10% | Redeemable (the exam mark replaces it if higher) — on simple linear regression |
| Online quiz | 5% | Redeemable, on Canvas — confirm the exact split & dates in your subject guide |
What STAT7038 covers
Seven topics, from one straight line to many predictors → one exam-ready map. Each links to its free chapter guide.
Simple Linear Regression
The model · LINE assumptions · least squares · b₁ = Sxy/Sxx · σ² = MSESimple linear regression models a response y as a straight-line function of one predictor x plus a random, normally distributed error of constant size — in symbols yi = β0 + β1xi + εi.
- The SLR model and the four LINE assumptions
- What the slope and intercept mean — read it aloud
- Least squares: the normal equations and b₁ = Sxy/Sxx, b₀ = ȳ − b₁x̄
- Fitted values, residuals, and the two residual identities
- Estimating the error variance: σ̂² = MSE = SSE/(n−2)
- Properties of the LS estimators: unbiasedness, variances, Gauss–Markov BLUE
Regression Inference
ANOVA · the F-test & R² · t-tests & CIs · CI vs PI · reading R outputOnce the line is fitted, inference cross-examines it.
- The ANOVA identity SST = SSR + SSE and the df bookkeeping
- The overall F-test F = MSR/MSE
- R² = SSR/SST and why R² = r²xy in SLR (F = t²)
- t-tests for a coefficient and the five-step ritual
- Standard errors and confidence intervals for β₀, β₁
- CI for the mean response vs PI for a new observation — the '+1'
Regression Diagnostics
Residuals vs fitted · Q–Q · outlier ≠ leverage ≠ influence · transformationsLeast squares always returns a line — even through data that has no business being modelled linearly — and the t-tests, F-tests, CIs and PIs are only valid when the error assumptions hold.
- Why diagnostics: inference is only valid if LINE holds
- The residuals-vs-fitted plot: curvature (L) and funnel (E)
- The normal Q–Q plot for the normality assumption
- Reading R's four plot(lm) panels
- Outlier ≠ leverage ≠ influence — three distinct ideas
- Leverage hₖₖ, studentised residuals, and influence measures
Matrix Form & Multiple Regression
y = Xβ · (X'X)⁻¹X'y · the hat matrix · partial coefficients · adjusted R²Writing the n equations one at a time is clumsy; stacking them into vectors and matrices collapses the entire least-squares machinery into a handful of expressions that work for any number of predictors.
- The stacked model y = Xβ + ε and the design matrix X
- The least-squares solution β̂ = (X'X)⁻¹X'y for any number of predictors
- The hat matrix H: symmetric, idempotent, diagonal = leverages
- The variance of the estimates, Var(β̂) = σ²(X'X)⁻¹, and reading vcov(lm)
- The MLR model and the 'partial' (holding-others-constant) coefficient
- Overall F vs individual t — two different questions
Multiple Regression Inference
Sequential vs partial SS · the nested F-test · dummies & interactionsIn multiple regression the regression sum of squares can be sliced up in more than one way, and knowing which slice answers which question is the most-tested idea in this half of the course.
- The MLR ANOVA decomposition and the overall F
- Sequential (Type I) sums of squares — order matters
- Partial sums of squares and the extra-sum-of-squares principle
- The partial (nested) F-test; q = 1 collapses to the t-test
- Qualitative covariates: indicator coding and the reference level
- The additive (parallel-lines) model: β₂ as the group shift
Advanced Diagnostics
Studentised & PRESS residuals · Cook's D, DFFITS, DFBETAS · VIF & κThis chapter sharpens the diagnostic toolkit for the multiple-regression setting and adds the dedicated treatment of multicollinearity.
- Three ideas kept straight: outlier, leverage, influence
- Types of residuals: raw, standardised, studentised, PRESS
- The studentised-residual outlier test and the Bonferroni correction
- Influence measures: Cook's D, DFFITS, DFBETAS, COVRATIO with cut-offs
- Reading an influence.measures() table and the never-auto-delete rule
- Multicollinearity: inflated SEs, the F/t split, sign-flips
Model Selection
Adjusted R² · Mallows' Cp · AIC · BIC · PRESS · stepwise · parsimonyAdding predictors can never make R² worse, so R² alone always picks the biggest model and is useless for comparison.
- Why R² cannot rank models of different size
- Adjusted R², Mallows' Cp, AIC, BIC, PRESS — formulas and directions
- Cp ≈ p as a bias gauge; PRESS as leave-one-out prediction
- Selection procedures: best-subset, forward, backward, stepwise
- Reading a step() trace: lowest resulting AIC wins; <none> stops
- The parsimony / bias-variance trade-off
You will estimate coefficients by least squares, split the variation with ANOVA, test slopes with t and the model with F, separate a confidence interval for the mean from a prediction interval for a new observation, check the assumptions with residual diagnostics, read everything off an R printout, and finish with model selection.
The final exam is 70% of your grade; it is open to one A4 double-sided typed sheet and supplies the calculator, the R outputs and the statistical tables — so it tests whether you can execute and interpret the method on fresh numbers, not whether you can recall it.
Worked example · free
The slope t-test & 95% CI — the signature SLR question, mark by mark
- +1Standard error of the slope. se(b1) = √(MSE / Sxx) = √(0.806 / 42) = 0.139.
- +1Hypotheses. H0: β1 = 0 (no linear relationship) vs Ha: β1 ≠ 0.
- +1Test statistic. t = b1 / se(b1) = 3.833 / 0.139 = 27.68, compared to tn−2 = t6.
- +1Critical value & decision. t6(0.975) = 2.447 (supplied table); |27.68| > 2.447 ⇒ reject H0.
- +195% CI for β1. 3.833 ± 2.447 × 0.139 = 3.833 ± 0.339 = (3.49, 4.17).
- +1Conclude in context. The slope is significantly non-zero; we are 95% confident each extra study hour adds between 3.49 and 4.17 marks.
Key terms
- Least-squares estimator
- The coefficients that minimise the total squared vertical distance from the points to the line. In simple regression b1 = Sxy/Sxx and b0 = ŷ − b1x̄; under the LINE assumptions they are unbiased and, by Gauss–Markov, the best linear unbiased estimators (BLUE).
- ANOVA decomposition
- The exact split of total variation in y into the part the line explains and the part it leaves: SST = SSR + SSE, with degrees of freedom (n−1) = (p−1) + (n−p). It drives the F-test and R².
- Confidence interval vs prediction interval
- Both are centred at the fitted value ŷh, but a CI brackets the mean response at xh while a PI brackets one new observation. The PI carries an extra ‘+1’ under the root (the new point’s own error), so it is always wider.
- Leverage (hat value)
- hii measures how far an observation’s x-values sit from the centre of the predictor space; it is the i-th diagonal of the hat matrix H = X(X'X)⁻¹X'. High leverage (hii > 2p/n) is only dangerous when paired with a large residual.
- Multicollinearity
- Near-linear dependence among the predictors, which inflates the variances of the coefficients. Its tell-tale sign is a highly significant overall F with no individual t significant; diagnosed by the variance inflation factor, VIF = 1/(1−R²j), flagged above 5 (serious above 10).
STAT7038 FAQ
Is STAT7038 hard?
It is method-dense rather than memory-heavy. Because the exam supplies the R outputs and the statistical tables, the difficulty is in driving the procedure under time — picking the right interval, the right critical value and the right test, then reading and interpreting the output correctly. Master simple linear regression and the multiple-regression half is largely the same results in matrix clothing.
How is STAT7038 assessed?
The final exam is 70% of your grade; the rest is an R-based assignment (about 15%, non-redeemable), an in-tutorial quiz (about 10%, redeemable) and an online quiz (about 5%, redeemable). Confirm this year’s exact split and dates in your subject guide, as details shift between cohorts.
What is allowed in the STAT7038 exam?
The final is open to one A4 double-sided, typed or printed notes sheet, and the paper itself supplies the calculator, the R outputs and the statistical tables (t, F, normal). So you do not waste sheet space on table values or R syntax — spend it on the boxed formulas, the five-step test ritual, the CI-vs-PI rule and the R-output map. Confirm the permitted-materials rule for your sitting in the subject guide.
Do I need to run R in the STAT7038 exam?
No — you read supplied summary() and anova() printouts rather than running R yourself. The skill being tested is mapping each cell (Estimate, Std. Error, t value, Pr(>|t|), the F-statistic) to a formula and recovering hidden quantities such as MSE and n.
Is using AskSia for STAT7038 cheating?
No. AskSia is a study reference written in our own words — we host none of your lecturer’s files, and Sia teaches you the method to earn the marks; it does not complete or sit your assessments.
Questions students ask in each STAT7038 chapter
- Why divide SSE by n − 2 instead of n?
- Two degrees of freedom are spent estimating the two parameters β0 and β1 before the residuals are formed, so only n−2 independent pieces of information about the error remain. Dividing by n−2 makes MSE an unbiased estimator of σ²; dividing by n (or n−1) underestimates it and biases every standard error, t-statistic and interval that uses MSE. Chapter 01 · Simple Linear Regression
- When do I use a confidence interval and when a prediction interval?
- Match the wording. 'The average score for students who study 7 hours' is a CI for the mean. 'Predict the score of one student who studies 7 hours' is a PI. Both are centred at ŷh, but the PI adds the new observation's own error (the +1 under the root) and is always wider. Picking the wrong interval, or dropping the +1, is the single most-marked simple-regression error. Chapter 02 · Regression Inference
- What is the difference between an outlier, a high-leverage point and an influential point?
- They answer different questions. An outlier is unusual in y given its x (a large residual, detected by the studentised ti). High leverage is unusual in x-space, far from x̄ (a large hii). An influential point is one whose removal moves the fit (Cook's D, DFFITS, DFBETAS). A point can be any combination — the rule is influence ≈ outlier × leverage, so it takes both a big residual and high leverage to be influential. Chapter 03 · Regression Diagnostics
- Why is β̂ = (X'X)⁻¹X'y worth memorising?
- It is the same least-squares solution for simple and multiple regression — only the width of the design matrix X changes. In simple regression it reduces algebraically to b1 = Sxy/Sxx and b0 = ŷ − b1x̄. The exam asks you to read matrix output, not to invert by hand, but you must know what each factor is and that it exists only when X'X is invertible (no exact collinearity). Chapter 04 · Matrix Form & Multiple Regression
- Why do the same predictors in a different order give different anova() significance?
- Because anova() is sequential: each line conditions only on the terms above it, so SSR(X2 | X1) depends on the order. Re-ordering changes the per-line sums of squares and verdicts. The summary() t-tests, by contrast, are partial (each term last, given all the others) and barely change. Two anova() tables that disagree under reordering are themselves a symptom of multicollinearity. Chapter 05 · Multiple Regression Inference
- What is the multicollinearity giveaway?
- A highly significant overall F with no individual t significant. The model jointly explains a lot, but no single predictor can be credited because they overlap. Partner symptoms: implausible coefficient signs or signs that flip when other predictors enter, very wide CIs, and sequential anova() verdicts that swing under reordering. When you see the F/t split, immediately check vif() and the correlation matrix. Chapter 06 · Advanced Diagnostics
- Why can't I just use R² to pick the best model?
- Adding any predictor — even pure noise — cannot raise SSE, so R² only ever climbs and always favours the biggest model. It cannot referee a comparison. Adjusted R² divides the sums of squares by their degrees of freedom, so it falls when an added term does not pull its weight; Mallows' Cp, AIC, BIC and PRESS each penalise size in their own way. Use those, not plain R². Chapter 07 · Model Selection
How to study for the exam
Build your one A4 sheet around the recurring chains, because every exam item is a procedure on supplied numbers. Drill four of them until they are automatic: Sxy/Sxx → b1, b0; SSE → MSE → se(bj) → t → decision; SST = SSR + SSE → F, R²; and xh → CI (mean) or PI (new obs).
Show every line on the written parts — method marks are real. Keep the high-yield distinctions sharp: CI vs PI (the ‘+1’), outlier ≠ leverage ≠ influence, and sequential vs partial sums of squares. Practise reading an R printout cold, and remember the multicollinearity giveaway: a significant overall F with no significant individual t.
