ETX5900 Chap.10 Regression Model Analysis
Regression Model Analysis
Regression Model Analysis is Week 10 of ETX5900 Business Statistics at Monash University (Berenson Ch 13), the inference half of simple linear regression.
Week 9 fitted the line Ŷ = b₀ + b₁X and read the correlation; this topic decomposes the variation as SST = SSR + SSE so that R² = SSR/SST measures the fit, runs the slope t-test (H₀: β₁ = 0, t = b₁/se(b₁), df = n−2) to decide whether a linear relationship really exists, and uses the line to predict — inside the observed range only.
What this chapter covers
- 01
The estimated line Ŷ = b₀ + b₁X and the least-squares slope b₁ = Σ(x−x̄)(y−ȳ)/Σ(x−x̄)²
- 02
Partitioning variation: SST = SSR + SSE (total = explained + residual)
- 03
Coefficient of determination R² = SSR/SST = 1 − SSE/SST, and R² = r² in simple regression
- 04
The slope t-test H₀: β₁ = 0 vs H₁: β₁ ≠ 0, two-tailed, testing whether a linear relationship exists
- 05
Test statistic t = b₁/se(b₁) with df = n − 2, and se(b₁) = √(MSE/Σ(x−x̄)²), MSE = SSE/(n−2)
- 06
Reject rule: reject H₀ if |t| > t_{n−2,α/2} (equivalently p-value < α)
- 07
Confidence interval for β₁: b₁ ± t_{n−2,α/2}·se(b₁), and its agreement with the test (excludes 0 ⇒ reject)
- 08
Prediction from Ŷ = b₀ + b₁x; interpolation (safe) vs extrapolation beyond the range (risky)
- 09
Interpreting the slope in context and R² as a share — not proof of linearity or causation
- 10
Reading a regression output (Coefficients, Standard Error, t Stat, p-value) and the provided t-table
Fit the line, measure R², then test whether the slope is significant
- +1Means. x̄ = 36/6 = 6 and ȳ = 300/6 = 50.
- +2Slope and intercept. Σ(xᵢ−x̄)(yᵢ−ȳ) = 80+24+4+5+27+70 = 210 and Σ(xᵢ−x̄)² = 25+9+1+1+9+25 = 70, so b₁ = 210/70 = 3 and b₀ = ȳ − b₁x̄ = 50 − 3·6 = 32. The line is Ŷ = 32 + 3X: each extra year adds ≈ $3,000.
- +1Prediction. At X = 8 (inside the observed 1–11 range, so interpolation) Ŷ = 32 + 3·8 = 56, i.e. predicted revenue ≈ $56,000.
- +2Sums of squares. SST = Σ(yᵢ−ȳ)² = 256+64+16+25+81+196 = 638; SSR = b₁·Σ(xᵢ−x̄)(yᵢ−ȳ) = 3·210 = 630; SSE = SST − SSR = 638 − 630 = 8.
- +1Coefficient of determination. R² = SSR/SST = 630/638 = 0.987, so the line explains about 98.7% of the variation in revenue (r = +√0.987 = +0.994).
- +2Slope test — hypotheses and statistic. H₀: β₁ = 0 vs H₁: β₁ ≠ 0 (two-tailed), α = 0.05. MSE = SSE/(n−2) = 8/4 = 2, se(b₁) = √(2/70) = 0.1690, so t = b₁/se(b₁) = 3/0.1690 = 17.75 with df = n − 2 = 4.
- +1Critical value and decision. Two-tailed at α = 0.05, df = 4: t_{4,0.025} = 2.776. Since |17.75| > 2.776 (p < 0.001), reject H₀ — there is a significant positive linear relationship between experience and revenue.
Key terms
- Estimated regression line (Ŷ = b₀ + b₁X)
- The least-squares line, with intercept b₀ = ȳ − b₁x̄ and slope b₁ = Σ(x−x̄)(y−ȳ)/Σ(x−x̄)². b₀, b₁ are sample estimates of the population parameters β₀, β₁.
- SST, SSR, SSE
- Total sum of squares SST = Σ(y−ȳ)² splits into the explained SSR = Σ(Ŷ−ȳ)² and the residual SSE = Σ(y−Ŷ)², with SST = SSR + SSE. Least squares is the choice of line that minimises SSE.
- Coefficient of determination (R²)
- R² = SSR/SST = 1 − SSE/SST, the share of the variation in Y explained by the line, between 0 and 1. In simple linear regression R² equals the square of the correlation r.
- Population slope β₁
- The true change in mean Y per one-unit rise in X. If β₁ = 0 the line is flat and X carries no linear information about Y; the slope t-test decides whether the estimate b₁ is far enough from 0 to reject β₁ = 0.
- Standard error of the slope se(b₁)
- How much the estimated slope b₁ would vary across samples: se(b₁) = √(MSE / Σ(x−x̄)²), where MSE = SSE/(n−2) is the residual variance. It is the denominator of the slope t-statistic.
- Slope t-test
- Test of H₀: β₁ = 0 vs H₁: β₁ ≠ 0 using t = b₁/se(b₁) with df = n − 2; reject H₀ if |t| > t_{n−2,α/2} (or p-value < α). It tests whether a linear relationship exists at all, and gives the same result as testing ρ = 0.
- Interpolation vs extrapolation
- Interpolation predicts Ŷ at an x inside the observed range (safe, because the pattern was seen there). Extrapolation predicts beyond the range, assuming the straight line continues where there is no evidence — an unreliable forecast.
- Degrees of freedom (n − 2)
- The df for slope inference in simple regression, because two parameters (b₀ and b₁) are estimated from the n data points. Using n − 1 reads the wrong t-table row.
Regression Model Analysis FAQ
Why does the slope t-test use df = n − 2 and not n − 1?
Because simple linear regression estimates two parameters from the data — the intercept b₀ and the slope b₁ — so two degrees of freedom are used up, leaving n − 2 for the residual variance MSE = SSE/(n−2) and for reading the t-table. (A one-sample mean test estimates a single parameter, so it uses n − 1.) Using n − 1 here reads the wrong row of the t-table and can flip a borderline reject / fail-to-reject decision.
Does a high R² mean the slope is statistically significant?
Not on its own. R² measures how much of the variation in Y the line explains, while the slope t-test measures whether β₁ differs from 0 at all — they answer different questions. With a large sample even a small R² can give a significant slope, and with a very small sample a high R² may not reach significance. Report both, and remember that neither R² nor a significant slope proves that X causes Y.
Can AI help me with regression model analysis in ETX5900?
Yes — Sia can explain the method step by step: how SST splits into SSR and SSE, why R² = SSR/SST, and how to run the slope t-test (why df = n − 2, how se(b₁) comes from MSE and Σ(x−x̄)², and how to read the provided t-table for the two-tailed critical value). Work your own practice numbers through it to build the routine. Sia explains and coaches; it does not sit your exam or promise guaranteed answers, grades or a pass.
Exam move
Practise the whole chain on one small dataset until it is automatic: compute x̄ and ȳ, then b₁ = Σ(x−x̄)(y−ȳ)/Σ(x−x̄)² and b₀ = ȳ − b₁x̄, and write Ŷ = b₀ + b₁X. Get the decomposition SST = SSR + SSE right (the shortcut SSR = b₁·Σ(x−x̄)(y−ȳ) saves time) so R² = SSR/SST falls out.
Then drill the slope t-test as a five-step hypothesis test — H₀: β₁ = 0 vs H₁: β₁ ≠ 0, t = b₁/se(b₁), df = n − 2, reject if |t| > t_{n−2,α/2} — always writing the explicit reject / fail-to-reject decision and one context sentence, and check it against the 95% CI for β₁.
Because the exam provides the formula sheet and statistical tables, practise reading the t-table at the right α/2 and df rather than memorising values, and rehearse interpreting the slope in units and R² as a share while never extrapolating beyond the data or claiming causation.
Working through Regression Model Analysis in ETX5900? Sia is AskSia’s AI Statistics tutor — ask any ETX5900 Regression Model Analysis question and get a clear, step-by-step explanation grounded in how ETX5900 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
