ETX5900 · Business Statistics
Regression Model Analysis
Regression Model Analysis is Week 10 of ETX5900 Business Statistics at Monash University (Berenson Ch 13), the inference half of simple linear regression. Week 9 fitted the line Ŷ = b₀ + b₁X and read the correlation; this topic decomposes the variation as SST = SSR + SSE so that R² = SSR/SST measures the fit, runs the slope t-test (H₀: β₁ = 0, t = b₁/se(b₁), df = n−2) to decide whether a linear relationship really exists, and uses the line to predict — inside the observed range only.
What this chapter covers
- 01The estimated line Ŷ = b₀ + b₁X and the least-squares slope b₁ = Σ(x−x̄)(y−ȳ)/Σ(x−x̄)²
- 02Partitioning variation: SST = SSR + SSE (total = explained + residual)
- 03Coefficient of determination R² = SSR/SST = 1 − SSE/SST, and R² = r² in simple regression
- 04The slope t-test H₀: β₁ = 0 vs H₁: β₁ ≠ 0, two-tailed, testing whether a linear relationship exists
- 05Test statistic t = b₁/se(b₁) with df = n − 2, and se(b₁) = √(MSE/Σ(x−x̄)²), MSE = SSE/(n−2)
- 06Reject rule: reject H₀ if |t| > t_{n−2,α/2} (equivalently p-value < α)
- 07Confidence interval for β₁: b₁ ± t_{n−2,α/2}·se(b₁), and its agreement with the test (excludes 0 ⇒ reject)
- 08Prediction from Ŷ = b₀ + b₁x; interpolation (safe) vs extrapolation beyond the range (risky)
- 09Interpreting the slope in context and R² as a share — not proof of linearity or causation
- 10Reading a regression output (Coefficients, Standard Error, t Stat, p-value) and the provided t-table
Fit the line, measure R², then test whether the slope is significant
- +1Means. x̄ = 36/6 = 6 and ȳ = 300/6 = 50.
- +2Slope and intercept. Σ(xᵢ−x̄)(yᵢ−ȳ) = 80+24+4+5+27+70 = 210 and Σ(xᵢ−x̄)² = 25+9+1+1+9+25 = 70, so b₁ = 210/70 = 3 and b₀ = ȳ − b₁x̄ = 50 − 3·6 = 32. The line is Ŷ = 32 + 3X: each extra year adds ≈ $3,000.
- +1Prediction. At X = 8 (inside the observed 1–11 range, so interpolation) Ŷ = 32 + 3·8 = 56, i.e. predicted revenue ≈ $56,000.
- +2Sums of squares. SST = Σ(yᵢ−ȳ)² = 256+64+16+25+81+196 = 638; SSR = b₁·Σ(xᵢ−x̄)(yᵢ−ȳ) = 3·210 = 630; SSE = SST − SSR = 638 − 630 = 8.
- +1Coefficient of determination. R² = SSR/SST = 630/638 = 0.987, so the line explains about 98.7% of the variation in revenue (r = +√0.987 = +0.994).
- +2Slope test — hypotheses and statistic. H₀: β₁ = 0 vs H₁: β₁ ≠ 0 (two-tailed), α = 0.05. MSE = SSE/(n−2) = 8/4 = 2, se(b₁) = √(2/70) = 0.1690, so t = b₁/se(b₁) = 3/0.1690 = 17.75 with df = n − 2 = 4.
- +1Critical value and decision. Two-tailed at α = 0.05, df = 4: t_{4,0.025} = 2.776. Since |17.75| > 2.776 (p < 0.001), reject H₀ — there is a significant positive linear relationship between experience and revenue.
Key terms
- Estimated regression line (Ŷ = b₀ + b₁X)
- The least-squares line, with intercept b₀ = ȳ − b₁x̄ and slope b₁ = Σ(x−x̄)(y−ȳ)/Σ(x−x̄)². b₀, b₁ are sample estimates of the population parameters β₀, β₁.
- SST, SSR, SSE
- Total sum of squares SST = Σ(y−ȳ)² splits into the explained SSR = Σ(Ŷ−ȳ)² and the residual SSE = Σ(y−Ŷ)², with SST = SSR + SSE. Least squares is the choice of line that minimises SSE.
- Coefficient of determination (R²)
- R² = SSR/SST = 1 − SSE/SST, the share of the variation in Y explained by the line, between 0 and 1. In simple linear regression R² equals the square of the correlation r.
- Population slope β₁
- The true change in mean Y per one-unit rise in X. If β₁ = 0 the line is flat and X carries no linear information about Y; the slope t-test decides whether the estimate b₁ is far enough from 0 to reject β₁ = 0.
- Standard error of the slope se(b₁)
- How much the estimated slope b₁ would vary across samples: se(b₁) = √(MSE / Σ(x−x̄)²), where MSE = SSE/(n−2) is the residual variance. It is the denominator of the slope t-statistic.
- Slope t-test
- Test of H₀: β₁ = 0 vs H₁: β₁ ≠ 0 using t = b₁/se(b₁) with df = n − 2; reject H₀ if |t| > t_{n−2,α/2} (or p-value < α). It tests whether a linear relationship exists at all, and gives the same result as testing ρ = 0.
- Interpolation vs extrapolation
- Interpolation predicts Ŷ at an x inside the observed range (safe, because the pattern was seen there). Extrapolation predicts beyond the range, assuming the straight line continues where there is no evidence — an unreliable forecast.
- Degrees of freedom (n − 2)
- The df for slope inference in simple regression, because two parameters (b₀ and b₁) are estimated from the n data points. Using n − 1 reads the wrong t-table row.
Regression Model Analysis FAQ
Why does the slope t-test use df = n − 2 and not n − 1?
Because simple linear regression estimates two parameters from the data — the intercept b₀ and the slope b₁ — so two degrees of freedom are used up, leaving n − 2 for the residual variance MSE = SSE/(n−2) and for reading the t-table. (A one-sample mean test estimates a single parameter, so it uses n − 1.) Using n − 1 here reads the wrong row of the t-table and can flip a borderline reject / fail-to-reject decision.
Does a high R² mean the slope is statistically significant?
Not on its own. R² measures how much of the variation in Y the line explains, while the slope t-test measures whether β₁ differs from 0 at all — they answer different questions. With a large sample even a small R² can give a significant slope, and with a very small sample a high R² may not reach significance. Report both, and remember that neither R² nor a significant slope proves that X causes Y.
Can AI help me with regression model analysis in ETX5900?
Yes — Sia can explain the method step by step: how SST splits into SSR and SSE, why R² = SSR/SST, and how to run the slope t-test (why df = n − 2, how se(b₁) comes from MSE and Σ(x−x̄)², and how to read the provided t-table for the two-tailed critical value). Work your own practice numbers through it to build the routine. Sia explains and coaches; it does not sit your exam or promise guaranteed answers, grades or a pass.
Exam move
Practise the whole chain on one small dataset until it is automatic: compute x̄ and ȳ, then b₁ = Σ(x−x̄)(y−ȳ)/Σ(x−x̄)² and b₀ = ȳ − b₁x̄, and write Ŷ = b₀ + b₁X. Get the decomposition SST = SSR + SSE right (the shortcut SSR = b₁·Σ(x−x̄)(y−ȳ) saves time) so R² = SSR/SST falls out. Then drill the slope t-test as a five-step hypothesis test — H₀: β₁ = 0 vs H₁: β₁ ≠ 0, t = b₁/se(b₁), df = n − 2, reject if |t| > t_{n−2,α/2} — always writing the explicit reject / fail-to-reject decision and one context sentence, and check it against the 95% CI for β₁. Because the exam provides the formula sheet and statistical tables, practise reading the t-table at the right α/2 and df rather than memorising values, and rehearse interpreting the slope in units and R² as a share while never extrapolating beyond the data or claiming causation.
Working through Regression Model Analysis in ETX5900? Sia is AskSia’s AI Statistics tutor — ask any ETX5900 Regression Model Analysis question and get a clear, step-by-step explanation grounded in how ETX5900 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
