26134 · Responsible Evidence-based Decisions
Modelling Relationships: Regression Analysis
Week 10 is Module 4: simple linear regression. You read a scatterplot, fit the least-squares line, interpret the slope and intercept, assess goodness of fit with R², and reason about confounding and the difference between correlation and a causal effect. Fitting or interpreting a regression line, and reading R², is a standard exam item.
What this chapter covers
- 01The model yᵢ = β₀ + β₁xᵢ + εᵢ and the fitted line ŷ = b₀ + b₁x
- 02Least-squares estimation: minimise Σ(yᵢ − ŷᵢ)²
- 03Slope b₁ = r·(s_y/sₓ) = Σ(xᵢ − x̄)(yᵢ − ȳ)/Σ(xᵢ − x̄)²
- 04Intercept b₀ = ȳ − b₁x̄; the line passes through (x̄, ȳ)
- 05Interpreting the slope (change in y per unit x) and intercept (y at x = 0)
- 06Goodness of fit: R² = SSR/SST = 1 − SSE/SST; in simple regression R² = r²
- 07Residuals eᵢ = yᵢ − ŷᵢ and why data scatter around the line
- 08Confounding / omitted variables; correlation is not causation; multiple regression holds other factors constant
Fitting and reading a least-squares line from summary statistics
- +1Slope. b₁ = r·(s_y/sₓ) = 0.8 × (15/4) = 0.8 × 3.75 = 3.0. Each extra $1,000 of advertising is associated with about 3 more units of sales.
- +1Intercept. b₀ = ȳ − b₁x̄ = 50 − 3.0 × 10 = 50 − 30 = 20. The fitted line is ŷ = 20 + 3x.
- +1Prediction at x = 12. ŷ = 20 + 3 × 12 = 20 + 36 = 56 units.
- +1Goodness of fit. In simple regression R² = r² = 0.8² = 0.64, so about 64% of the variation in sales is explained by advertising spend; the remaining 36% is residual variation.
Key terms
- Simple linear regression model
- yᵢ = β₀ + β₁xᵢ + εᵢ, where y is the response, x the predictor, β₀ the intercept, β₁ the slope, and ε a random error absorbing everything x does not explain. The fitted version is ŷ = b₀ + b₁x.
- Least-squares estimation (OLS)
- Chooses b₀ and b₁ to minimise the sum of squared residuals Σ(yᵢ − ŷᵢ)². This yields b₁ = Σ(xᵢ − x̄)(yᵢ − ȳ)/Σ(xᵢ − x̄)² and b₀ = ȳ − b₁x̄, the line through (x̄, ȳ).
- Slope and intercept
- The slope b₁ is the estimated change in y per one-unit increase in x; the intercept b₀ is the predicted y when x = 0. Interpret the slope in the data's units, and treat the intercept cautiously if x = 0 is outside the data range.
- Residual
- eᵢ = yᵢ − ŷᵢ, the vertical gap between an observed value and the fitted line. Residuals exist because real data scatter around the line; least squares makes their squared sum as small as possible.
- Coefficient of determination (R²)
- R² = SSR/SST = 1 − SSE/SST, the proportion of variation in y explained by the model, between 0 and 1. In simple regression R² equals the square of the correlation, r².
- Confounding / omitted variable
- A third factor that influences y and is related to x; ignoring it biases the estimated slope. Multiple regression adds such variables to isolate the effect of one predictor holding the others constant — the 'everything-else-equal' interpretation.
Modelling Relationships: Regression Analysis FAQ
How do I interpret the slope and intercept?
The slope b₁ is the estimated change in y for each one-unit increase in x, stated in the variables' units (e.g. '3 more units of sales per extra $1,000 spent'). The intercept b₀ is the predicted y when x = 0; treat it literally only if x = 0 is within or near the observed data, otherwise it is just where the line meets the axis.
What does R² tell me, and what is a 'good' value?
R² is the share of variation in y explained by the model, from 0 to 1; R² = 0.64 means 64% is explained and 36% is residual. There is no universal 'good' threshold — it depends on the field. In simple regression R² = r², so a strong correlation implies a high R².
Does a regression slope prove that x causes y?
No. A fitted slope quantifies an association, but confounding variables or reverse causation can produce it without x causing y. Establishing causation needs controlled design or careful methods; multiple regression can help by holding other measured factors constant, but it does not by itself prove cause and effect.
Can AI help me with regression in 26134?
Yes, as a study aid. Sia can walk you through computing the slope and intercept from summary statistics, making a prediction, interpreting R², and explaining confounding, step by step, while checking your working. Use it to rehearse the method; it does not do your graded assessment, and the UTS academic-integrity policy applies.
Exam move
Regression questions are formulaic, so drill the sequence: slope b₁ = r·(s_y/sₓ), intercept b₀ = ȳ − b₁x̄, write ŷ = b₀ + b₁x, predict at a given x, then state R² = r² with a one-line interpretation. Practise interpreting the slope in the variables' own units and flagging that association is not causation — that responsible-interpretation point is examinable. Use the fact that the line passes through (x̄, ȳ) as a self-check. Keep the slope, intercept and R² formulas on your printed exam notes, along with the Excel regression output layout. When the slope formula or the R²-versus-r distinction trips you, ask Sia to re-derive it on a fresh dataset; confirm assessment details on Canvas.
Working through Modelling Relationships: Regression Analysis in 26134? Sia is AskSia’s AI Statistics tutor — ask any 26134 Modelling Relationships: Regression Analysis question and get a clear, step-by-step explanation grounded in how 26134 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
