CVEN9407 Chap.5 Two-Variable Regression and Ordinary Least Squares
Two-Variable Regression and Ordinary Least Squares
This chapter fits the first regression line of the course by hand. It defines regression as the study of conditional means, separates the population regression function from the sample line that estimates it, and explains the role of the stochastic error term.
Least squares is derived from the goal of minimising squared residuals, and a ten-household trip-generation example is worked from a full table of sums to the slope, the intercept and R squared. The chapter closes with what R squared can and cannot tell you.
The two-variable model is small enough to compute on paper, which is why it is the place to understand what software does in every later regression.
The same logic of conditional means, residuals and squared-error minimisation carries straight into the multiple regression of crash frequency that forms the first project deliverable, and the Week 2 demonstration runs the same fit in R.
What this chapter covers
- 01
Regression as a line of conditional means
- 02
Population and sample regression functions
- 03
The stochastic error term and the residual
- 04
Least squares and the normal equations
- 05
Slope and intercept formulas
- 06
Total, explained and residual sums of squares
Worked example · free
Fitting a line from five summary totals
- 1Means: X bar = 20 / 5 = 4 and Y bar = 35 / 5 = 7.
- 1Deviation sums: sum of x squared = 90 - 5(16) = 10 and sum of xy = 150 - 5(4)(7) = 10.
- 1Slope = 10 / 10 = 1.0 and intercept = 7 - 1.0(4) = 3, so the fitted line is Y hat = 3 + 1.0X.
Key terms
- Population regression function
- The population regression function gives the expected value of the dependent variable for each value of the explanatory variable.
- Stochastic disturbance
- A stochastic disturbance is the deviation of an individual observation from its conditional mean in the population model.
- Normal equations
- The normal equations are the first-order conditions of least squares, solved together to give the intercept and slope.
- Total sum of squares
- The total sum of squares is the summed squared deviation of the dependent variable from its mean, split into explained and residual parts.
- Sample regression function
- The sample regression function is the fitted line computed from one sample, used to estimate the population regression function.
- Explained sum of squares
- The explained sum of squares is the variation of the fitted values around the mean of the dependent variable.
Two-Variable Regression and Ordinary Least Squares FAQ
Why are residuals squared rather than simply added?
Positive and negative residuals would cancel out, so many different lines could make their sum zero. Squaring removes the sign and gives more weight to large misses, which produces a unique best-fitting line.
Does a high R squared mean the model is good?
Not on its own. R squared only measures how much variation the line explains in this sample. A model can have a high value and still have biased coefficients, misbehaving errors or no causal meaning.
What is the difference between the error term and the residual?
The error term is the unobservable deviation from the population line. The residual is the observable deviation from the fitted sample line, and it serves as the estimate of the error term.
Why does the fitted line always pass through the point of means?
The first normal equation forces the residuals to sum to zero, which can only happen if the fitted value at the mean of X equals the mean of Y. That property also gives a quick check on any hand calculation.
How do I read the slope in its units?
The slope is the change in the expected value of Y for a one-unit increase in X, in Y's units per X's units. In a trips-on-vehicles model, a slope of 0.7 means about 0.7 extra daily trips per extra vehicle, on average across households.
Assessment move
Fit at least two small datasets by hand using the shortcut formulas, then check them in R or Colab so you trust both. Keep four decimals until the final line. For each fit, state the slope in words with its units, and say what share of the variation the line explains. Then fit the same line in R or Colab and match every number on the output to the formula that produced it.
Working through Two-Variable Regression and Ordinary Least Squares in CVEN9407? Sia is AskSia’s AI Engineering tutor — ask any CVEN9407 Two-Variable Regression and Ordinary Least Squares question and get a clear, step-by-step explanation grounded in how CVEN9407 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
