BME5150 Chap.4 Regression Error, Bias and Variance
Regression Error, Bias and Variance
Residuals preserve direction
A residual is the observed outcome minus the fitted prediction. Positive and negative values show which side of the prediction line a case falls on. Simply averaging signed residuals allows cancellation, so evaluation uses absolute values, squares or other transformations.
A residual plot can reveal curvature, changing variance and subgroup patterns that a single summary hides.
Error metrics punish misses differently
Mean absolute error averages absolute residuals and remains in the outcome unit. Mean squared error squares residuals, giving large misses extra influence. Root mean squared error returns that loss to the original unit while preserving the emphasis.
Mean absolute percentage error supports scale comparison but becomes unstable near zero. Choose from the consequence of error, not convenience.
Correlation and explained variation are narrower claims
Correlation measures linear association and direction. R-squared describes the fraction of observed outcome variation accounted for by the fitted linear model under its assumptions.
Neither proves causation, unbiased predictions or external transport. Adjusted R-squared penalises added predictors but still does not replace held-out testing.
Bias and variance use two error views
High training and validation errors suggest a model or representation too rigid for the signal. Low training error with much higher validation error suggests overfitting and high variance.
Remedies differ: revisit representation or complexity for underfitting, and strengthen regularisation, simplify or add representative data for overfitting.
The final test set remains untouched
Use development resampling to choose model form and tuning. Once a test result changes the model, that test set is part of development.
The final evaluation should describe the locked pipeline and the population on which the error was measured.
What this chapter covers
- 01
Observed and predicted values
- 02
MAE, MSE and RMSE
- 03
Percentage errors and near-zero outcomes
- 04
Correlation and R squared
- 05
Bias and variance patterns
- 06
Residual diagnostics and held-out testing
Worked example · free
Compare absolute and squared errors
- 3Calculate signed and absolute residuals.
- 4Calculate MAE, MSE and RMSE.
- 2Interpret the effect of the largest miss.
Key terms
- Residual
- A residual is an observed outcome minus its predicted value.
- Mean Absolute Error
- Mean absolute error averages residual magnitudes in the outcome unit.
- Root Mean Squared Error
- Root mean squared error is the square root of average squared residuals and emphasises large misses.
- R Squared
- R squared is the proportion of observed variation accounted for by a fitted linear model.
- High Variance
- High variance is sensitivity to development data that appears as much worse held-out than training performance.
Regression Error, Bias and Variance FAQ
When should I prefer RMSE to MAE?
Prefer RMSE when large errors deserve extra influence and that weighting matches the clinical consequence. Prefer MAE for a linear, unit-preserving summary that is less dominated by extremes. Reporting both can reveal whether a small number of large misses drive performance.
Does a high R squared mean predictions are clinically accurate?
No. R squared summarises explained variation in the fitted data under a linear model. Predictions can still be biased, poorly calibrated or unreliable outside the development population. Inspect residuals and evaluate the locked model on held-out cases.
How do I distinguish underfitting from overfitting?
Compare training and held-out errors. High errors on both sides suggest high bias or missing structure. Low training error with substantially higher held-out error suggests high variance. Diagnose the pattern before choosing a remedy such as new features or stronger regularisation.
Assessment move
Create three small residual lists: one balanced around zero, one with a large outlier and one with a systematic positive shift. Compute MAE and RMSE for each and explain which pattern the averages hide. Next, draw a two-axis table with training error and validation error, then place underfitting and overfitting in the correct cells.
For every regression result in your notes, write the outcome unit beside MAE or RMSE and a population beside the held-out value. Finish by checking whether a reported R-squared statement has accidentally become a causal or generalisation claim. Sketch residuals against fitted values for the three lists and name the visible pattern before suggesting a remedy. This trains the move from summary error to structural diagnosis.
Working through Regression Error, Bias and Variance in BME5150? Sia is AskSia’s AI Computer Science tutor — ask any BME5150 Regression Error, Bias and Variance question and get a clear, step-by-step explanation grounded in how BME5150 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
