QBUS2820 Chap.8 Lasso and Elastic Net
Lasso and Elastic Net
Lasso uses an absolute-value coefficient penalty, allowing the fitted solution to land exactly on zero for some predictors. This creates sparse models but can produce unstable selections among correlated variables. Ridge tends to spread weight; lasso tends to choose; elastic net combines squared and absolute penalties to support both shrinkage and grouped behaviour.
Group lasso extends the idea to conceptual blocks such as categorical indicators. Scaling, resampling and stability checks remain essential because algorithmic exclusion is not proof of no population relationship. Sparsity changes the communication burden but does not eliminate uncertainty. A compact lasso equation can be easier to deploy, yet its selected set may vary when predictors are correlated or the sample is modest.
Selection frequencies across folds reveal whether the apparent explanation is stable. Elastic net introduces a mixing parameter as well as lambda, so both must be tuned inside the resampling design. At one end the behaviour resembles ridge; at the other it approaches lasso.
Group penalties are valuable when several columns encode one conceptual feature, such as a categorical variable or a spline basis, because independent deletion can leave a fragment that is hard to interpret. Standardisation and intercept treatment follow the same discipline as ridge. Post-selection coefficients are shrunken and chosen through a search, so ordinary inferential statements require caution.
Nearby tuning values should be examined to see whether prediction remains stable while the selected story changes. That contrast is itself a useful result.
What this chapter covers
- 01
Absolute coefficient penalties
- 02
Sparse fitted solutions
- 03
Lasso constraint geometry
- 04
Correlated predictor instability
- 05
Elastic net mixing
- 06
Group lasso penalties
- 07
Joint tuning of alpha and lambda
- 08
Selection stability across resamples
Worked example · free
Choose a penalty family
- +1The marks in this comparison guide revision and are not a University published marking scheme. Confirm all methods use the same folds, loss and training-only scaling.
- +1Elastic net has the smallest validation error at 8.1.
- +1Lasso's changing selections signal instability among correlated inputs.
- +1Elastic net can retain related groups while preserving some sparsity.
- +1Recommend elastic net, then refit and evaluate on untouched test cases.
Key terms
- Lasso Penalty
- Lambda times the sum of absolute slope values added to residual fit.
- Sparse Model
- A fitted model in which some candidate coefficients are exactly zero.
- Elastic Net
- A penalty combining absolute and squared coefficient components.
- Group Lasso
- A penalty designed to include or exclude related coefficient blocks together.
- Mixing Parameter
- The elastic-net value controlling the balance between lasso and ridge penalties.
- Selection Stability
- The consistency with which predictors remain selected across resampled datasets.
- Selection Frequency
- The proportion of resampled fits in which a predictor retains a nonzero coefficient under the chosen procedure.
Lasso and Elastic Net FAQ
Why can lasso set coefficients to zero?
The absolute-value constraint has corners on coefficient axes. A likelihood or residual contour can first touch a corner, producing an optimum with an exact zero slope.
When is elastic net preferable?
Elastic net is useful when correlated predictors form meaningful groups and some sparsity is still desired. Its squared component encourages grouped behaviour while its absolute component can remove weak features.
Does a zero lasso coefficient prove no effect?
No. It means the predictor was excluded by this fitted sample, candidate set and penalty. Correlated alternatives, sampling variation and tuning choices can change that result.
Why inspect selection frequencies?
A single sparse fit hides sampling variability. Counting how often features remain nonzero across folds or bootstrap samples shows whether the selected explanation is reproducible or interchangeable with correlated alternatives.
What does the elastic-net mixing value control?
It sets the balance between absolute and squared penalties. Values nearer the lasso end favour sparsity, while more ridge contribution encourages correlated predictors to shrink together.
Exam move
Draw the ridge circle and lasso diamond on the same coefficient axes and use their geometry to explain why lasso reaches exact zeros. Write the lasso and elastic-net objectives, naming lambda and the mixing parameter. Create a correlated-predictor example where different folds select different representatives, then calculate selection frequencies.
Compare ridge, lasso and elastic net using held-out loss, sparsity and stability rather than one property alone. For a categorical variable expanded into indicators, explain why group lasso may preserve the factor's meaning better than independent penalties. Keep preprocessing inside every fold and retain an untouched final test set.
When writing an interpretation, call zero an algorithmic exclusion, not evidence of a population zero. Finish by documenting the penalty grid and how the chosen sparse structure behaves under nearby tuning values.
Working through Lasso and Elastic Net in QBUS2820? Sia is AskSia’s AI Statistics tutor — ask any QBUS2820 Lasso and Elastic Net question and get a clear, step-by-step explanation grounded in how QBUS2820 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
