ECON7880 Chap.6 Support Vector Machines and Margins
Support Vector Machines and Margins
Classify by the side of a hyperplane
A linear support vector machine predicts from the sign of a fitted score and uses a margin to separate classes robustly.
The points nearest or inside that margin are support vectors and carry the local evidence determining the boundary.
Read hinge loss geometrically
Actual sign times fitted score separates safe correct points, fragile correct points and misclassifications.
The C parameter controls the trade-off between margin width and tolerated violations, with feature scaling required before coefficient size is meaningful.
Separate prediction from margin status
The score sign alone gives the predicted class. Margin status needs the actual label multiplied by the score.
A product at least one is correctly placed on or beyond the margin, a product between zero and one is correct but fragile, and a negative product is misclassified. Hinge loss is max(0, 1 - y f(x)), converting these geometric regions into an optimisation objective.
Understand why support vectors matter
Distant correctly classified observations do not directly determine the separating line.
Points on or inside the margins do. Inspect them for duplicate records, uncertain labels or unusual feature combinations because small changes to those cases may move the boundary.
The support-vector count is diagnostic rather than a target: a large share can reflect overlapping classes, weak predictors or strong regularisation.
Scale before comparing geometry
A predictor measured in thousands can dominate a distance-based boundary beside one measured in fractions. Fit scaling parameters on training data only and place the scaler inside each validation fold.
Raw score magnitudes are comparable only within the same fitted and scaled model. Report the scaling method with C and class coding so another analyst can reproduce the sign and loss interpretation.
Use C in the correct direction
In the course convention, larger C means weaker regularisation and less tolerance of hinge loss, often yielding larger coefficients and a narrower margin.
Smaller C accepts more violations to obtain a wider and simpler boundary. Compare candidate values with cross-validation and keep the linear model as a baseline before adding a nonlinear kernel whose flexibility requires additional tuning.
What this chapter covers
- 01
Separating hyperplanes
- 02
Score sign
- 03
Parallel margins
- 04
Support vectors
- 05
Hinge loss
- 06
C and regularisation strength
Worked example · free
Calculate three margin violations
- 3Multiply each label by its score.
- 3Apply the positive-part loss.
- 3Classify margin status.
Key terms
- Separating Hyperplane
- A linear boundary whose fitted score is zero and whose two sides receive different class labels.
- Margin Violation
- An observation lying inside its class margin or on the wrong side of the separating boundary.
- C Parameter
- An inverse-regularisation parameter where larger values penalise loss more strongly and often narrow the margin.
Support Vector Machines and Margins FAQ
What makes an observation a support vector?
It lies on or inside a margin and therefore directly influences the fitted boundary. Distant correctly classified points usually do not move the separator because they already satisfy the margin condition comfortably.
How is hinge loss interpreted?
Zero means correct and on or beyond the margin. A value between zero and one means correct but within the margin, while a value above one indicates a misclassification on the wrong side.
Why scale predictors for an SVM?
The margin and coefficient penalty depend on predictor units. Without scaling, a feature’s numerical scale can dominate geometry and regularisation for the wrong reason. Fit the scaler on training folds only and report it with the selected C value. Otherwise the margin cannot be reproduced reliably on another data sample.
Exam move
Draw a two-feature boundary with margins and place examples in every hinge-loss region. Compute label-times-score and hinge loss for each point, then identify which cases are support vectors. Rescale one feature and explain why geometry changes unless preparation is standardised. Compare smaller and larger C in words before checking validation evidence.
Finish with a short report containing class coding, scaling, selected C, support-vector count, fold performance and the costly error at the operating decision boundary. Move a distant correct point and predict whether the separator changes; then move a support vector and explain why the answer differs.
Compare a linear boundary with a flexible kernel on the same folds and demand held-out improvement before accepting extra complexity. Rehearse the inverse relationship between C and regularisation until the direction is automatic. Count safe points, margin violations and misclassifications on validation data, then explain what those regions reveal about robustness.
Working through Support Vector Machines and Margins in ECON7880? Sia is AskSia’s AI Economics tutor — ask any ECON7880 Support Vector Machines and Margins question and get a clear, step-by-step explanation grounded in how ECON7880 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
