QBUS2820 Chap.9 Classification and Logistic Regression
Classification and Logistic Regression
Classification predicts categories by estimating probabilities or applying decision boundaries. This chapter introduces the Bayes classifier, KNN class voting, binary logistic regression, log-odds, odds ratios and multinomial extensions. It separates model output from action: a probability is converted into a class only after a threshold is chosen.
Since a cutoff of 0.5 assumes a particular error trade-off, class imbalance and unequal consequences require decision-aware threshold selection rather than automatic majority rules. Probability estimation and class assignment should be evaluated separately. Calibration asks whether cases assigned a probability near a value experience the event at about that rate.
Discrimination asks whether positive cases tend to receive higher scores than negative cases. A threshold then adds costs, benefits and operational capacity. Logistic coefficients live on the log-odds scale, and odds ratios are multiplicative rather than percentage-point effects. Representative probability calculations can make those effects understandable without losing the exact model statement.
KNN classification provides a contrasting local probability estimate based on neighbour proportions and inherits the scaling and dimensionality issues of KNN regression. Class prevalence matters: a sample created by oversampling positives can help fitting but may distort raw probability calibration unless corrected.
Multinomial settings require a coherent extension rather than several unrelated binary decisions when categories compete. Monitoring should track calibration and error types as prevalence or behaviour changes.
What this chapter covers
- 01
Conditional class probabilities
- 02
Bayes classifier decisions
- 03
KNN class voting
- 04
Logistic probability curves
- 05
Log-odds equations
- 06
Odds ratio interpretation
- 07
Probability cutoffs
- 08
Class imbalance and calibration
Worked example · free
Convert a logistic score to action
- +1The marks used in this probability drill are not an official University assessment scheme. Compute η = −2.0 + 0.8 × 3 = 0.4.
- +1Compute p = exp(0.4)/(1 + exp(0.4)) = 0.5987.
- +1At cutoff 0.5, classify the case as a response.
- +1At cutoff 0.7, classify the same case as no response.
- +1exp(0.8) = 2.23, so one more purchase multiplies fitted odds by about 2.23, holding other predictors fixed.
Key terms
- Bayes Classifier
- The probability-based rule with minimum theoretical error under equal misclassification costs.
- Decision Boundary
- The predictor-space location where the assigned class changes.
- Log-Odds
- The logarithm of probability divided by one minus probability.
- Odds Ratio
- The multiplicative change in fitted odds for a one-unit predictor increase.
- Probability Cutoff
- The threshold converting a fitted probability into a class action.
- Calibration
- Agreement between predicted probabilities and observed event frequencies.
- Positive Class
- The event class used to define true positives, sensitivity and other directional classification metrics.
Classification and Logistic Regression FAQ
Why use logistic regression instead of a line?
The logistic link keeps fitted probabilities between zero and one and expresses predictor effects linearly on log-odds. A plain linear probability fit can produce impossible values.
How do I interpret an odds ratio?
Exponentiate the coefficient. The result multiplies fitted odds for a one-unit predictor increase while other included predictors are held fixed. It is not a constant probability-point change.
Why might a 0.5 cutoff be unsuitable?
The threshold ignores asymmetric error costs, class imbalance, review capacity and business value. Select a cutoff from validated consequences rather than treating 0.5 as universally optimal.
What is the difference between calibration and discrimination?
Calibration concerns whether predicted probabilities match observed frequencies. Discrimination concerns ranking positives above negatives. A classifier can perform well on one dimension and poorly on the other.
How does oversampling affect probabilities?
Oversampling a rare positive class can help the fitting algorithm see enough examples, but the resulting raw probabilities may reflect the altered sample prevalence. Recalibrate or correct them for deployment conditions.
Exam move
Work logistic examples as three separate operations: compute the linear score, transform it to probability, then apply a stated threshold. Practise converting coefficients into odds ratios and explain why the corresponding probability change depends on the starting risk. Draw a logistic curve with a horizontal cutoff and mark the decision boundary.
Compare logistic regression with KNN classification: one imposes a smooth parametric log-odds structure, while the other estimates local class proportions. Create an imbalanced example where predicting the majority class gives high accuracy but zero useful detection. Add a calibration question by grouping cases with similar predicted probabilities and comparing observed rates.
End every solution by naming the positive class, threshold, intended action and evaluation population so the mathematical probability cannot be mistaken for a complete business decision.
Working through Classification and Logistic Regression in QBUS2820? Sia is AskSia’s AI Statistics tutor — ask any QBUS2820 Classification and Logistic Regression question and get a clear, step-by-step explanation grounded in how QBUS2820 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
