ECON7880 Chap.5 Logistic Regression and Probability
Logistic Regression and Probability
Transform a score into probability
Logistic regression represents log-odds with a linear predictor and uses the sigmoid to keep estimated probabilities between zero and one. This chapter moves explicitly from score to odds to probability before any class label is assigned.
Choose thresholds from consequences
A threshold is an operating rule, not part of the fitted probability.
Lowering it usually raises recall and false positives, while log loss evaluates probability quality and penalises confident mistakes without committing to one cutoff.
Keep four mathematical objects separate
The linear predictor z can take any real value. Its exponential gives odds, and the sigmoid p = 1/(1 + exp(-z)) gives probability. A threshold then converts probability into a label.
Score zero corresponds to odds one and probability one half. Changing the threshold changes the label and confusion counts but leaves the fitted score and probability unchanged.
Interpret coefficients through odds
A coefficient adds to log-odds for a one-unit predictor increase while other included predictors are held fixed. Exponentiating it produces an odds multiplier.
Probability does not rise by a constant number of percentage points because the sigmoid is steepest near its centre and flat near the ends. Use a realistic baseline case, update the odds and convert back to probability when communicating practical magnitude.
Evaluate probabilities before selecting action
Log loss rewards probability assigned to the observed class and heavily penalises confident errors.
Compare it on identical holdout cases and against a base-rate predictor. Calibration asks whether groups assigned a probability near a stated value experience the outcome at a similar frequency. Ranking can remain useful when calibration is weak, but cost forecasts require credible probability levels.
Report the operating point completely
Select a threshold from capacity and error costs using validation data.
Report the flagged workload, precision, recall, false positives and false negatives together. Class prevalence can shift after deployment and change precision even when ranking quality stays stable, so monitor both prevalence and threshold performance. A statistically sound probability model can still support an unusable action rule if its queue exceeds capacity.
What this chapter covers
- 01
Linear classification scores
- 02
Log-odds and odds
- 03
Sigmoid probabilities
- 04
Decision thresholds
- 05
Odds multipliers
- 06
Binary log loss
Worked example · free
Convert a score into an operating decision
- 2Evaluate the linear score.
- 3Convert score to probability.
- 3Apply and compare thresholds.
Key terms
- Sigmoid Function
- The S-shaped transformation mapping any real linear score into a probability between zero and one.
- Odds Multiplier
- The exponentiated coefficient showing how odds change for a one-unit predictor increase.
- Log Loss
- A probability loss that penalises confident predictions heavily when they assign low probability to the observed class.
Logistic Regression and Probability FAQ
Why not use linear regression for class probability?
An unconstrained line can predict below zero or above one. Logistic regression models log-odds linearly and maps the result into the legal probability range through the sigmoid function.
Does a positive coefficient add a fixed probability?
No. It adds a fixed amount to log-odds and multiplies odds by a fixed factor. The probability-point change depends on the starting case and the values of other included predictors held fixed.
Why can two teams label the same case differently?
They may apply different thresholds to the same fitted probability because their capacities or error costs differ. Report the threshold and flagged workload whenever labels or confusion metrics are compared, while keeping the underlying probability estimate visible for both teams. This distinguishes model from policy disagreement and makes the operational trade-off reviewable.
Exam move
Practise the chain from score to odds to probability to label. Use a positive, zero and negative score, then change only the threshold and explain which quantities remain fixed. Convert one coefficient into an odds multiplier and apply it to two different baseline probabilities to see why percentage-point effects differ. Calculate log loss for a confident correct prediction and a confident mistake.
Finish by choosing an operating threshold for a fixed queue and reporting precision, recall and workload without describing the default cutoff as automatically optimal. Build a five-case calibration table containing predicted probabilities and observed outcomes, then explain why such a tiny table cannot establish calibration even though it illustrates the calculation.
Reverse the positive-class coding and trace how coefficient signs and labels change while the underlying cases remain the same. Build a three-threshold table and select one from stated capacity rather than the most attractive isolated metric.
Working through Logistic Regression and Probability in ECON7880? Sia is AskSia’s AI Economics tutor — ask any ECON7880 Logistic Regression and Probability question and get a clear, step-by-step explanation grounded in how ECON7880 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
