INFS3822 Chap.6 Regression and Classification
Regression and Classification
Linear regression predicts a numeric outcome with an interpretable fitted relationship, while logistic regression estimates the probability of a binary outcome. This chapter distinguishes forecasts from class decisions, interprets coefficients with their units and limits, computes residuals and shows why a probability threshold belongs to business policy. It also separates association in a model from causal evidence.
What this chapter covers
- 01
Continuous numeric targets
- 02
Binary probability outputs
- 03
Linear coefficients and units
- 04
Logistic coefficient signs
- 05
Residual interpretation
- 06
Decision thresholds
- 07
Regression and classification metrics
Worked example · free
Forecast workload and apply a threshold
- +1Substitute 800: 18 + 0.035 x 800 = 18 + 28 = 46 predicted hours.
- +1At 0.70, 0.68 is below the threshold, so the class is negative.
- +1At 0.60, 0.68 meets the threshold, so the class is positive; lowering the threshold creates more positive decisions.
Key terms
- Linear regression
- A model that predicts a continuous numeric target through a fitted linear relationship.
- Logistic regression
- A classification model that maps a weighted score to a probability between zero and one.
- Residual
- The difference between an observed numeric outcome and its fitted prediction under a stated sign convention.
- Decision threshold
- The probability cutoff used to turn a score into an operational class.
Regression and Classification FAQ
Why is logistic regression used for classification?
It transforms a weighted combination of features into a probability between zero and one. That probability can be retained for ranking or compared with a threshold to produce a class. The threshold is chosen from operational and error costs rather than learned as a universal truth.
How should a linear coefficient be interpreted?
State the expected change in the numeric prediction for a one-unit increase in that feature, holding other included features constant. Include units and avoid causal language unless the study design supports it. Coefficient magnitude comparisons also require attention to feature scales.
What happens when a classification threshold falls?
More observations are labelled positive. Recall often rises because fewer actual positives are missed, while false positives and review workload can rise and precision can fall. The preferred threshold depends on intervention cost, available capacity and the harm of a missed positive.
How can I check a calculation before interpreting it?
Verify units, range and direction. A regression output should use the target's units and a residual should follow the declared sign convention. A sigmoid probability must lie between zero and one. On the same scores, lowering the positive threshold cannot reduce the number classified positive. These checks catch arithmetic and denominator mistakes before they shape a business conclusion.
Assessment move
Practise one linear substitution, one residual and one threshold comparison together. Label every number as target units, probability or class. Then write a restrained interpretation that says associated rather than caused. Finish each threshold exercise by naming the false-positive and false-negative consequences for the business.
For regression, annotate the intercept, each coefficient, input units, predicted unit and valid range. Substitute values line by line and calculate the residual as observed minus predicted using one convention consistently. Interpret a coefficient as a conditional model association, not proof that changing the input will cause the target to move.
Inspect residual patterns for nonlinearity, changing spread and unusual cases; a single error summary can hide all three. Compare the fitted model with a simple baseline such as the training mean and evaluate on unseen rows. For binary classification, keep three objects separate: a score, an estimated probability and a thresholded class.
Logistic regression maps a linear score through a sigmoid, but the cut-off is a decision setting rather than a parameter that must always equal 0.50. Recalculate predicted classes at two thresholds and write which cases change. Then link the movement to false positives, false negatives and review volume. Build a four-cell confusion matrix from a short list of predictions before using memorised formulas.
Label the predicted-positive denominator for precision and the actual-positive denominator for recall. Explain when accuracy is misleading because one class dominates. Use a compact comparison table for mean squared error, accuracy, precision, recall, F1 and ROC/AUC: what each summarises, what denominator or scale it uses, and what it does not decide. Add calibration when probability magnitude matters.
Finish with a mixed business case in which a regression forecast estimates workload and a classifier prioritises cases. Explain why their outputs, metrics and actions differ even if both models use the same source table. This prevents formulas from becoming detached from the task they answer.
Working through Regression and Classification in INFS3822? Sia is AskSia’s AI Computer Science tutor — ask any INFS3822 Regression and Classification question and get a clear, step-by-step explanation grounded in how INFS3822 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
