IS6400 Chap.10 Model Assessment and Generative AI
Model Assessment and Generative AI
Selection requires one fixed protocol
Compare candidates on the same row definition, target, horizon, split, preprocessing and metric. Validation supports tuning, while one final holdout estimates performance after choices are complete. Repeatedly consulting the holdout turns it into another training signal.
Generative systems need labelled failure categories
Fluency is not factual support.
Evaluate task completion, groundedness, safety, consistency and reviewer effort separately on ordinary, boundary and adversarial cases. Preserve prompts, allowed evidence, model version and corrections so a failure can be traced and monitored.
Begin with counts and a baseline
For classification, reconstruct the confusion matrix at the intended threshold and name the positive event.
Accuracy can look excellent when the costly class is rare, so report precision, recall and workload with their denominators. For regression, compare MAE or another justified loss by horizon and segment against a naive rule.
Keep the row definition, split and preprocessing identical so a difference belongs to the model rather than the evaluation setup.
Price errors before choosing a threshold
Translate false positives into review effort and false negatives into missed-event harm, then test whether the queue fits capacity. Vary uncertain costs to see whether the preferred model reverses under a small change.
Report uncertainty across folds or time windows and calibration when scores are treated as probabilities. A slightly weaker average may be safer when its segment performance is stable and its failures are easier to detect and repair.
Evaluate generative assistance as a workflow
Build reviewed cases that cover ordinary, boundary and adversarial requests.
Score factual support, instruction completion, safety, consistency, escalation and human correction time separately; do not average a harmless style preference with a fabricated figure. Define the evidence the system may use and log prompts, retrieved context, model version and corrections. Monitoring then tracks severity and workload, while a named escalation path handles unsupported content or missing evidence.
What this chapter covers
- 01
Training, validation and final holdout
- 02
Baseline comparisons
- 03
Metrics under class imbalance
- 04
Calibration and threshold cost
- 05
Generative task completion
- 06
Groundedness, review effort and monitoring
Worked example · free
Select a threshold under rare events
- 3Construct the confusion counts. The marks used here are not a University assessment scheme.
- 3Compute the three rates.
- 3Name the operational evidence still needed.
Key terms
- Validation Set
- Held-out data used to compare choices and tune a modelling pipeline.
- Final Holdout
- Data preserved from iterative decisions and consulted once for a final performance estimate.
- Calibration
- Agreement between predicted score levels and observed event frequencies.
- Groundedness
- The extent to which a generated claim is supported by the evidence the system was permitted to use.
- Adversarial Case
- An intentionally difficult input designed to expose boundary, safety or instruction-following failures.
Model Assessment and Generative AI FAQ
Why keep a final holdout separate from validation?
Validation results influence choices, so repeated use gradually adapts the pipeline to that data. A final untouched set provides a less biased estimate after model and threshold decisions are finished.
Which metric is best for an imbalanced classification problem?
No metric is universally best. Start with confusion counts, identify the costly error, compare precision and recall at feasible thresholds, check calibration if scores are used as probabilities, and include operating capacity.
How should a generative business tool be tested?
Build reviewed cases spanning ordinary, boundary and adversarial requests. Score factual support, instruction completion, safety and correction effort separately, and retain the prompt, evidence, version and human edit for traceability.
Exam move
Create one comparison table with baseline, candidate, split, horizon, metric, threshold, cost and monitoring owner. Recompute confusion counts for rare-event examples until accuracy no longer distracts from the harmful error. For generative AI, write a small evaluation rubric whose categories do not collapse into one average: factual support, task completion, safety, consistency and reviewer time.
Add one boundary case and one adversarial case for every normal case you design. Vary the threshold and price both error types under a fixed capacity limit, then test whether the chosen model changes when costs move within a plausible range. Report uncertainty across folds or time windows and reserve one final holdout.
For a language-model workflow, record the allowed evidence, prompt class, retrieved context, correction and escalation outcome so a fluent but unsupported answer is counted by severity rather than hidden inside an average quality score. Finish with a selection receipt naming rejected alternatives, uncertainty, costly error, capacity limit, monitoring owner and evidence that would reopen the decision.