City University of Hong Kong · FACULTY OF INFORMATION TECHNOLOGY

IS6400 Chap.10 Model Assessment and Generative AI

- one subject, every graph, every model, every mark
6 Chapters3-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 10 of 10 · IS6400

Model Assessment and Generative AI

Selection requires one fixed protocol

Compare candidates on the same row definition, target, horizon, split, preprocessing and metric. Validation supports tuning, while one final holdout estimates performance after choices are complete. Repeatedly consulting the holdout turns it into another training signal.

Generative systems need labelled failure categories

Fluency is not factual support.

Evaluate task completion, groundedness, safety, consistency and reviewer effort separately on ordinary, boundary and adversarial cases. Preserve prompts, allowed evidence, model version and corrections so a failure can be traced and monitored.

Begin with counts and a baseline

For classification, reconstruct the confusion matrix at the intended threshold and name the positive event.

Accuracy can look excellent when the costly class is rare, so report precision, recall and workload with their denominators. For regression, compare MAE or another justified loss by horizon and segment against a naive rule.

Keep the row definition, split and preprocessing identical so a difference belongs to the model rather than the evaluation setup.

Price errors before choosing a threshold

Translate false positives into review effort and false negatives into missed-event harm, then test whether the queue fits capacity. Vary uncertain costs to see whether the preferred model reverses under a small change.

Report uncertainty across folds or time windows and calibration when scores are treated as probabilities. A slightly weaker average may be safer when its segment performance is stable and its failures are easier to detect and repair.

Evaluate generative assistance as a workflow

Build reviewed cases that cover ordinary, boundary and adversarial requests.

Score factual support, instruction completion, safety, consistency, escalation and human correction time separately; do not average a harmless style preference with a fabricated figure. Define the evidence the system may use and log prompts, retrieved context, model version and corrections. Monitoring then tracks severity and workload, while a named escalation path handles unsupported content or missing evidence.

In this chapter

What this chapter covers

  • 01

    Training, validation and final holdout

  • 02

    Baseline comparisons

  • 03

    Metrics under class imbalance

  • 04

    Calibration and threshold cost

  • 05

    Generative task completion

  • 06

    Groundedness, review effort and monitoring

Worked example · free

Select a threshold under rare events

Q [9 marks]. AskSia-authored practice. The listed marks are AskSia study allocations, not a University rubric. In 500 cases, 25 events occur. A model flags 40 cases and catches 20 events. Compute precision, recall and accuracy, then state which evidence is missing for deployment.
All four cells belong in one selection decision.All four cells belong in one selection decision. Held-out performance and error distribution become useful only after costs, capacity and monitoring are specified.Model selection joins statistical evidence to business consequenceheld-out performancesame horizon and uniterror distributionsegments and regimesdecision costthreshold and capacitymonitoring plandrift and escalation
All four cells belong in one selection decision. Held-out performance and error distribution become useful only after costs, capacity and monitoring are specified.
  • 3Construct the confusion counts. The marks used here are not a University assessment scheme.
  • 3Compute the three rates.
  • 3Name the operational evidence still needed.
True positives are 20, false positives 20, false negatives 5 and true negatives 455. Precision is 20/40 = 0.50, recall is 20/25 = 0.80, and accuracy is 475/500 = 0.95. The high accuracy mostly reflects rare events. Deployment still needs the cost of five misses, the capacity to review forty alerts, calibration or threshold evidence on held-out periods, segment results and a monitoring trigger. Selection should minimise decision cost under capacity rather than maximise accuracy.
Sia tip — Under imbalance, calculate counts and minority-event recall before allowing accuracy into the discussion.
Glossary

Key terms

Validation Set
Held-out data used to compare choices and tune a modelling pipeline.
Final Holdout
Data preserved from iterative decisions and consulted once for a final performance estimate.
Calibration
Agreement between predicted score levels and observed event frequencies.
Groundedness
The extent to which a generated claim is supported by the evidence the system was permitted to use.
Adversarial Case
An intentionally difficult input designed to expose boundary, safety or instruction-following failures.
FAQ

Model Assessment and Generative AI FAQ

Why keep a final holdout separate from validation?

Validation results influence choices, so repeated use gradually adapts the pipeline to that data. A final untouched set provides a less biased estimate after model and threshold decisions are finished.

Which metric is best for an imbalanced classification problem?

No metric is universally best. Start with confusion counts, identify the costly error, compare precision and recall at feasible thresholds, check calibration if scores are used as probabilities, and include operating capacity.

How should a generative business tool be tested?

Build reviewed cases spanning ordinary, boundary and adversarial requests. Score factual support, instruction completion, safety and correction effort separately, and retain the prompt, evidence, version and human edit for traceability.

Study strategy

Exam move

Create one comparison table with baseline, candidate, split, horizon, metric, threshold, cost and monitoring owner. Recompute confusion counts for rare-event examples until accuracy no longer distracts from the harmful error. For generative AI, write a small evaluation rubric whose categories do not collapse into one average: factual support, task completion, safety, consistency and reviewer time.

Add one boundary case and one adversarial case for every normal case you design. Vary the threshold and price both error types under a fixed capacity limit, then test whether the chosen model changes when costs move within a plausible range. Report uncertainty across folds or time windows and reserve one final holdout.

For a language-model workflow, record the allowed evidence, prompt class, retrieved context, correction and escalation outcome so a fluent but unsupported answer is counted by severity rather than hidden inside an average quality score. Finish with a selection receipt naming rejected alternatives, uncertainty, costly error, capacity limit, monitoring owner and evidence that would reopen the decision.

A+Everything unlocked
Unlocks this Bible + all 31 of your subjects at City University of Hong Kong, plus 1,000+ more Bibles across the AskSia library.
Sia - your IS6400 tutor, unlimited, worked the way the exam marks it
The full 3-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works