COMP5577 Chap.3 Data Poisoning and Backdoor Attacks
Data Poisoning and Backdoor Attacks
Poisoning changes the learned rule
Data poisoning enters before deployment through examples, labels, feedback or model weights. Untargeted attacks lower broad utility, while targeted attacks change a selected outcome.
Label flipping keeps inputs and changes labels, making the malicious association part of ordinary training.
Backdoors hide behind normal performance
A backdoor preserves clean behavior and adds a trigger-conditioned route to an attacker-chosen class.
Clean evaluation omits the trigger, so a complete verdict reports clean accuracy, poisoning rate and attack success on triggered non-target inputs.
The supply chain is part of training
A checkpoint obtained from another party can carry a narrow behavior even when its headline benchmark matches the expected model. Small weight edits may alter one association while unrelated outputs remain stable.
Artifact verification, signed provenance and task-specific tests address this risk more directly than adding ordinary clean examples, because the attacker may never have touched the visible training set.
Interpret the pair of evaluations
Clean accuracy describes utility on normal inputs. Triggered attack success describes the adversary’s conditional path.
High values for both are not contradictory; together they characterize a stealthy backdoor. The triggered denominator should use non-target inputs so ordinary correct target predictions are not credited to the attacker. After cleaning or retraining, repeat both evaluations. A lower count of suspected samples is not recovery evidence if the trigger continues to control predictions.
What this chapter covers
- 01
Data contributors and training access
- 02
Untargeted and targeted poisoning
- 03
Label-flipping attacks
- 04
Backdoor triggers and target classes
- 05
Clean accuracy and triggered attack success
- 06
Model poisoning and checkpoint provenance
Worked example · free
Evaluate a small poisoning budget
- 2Calculate the poisoning rate.
- 2Calculate triggered attack success.
- 4Interpret beside clean performance.
Key terms
- Data Poisoning
- A training-time attack that changes learning data or labels to alter model behavior.
- Label Flipping
- A poisoning method that changes selected labels while leaving their inputs unchanged.
- Backdoor
- A learned trigger-conditioned behavior that maps attacked inputs to a chosen target while clean behavior remains normal.
- Poisoning Rate
- The poisoned or replaced examples divided by the final training-set size.
Data Poisoning and Backdoor Attacks FAQ
How does targeted poisoning differ from a backdoor?
Targeted poisoning changes a selected case or subgroup while preserving broad performance. A backdoor adds a secret trigger condition that causes an attacker-chosen output at inference time. Every backdoor is targeted poisoning in this framing, but not every targeted poisoning attack uses a trigger.
Why can clean accuracy stay high after poisoning?
The attacker may constrain the attack to a narrow condition and deliberately preserve ordinary behavior. Clean tests do not contain that condition, so they sample the normal branch. Triggered evaluation is needed to reveal the malicious branch and calculate attack success rate.
What does benchmark parity prove about a downloaded model?
It shows similar behavior on the benchmark distribution. It does not prove checkpoint provenance or rule out a narrow weight edit, hidden trigger or changed association outside the benchmark. Artifact verification and threat-model-specific tests address the supply-chain claim more directly.
Exam move
For each poisoning scenario, identify who can change examples, labels or weights, then decide whether the objective is broad, targeted or trigger-conditioned. Always sketch two evaluation paths: clean inputs and attacked inputs. Practise explaining why their metrics cannot be merged. Calculate the poisoning rate using the final training-set size, and define the triggered denominator so genuine target examples are excluded.
Then vary stealth: keep the trigger but raise clean damage, remove the visible trigger but preserve a label change, or place the edit directly in model weights. Name which detection method loses power in each version. Close every repair answer by repeating both clean and triggered evaluation.
If you find yourself reporting only how many suspicious samples were removed, add the model-behavior evidence that determines whether the backdoor survived. Build one numerical card where the attacker adds examples and another where examples are replaced, because those operations produce different final denominators. On the back, record poison rate, clean accuracy, attack success rate and the excluded cases.
Rework each card after changing only the target class or trigger prevalence. The invariant calculation steps should remain visible even as the attack story changes. Label every reported percentage with its numerator, denominator and evaluation condition.
Working through Data Poisoning and Backdoor Attacks in COMP5577? Sia is AskSia’s AI Computer Science tutor — ask any COMP5577 Data Poisoning and Backdoor Attacks question and get a clear, step-by-step explanation grounded in how COMP5577 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.