COMP5577 Chap.5 Evasion Attacks and Robust Defenses
Evasion Attacks and Robust Defenses
Evasion changes inputs after training
Adversarial examples exploit the geometry of a fixed model. FGSM takes one step in the sign of the input gradient for a non-targeted objective, while PGD repeats projected steps within a perturbation region.
Targeted attacks reverse the loss direction toward a chosen class.
Evidence strength must be named
Sanitization, gradient masking, adversarial training and certification protect in different ways. Empirical performance is tied to tested attacks and budgets.
Certified robustness is a formal statement over every permitted perturbation in a region under explicit assumptions.
Calculate before interpreting
Hold model weights fixed, compute the input gradient, take its element-wise sign, scale by the budget and apply the perturbation. Recompute the score or probabilities using the printed decision rule.
A point that reaches but does not cross a boundary may keep its class depending on how equality is defined. The final verdict follows from the model rule, not from the fact that a perturbation was applied.
Audit the attack as well as the model
A failed attack can reflect an incorrect objective, weak iterations, insufficient restarts or gradient obstruction rather than genuine robustness.
Name the attack family, norm or transformation budget, attacker knowledge and dataset. Report clean performance beside adversarial performance, and let the attack adapt to the defense where the threat model permits it. Even a broad empirical campaign remains different from a certificate that covers every allowed perturbation under stated assumptions.
What this chapter covers
- 01
Inference-time evasion
- 02
FGSM gradient-sign updates
- 03
PGD and targeted objectives
- 04
Input sanitization and detection
- 05
Adversarial training and trade-offs
- 06
Empirical evaluation versus certification
Worked example · free
Verify one gradient-sign update
- 2Calculate the original score.
- 3Form and apply the perturbation.
- 3Recompute the score and interpret.
Key terms
- Adversarial Example
- An input deliberately perturbed to change a model prediction while remaining within an attack constraint.
- FGSM
- A one-step attack that scales the element-wise sign of the input gradient by a perturbation budget.
- PGD
- An iterative gradient attack that projects each update back into the allowed perturbation region.
- Adversarial Accuracy
- The fraction of attacked inputs that the model still classifies correctly under the stated attack.
Evasion Attacks and Robust Defenses FAQ
Why is FGSM called a white-box attack?
Its standard form needs the gradient of loss with respect to the input, which requires knowledge of the model and differentiation path. A transfer attack may use a substitute model, but the gradient still comes from a model the attacker can inspect.
Why can gradient masking give a false sense of security?
It can make naive gradient attacks fail by hiding or disrupting useful gradients without eliminating adversarial inputs. Adaptive or alternative attacks may bypass the obstruction. Report the tested threat model and do not promote lower success for one attack into a general guarantee.
How should a defense be chosen under a clean-accuracy rule?
Apply the clean-performance threshold first and remove ineligible methods. Compare adversarial accuracy only among the remaining defenses, then report improvement over baseline and the tested attack budget. This prevents a robust but unusable method from winning the wrong optimization problem.
Exam move
Recompute FGSM examples in a fixed order: score, class, gradient, sign, perturbation, adversarial point, new score and verdict. Add one sentence stating what the experiment does not prove. Vary the equality rule and check whether a point on the boundary changes class. Then convert the same case from non-targeted to targeted and explain why the loss direction reverses.
For defense tables, circle eligible rows before comparing adversarial performance, calculate percentage-point improvement and name the tested budget. Build a claim ladder from one failed FGSM attempt, through multi-start empirical evaluation, to a formal certificate. For each rung, write the strongest conclusion allowed and the first stronger attack that remains open.
This keeps impressive numbers from turning into unsupported universal claims. Maintain a separate assumptions box beside every calculation: norm, perturbation budget, attack knowledge, number of starts and decision rule at equality. Redo one example after changing each assumption in isolation. If the numerical point stays the same but the verdict changes, explain why.
That habit keeps an arithmetic answer tied to its threat model and makes comparison tables defensible rather than decorative. Never compare robust accuracies until the perturbation norm and budget match.
Working through Evasion Attacks and Robust Defenses in COMP5577? Sia is AskSia’s AI Computer Science tutor — ask any COMP5577 Evasion Attacks and Robust Defenses question and get a clear, step-by-step explanation grounded in how COMP5577 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.