MAIE5103 Chap.9 Privacy, Recourse, Oversight and Power
Privacy, Recourse, Oversight and Power
Privacy across the whole data flow
The Beyond Bias lecture treats privacy as a property of a system rather than a checkbox at collection. Each stage has its own control: minimise and explain what is collected, restrict reuse in training and check vendor terms, enforce each user's access before retrieval, redact sensitive prompts and outputs in logs, and set expiry and deletion for what is retained.
It also names an audit paradox: measuring a disparity may need the sensitive attributes privacy protects, so audit data should be gathered for a stated purpose, kept apart from decision-makers, deleted on schedule and reported only in aggregate.
Explanation, justification and recourse
These three answer different questions about one decision.
An explanation says why the system produced its result, and feature attributions describe the model rather than necessarily a cause. A justification says why the rule itself is acceptable, and a fully transparent rule can still be illegitimate. Recourse says what the affected person can actually do, and it has to be feasible, accessible and reviewed by someone with the authority to change the outcome.
The fairness lecture adds that accountability needs names, the developer, deployer, data owner and reviewer, never just the model.
Oversight and power
Human oversight works only when the reviewer has authority, time, evidence and incentives together; without them, review becomes a rubber stamp, and the way to tell is to test with known errors rather than to count overrides.
The chapter ends with power: who benefits from a system, who bears its risks, who decides, and whose costs never appear in the objective function. Participation by affected people helps only if it can change the product.
What this chapter covers
- 01
Privacy controls at collection, training, retrieval, logging and retention
- 02
The audit paradox and how to manage it
- 03
Explanation, justification and recourse as separate questions
- 04
Accountability with named roles
- 05
Explaining to an affected person and to an auditor
- 06
Oversight capacity and the rubber-stamp test
- 07
Benefits, burdens and leverage across stakeholders
- 08
Participation that can change a product
Worked example · free
Classify the parts of a rejection letter
- 3Classify each of the three sentences.
- 3Name what each of the three questions still lacks.
Key terms
- Audit Paradox
- The tension that detecting a group disparity may require collecting the sensitive attributes that privacy protections aim to limit.
- Justification
- The reason a decision rule is acceptable, which is a separate question from how the system reached a particular result.
- Oversight Capacity
- The combination of authority, time, evidence and incentives a reviewer needs for human oversight to work.
- Rubber Stamp
- Human review that approves automated recommendations without the evidence, time or freedom to override them.
- Participatory Design
- Involving affected people in choosing a system's metric, harms and overrides, with pay, the real interface and a written reply on what changed.
Privacy, Recourse, Oversight and Power FAQ
What is the difference between explainability and accountability in AI?
Explainability is about why a system produced a result. Accountability is about who answers for it: the fairness lecture asks for named roles, the developer, the deployer, the data owner and the reviewer, and records such as model cards, datasheets and incident logs. An attribution score can explain a model perfectly and still leave nobody accountable, especially if the construct the model predicts is wrong.
How can you tell whether human oversight of an AI system is meaningful?
Test it. Inject cases with known errors into the reviewer's queue under safe conditions and measure how many are detected, how well they are corrected and how appeals are resolved. Counting overrides is not enough, because a reviewer who lacks evidence, has seconds per case or is penalised for disagreeing will rarely override even when the model is wrong.
Does collecting sensitive data for fairness audits violate privacy?
It can create risk, which is why the lecture calls it a paradox rather than a contradiction. The answer it gives is a justified audit purpose, access separated from the operational decision, limited retention and controlled, aggregated reporting. Collection should never become unrestricted, and the attribute should never become a new input to the decision it is meant to audit.
Assessment move
Take three real decision notices you can find, from a bank, a university or a platform, and mark each sentence as explanation, justification or recourse. Then write the oversight test you would run on whoever reviews those decisions. Practising on real documents builds the habit of asking all three questions, which is the habit the reading response and case study analysis reward.
Working through Privacy, Recourse, Oversight and Power in MAIE5103? Sia is AskSia’s AI Arts and Humanities tutor — ask any MAIE5103 Privacy, Recourse, Oversight and Power question and get a clear, step-by-step explanation grounded in how MAIE5103 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.