MDIA1091 Chap.7 Algorithms, AI and Social Injustice
Algorithms, AI and Social Injustice
Critical algorithm and AI studies follows the whole decision system rather than treating a model as neutral magic or an independent villain. Historical data, labels, proxies, objectives, thresholds, staff practice and appeal routes can all shape outcomes. Aggregate accuracy may conceal uneven false positives, delays or lost opportunities across social groups. Opacity can be technical, commercial or organisational.
A responsible audit separates purpose, process and outcomes, then identifies the institution with authority to explain, suspend, monitor or redesign the system when material harm appears.
What this chapter covers
- 01
Automated systems transform institutional objectives into classifications, rankings, predictions or generated media
- 02
Historical data and labels can reproduce unequal prior practices before model training begins
- 03
Proxies simplify complex qualities and may indirectly encode protected or socially patterned differences
- 04
Thresholds allocate error, so aggregate performance should be compared with group-level outcomes and material stakes
- 05
Opacity can arise from complexity, secrecy, poor documentation or organisational inability to explain use
- 06
Accountability includes notice, explanation, appeal, correction, monitoring and the power to stop a harmful deployment
Worked example · free
Auditing civic service request ranking
- +1Identify the proxy: duplicate report count stands in for need, but it also measures access, awareness and willingness to use the app.
- +1Trace the mechanism: lower participation can reduce ranking visibility and convert unequal access into slower or weaker service response.
- +1Specify evidence: compare request severity, ranking, response time and unresolved harm across areas while accounting for relevant service differences.
- +1Propose repair: add reporting channels, incorporate need-based review, publish disaggregated outcomes and provide a route to contest mistaken prioritisation.
Key terms
- Algorithm
- An algorithm is a coded procedure that transforms inputs according to specified operations or learned patterns.
- Training data
- Training data is the collection of examples from which a model learns patterns, categories or relationships.
- Proxy
- A proxy is a measurable variable used to represent a more complex quality that may not be directly observable.
- Threshold
- A threshold is the decision boundary that converts a score or prediction into a category or action.
- Algorithmic opacity
- Algorithmic opacity is the practical difficulty of understanding, explaining or contesting an automated operation and its use.
- Disparate outcome
- A disparate outcome is a materially different distribution of error, access or consequence across social groups.
Algorithms, AI and Social Injustice FAQ
Where can bias enter an AI system?
Bias can enter through the institutional goal, historical data, labels, missing cases, proxies, model design, thresholds, staff reliance and deployment context. It can also emerge after a decision through weak appeal or correction. Auditing only the model may miss the mechanism that creates the material inequality.
Does high accuracy mean a system is fair?
Not necessarily. Aggregate accuracy can conceal who receives false positives, false negatives, delays or lost opportunities. Fairness also depends on the stakes, objective, access conditions and ability to contest a decision. Compare disaggregated outcomes and examine whether the measured target represents the social purpose.
How should I analyse a generative AI example?
Document model version, interface, prompt, settings and sampling method. One output can demonstrate a possible failure but not prevalence. Use repeated, comparable tests for pattern claims, analyse representation and consequence, and identify the company or institution that governs deployment and response.
Assessment move
Draw the pipeline from social goal to data, labels, proxies, model, threshold, decision and material outcome. Under each stage, write the institution or worker with authority and the evidence required to audit it. Test one counterfactual design to reveal the trade-off rather than assuming a technical choice eliminates uncertainty. Practise distinguishing a failure case from a patterned disparity.
For generative systems, keep prompts and settings reproducible. End every answer with repair at the level of the mechanism and governance: new access channels, revised objectives, disaggregated monitoring, meaningful explanation, appeal or suspension. Replace sentences in which AI is the sole actor with the organisations, workers and operations that produce the result. Turn the pipeline into an audit worksheet.
At each stage, ask what is measured, who is missing, how error is distributed, who can contest the result and what happens while a challenge is reviewed. Keep accuracy separate from justice: a system can predict its chosen label consistently while the label, goal or allocation rule remains unfair.
Write one counterfactual in which the same person encounters a different access channel or threshold, then locate the decision point producing the changed outcome. Evaluate repairs against several criteria, including effectiveness, burden, transparency, privacy and institutional accountability. A strong proposal names the responsible body, the operational change and the evidence needed to determine whether harm declined.
