MAIE5103 Chap.1 Bias, Fairness and Harm in Landmark Cases
Bias, Fairness and Harm in Landmark Cases
Three words that do different jobs
The opening of the fairness and bias lectures asks for a discipline that sounds pedantic and turns out to decide most arguments: keep bias, fairness and harm apart. Bias is a systematic deviation from a stated target, and it lives in a pipeline of data, labels, models and deployment rather than in any group of people.
Fairness is not one number but a family of criteria, some about groups and some about individuals. Harm is what a person actually lives with after the decision, whether that is a denial, a stigma, surveillance or being left out of the picture altogether.
Allocative and representational harm
The chapter separates the two harm families the lecture names.
Allocative harm hands out or withholds something countable through a score, such as a credit limit, a place on a shortlist, a benefit or pretrial liberty, so its first question is who was denied and at what rate of wrongful flags.
Representational harm works through outputs that stereotype or erase, in image models, search rankings and generated biographies, and it accumulates rather than arriving as one decision, so its first question is who is shown and who is missing.
Landmark cases as evidence, not anecdotes
Each case in the lecture is cited with a date, a jurisdiction and a source, and the chapter keeps them that way: the 2016 risk-score analysis in Broward County, the experimental resume ranker reported in 2018, the commercial face-analysis benchmarks of the same year, the 2019 care-management study, the 2020 grading algorithm in England, the Dutch childcare-benefit scandal, the cropping audit, and a hiring screen that ended in a consent decree.
For every one, the useful skill is to find the single number that exposed the problem and to say which harm family it belongs to.
What this chapter covers
- 01
Bias, fairness and harm as three separate objects
- 02
Statistical and sociotechnical readings of bias
- 03
Allocative harm and its first question
- 04
Representational harm and its first question
- 05
Why accuracy hid the problem in the risk-score case
- 06
The resume ranker, the face benchmarks and the care-cost proxy
- 07
Public-sector cases in England and the Netherlands
- 08
Repository studies, the cropping audit and the New York audit law
Worked example · free
Classify two complaints about one university portal
- 2Assign each complaint to a harm family with a reason.
- 2Give the opening question that fits each family.
- 2Say what the 18% against 31% gap does not yet establish.
Key terms
- Allocative Harm
- Harm that occurs when a score grants or withholds opportunity, money or liberty, such as a credit limit, a benefit or a place on a shortlist.
- Representational Harm
- Harm that occurs when outputs stereotype, erase or mock a group, accumulating across many outputs rather than arriving as one decision.
- Sociotechnical Bias
- A process in which data, labels, models and deployment together tilt outcomes by group, which makes bias a property of the pipeline rather than of the people in it.
- Statistical Bias
- The gap between an estimator and the parameter it is meant to estimate.
- Bias Audit
- An independent check of an automated decision tool for disparities between groups, which New York City requires within the year before an employment tool is used.
- Cost Proxy
- A measurable stand-in such as health-care spending used in place of the quantity a decision actually needs, such as health need.
Bias, Fairness and Harm in Landmark Cases FAQ
Is a high overall accuracy figure evidence that a model is fair?
No. Accuracy blends the true-positive and false-positive rates with each group's base rate, so two groups can share one accuracy figure while one of them carries far more wrongful flags. In the 2016 risk-score analysis overall accuracy was similar, about 61%, while the false-positive rate for Black defendants was 45% against 23% for White defendants. Ask for one confusion matrix per group instead.
What is the difference between allocative and representational harm?
Allocative harm withholds or grants something through a score, such as credit, a job interview or a benefit, and it can be counted person by person. Representational harm comes from outputs that stereotype or leave a group out, such as image labels, rankings or generated text, and it builds up over many outputs. Each needs its own test: who is denied, against who is shown and who never appears.
Why did editing the resume ranker's vocabulary not fix it?
Because the problem was the target, not a word list. The model had learned from ten years of mostly male hiring, so it rewarded patterns correlated with past hires. Removing terms it penalised, such as the word for women's organisations, left other correlated signals in place, and the project was eventually disbanded. A repair would have needed a definition of job skill that did not depend on who had been hired before.
Assessment move
Make a one-page card for each landmark case with four lines: date and jurisdiction, the system and its decision, the single number that exposed the problem, and the harm family. Then cover the cards and practise placing a new news story on the same four lines in under five minutes, because that is the move the news presentation and the case study analysis both reward.
Working through Bias, Fairness and Harm in Landmark Cases in MAIE5103? Sia is AskSia’s AI Arts and Humanities tutor — ask any MAIE5103 Bias, Fairness and Harm in Landmark Cases question and get a clear, step-by-step explanation grounded in how MAIE5103 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.