The Hong Kong University of Science and Technology · FACULTY OF ARTS & HUMANITIES

MAIE5103 Chap.1 Bias, Fairness and Harm in Landmark Cases

- one subject, every graph, every model, every mark
8 Chapters4-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 1 of 14 · MAIE5103

Bias, Fairness and Harm in Landmark Cases

Three words that do different jobs

The opening of the fairness and bias lectures asks for a discipline that sounds pedantic and turns out to decide most arguments: keep bias, fairness and harm apart. Bias is a systematic deviation from a stated target, and it lives in a pipeline of data, labels, models and deployment rather than in any group of people.

Fairness is not one number but a family of criteria, some about groups and some about individuals. Harm is what a person actually lives with after the decision, whether that is a denial, a stigma, surveillance or being left out of the picture altogether.

Allocative and representational harm

The chapter separates the two harm families the lecture names.

Allocative harm hands out or withholds something countable through a score, such as a credit limit, a place on a shortlist, a benefit or pretrial liberty, so its first question is who was denied and at what rate of wrongful flags.

Representational harm works through outputs that stereotype or erase, in image models, search rankings and generated biographies, and it accumulates rather than arriving as one decision, so its first question is who is shown and who is missing.

Landmark cases as evidence, not anecdotes

Each case in the lecture is cited with a date, a jurisdiction and a source, and the chapter keeps them that way: the 2016 risk-score analysis in Broward County, the experimental resume ranker reported in 2018, the commercial face-analysis benchmarks of the same year, the 2019 care-management study, the 2020 grading algorithm in England, the Dutch childcare-benefit scandal, the cropping audit, and a hiring screen that ended in a consent decree.

For every one, the useful skill is to find the single number that exposed the problem and to say which harm family it belongs to.

In this chapter

What this chapter covers

  • 01

    Bias, fairness and harm as three separate objects

  • 02

    Statistical and sociotechnical readings of bias

  • 03

    Allocative harm and its first question

  • 04

    Representational harm and its first question

  • 05

    Why accuracy hid the problem in the risk-score case

  • 06

    The resume ranker, the face benchmarks and the care-cost proxy

  • 07

    Public-sector cases in England and the Netherlands

  • 08

    Repository studies, the cropping audit and the New York audit law

Worked example · free

Classify two complaints about one university portal

Q [6 marks]. AskSia-authored practice. A university admissions portal uses one model to rank applicants for interview and another to write a short profile of each applicant for the panel. Applicants from rural schools are invited at 18% against 31% for city schools, and the profiles describe rural applicants' achievements in noticeably vaguer language. Name the harm family behind each complaint, the first question to ask about each, and the one claim nobody can make yet. The marks shown are an AskSia study allocation, not a University marking scheme.
Read the arrows as 'is measured by' and 'stands in for'.Read the arrows as 'is measured by' and 'stands in for'. A fairness metric measures one kind of bias in a pipeline and is only a proxy for harm, which then splits into allocative and representational families with different first questions.Biasin the pipelineFairness metrica named proxyHarma lived consequenceAllocativeWho is denied?RepresentationalWho never appears?
Read the arrows as 'is measured by' and 'stands in for'. A fairness metric measures one kind of bias in a pipeline and is only a proxy for harm, which then splits into allocative and representational families with different first questions.
  • 2Assign each complaint to a harm family with a reason.
  • 2Give the opening question that fits each family.
  • 2Say what the 18% against 31% gap does not yet establish.
The interview gap is allocative: an opportunity is being granted or withheld by a score, so the first question is how many rural applicants who meet the admissions standard were not invited. The vague profiles are representational: no single decision is made by the text, but a group is described in a way that diminishes it across every panel that reads a profile, so the first question is how rural achievements are depicted and what is left out. The gap between 18% and 31% is a bias finding about the pipeline. It is not yet a fairness verdict, because no criterion has been named, and it is not yet a measured harm, because the qualified rural applicants have not been counted.
Sia tip — Before calling a gap unfair, write the criterion you would test it against in one line; until you can, call it a disparity.
Glossary

Key terms

Allocative Harm
Harm that occurs when a score grants or withholds opportunity, money or liberty, such as a credit limit, a benefit or a place on a shortlist.
Representational Harm
Harm that occurs when outputs stereotype, erase or mock a group, accumulating across many outputs rather than arriving as one decision.
Sociotechnical Bias
A process in which data, labels, models and deployment together tilt outcomes by group, which makes bias a property of the pipeline rather than of the people in it.
Statistical Bias
The gap between an estimator and the parameter it is meant to estimate.
Bias Audit
An independent check of an automated decision tool for disparities between groups, which New York City requires within the year before an employment tool is used.
Cost Proxy
A measurable stand-in such as health-care spending used in place of the quantity a decision actually needs, such as health need.
FAQ

Bias, Fairness and Harm in Landmark Cases FAQ

Is a high overall accuracy figure evidence that a model is fair?

No. Accuracy blends the true-positive and false-positive rates with each group's base rate, so two groups can share one accuracy figure while one of them carries far more wrongful flags. In the 2016 risk-score analysis overall accuracy was similar, about 61%, while the false-positive rate for Black defendants was 45% against 23% for White defendants. Ask for one confusion matrix per group instead.

What is the difference between allocative and representational harm?

Allocative harm withholds or grants something through a score, such as credit, a job interview or a benefit, and it can be counted person by person. Representational harm comes from outputs that stereotype or leave a group out, such as image labels, rankings or generated text, and it builds up over many outputs. Each needs its own test: who is denied, against who is shown and who never appears.

Why did editing the resume ranker's vocabulary not fix it?

Because the problem was the target, not a word list. The model had learned from ten years of mostly male hiring, so it rewarded patterns correlated with past hires. Removing terms it penalised, such as the word for women's organisations, left other correlated signals in place, and the project was eventually disbanded. A repair would have needed a definition of job skill that did not depend on who had been hired before.

Study strategy

Assessment move

Make a one-page card for each landmark case with four lines: date and jurisdiction, the system and its decision, the single number that exposed the problem, and the harm family. Then cover the cards and practise placing a new news story on the same four lines in under five minutes, because that is the move the news presentation and the case study analysis both reward.

Working through Bias, Fairness and Harm in Landmark Cases in MAIE5103? Sia is AskSia’s AI Arts and Humanities tutor — ask any MAIE5103 Bias, Fairness and Harm in Landmark Cases question and get a clear, step-by-step explanation grounded in how MAIE5103 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 13 of your subjects at The Hong Kong University of Science and Technology, plus 1,000+ more Bibles across the AskSia library.
Sia - your MAIE5103 tutor, unlimited, worked the way the exam marks it
The full 4-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works