PHIL7002 Chap.3 Bias, Fairness and Justice
Bias, Fairness and Justice
Bias is not always a fault
Computer scientists point out that bias in the statistical sense is part of what makes machine learning work: the fact that a person's past choices carry a pattern is exactly why a recommendation can be useful, and exploiting such regularities underlies personalisation, market segmentation and traffic management.
An answer that treats bias as a defect to be eliminated has therefore already lost the argument. The question is always which regularities may be exploited, in which context, at what stakes.
Removing the human does not remove the bias
The chapter begins with human shortcuts.
The availability heuristic estimates likelihood from how often you encounter something, which works for meeting a cat and fails for being murdered because reporting makes the second artificially available.
People are also partial and skilled at rationalising, and the compensations we have built, committees, structured decision forms, requirements that judges give reasons, are cumbersome and gameable, with professionals known to complete structured forms so as to reach the answer their intuition already favoured.
Automation looks like the clean solution, and it is not, because humans decide how a system is built and trained.
Three gates, and why the repair has to match the gate
The taught taxonomy divides bias by where the human decision sits: in how a system is made, in how it is trained, and in how it is used in a particular context.
Built-in bias enters through variable selection, since the choice of what counts as evidence encodes the developer's beliefs, and omitting a genuinely predictive variable harms the people it would have identified. Undocumented collection and labelling choices compound it, and carrying a tool built for one purpose into another degrades both accuracy and fairness.
The lecture material offers a second cut of the same territory, distinguishing data bias in who is seen and labelled, historical bias where past practice has encoded inequity, and operational bias in where outputs are used.
The two cuts are compatible, and using both is stronger than defending either.
Cases that reward careful reading
A plagiarism detector that compares long character strings will score non-native paraphrase higher, because learners reproduce longer fragments while getting vocabulary and structure right, so a tool built to reduce marker bias introduces a new one.
Predictive policing concentrates observation where a model points, and the resulting records feed back until selection bias has compounded into confirmation.
Deleting a protected variable rarely helps, because its effects remain in correlated features, and stripping every proxy would remove much of the predictive signal.
Three fairness criteria that cannot all hold
Anti-classification requires that protected attributes and their proxies are not used. Classification parity requires that error rates match across groups.
Calibration requires that a risk estimate means the same thing whoever it is about. When the underlying rate of the predicted outcome differs between groups, these cannot be satisfied together, and the course states that they are logically incompatible rather than merely difficult to reconcile.
The well-known criminal justice dispute is the worked case: a vendor could show its tool was well calibrated while a news organisation showed error rates differed by group, and both were right. An impossibility result of this kind does not remove a decision. It reveals that one has already been made, and the course treats choosing between fairness standards as a matter for open democratic debate.
What this chapter covers
- 01
Why statistical bias is what makes prediction work
- 02
Human heuristics, and why committees and forms do not fix them
- 03
Bias in making, training and using a system
- 04
The portability trap and undocumented labelling
- 05
Feedback loops in predictive deployment
- 06
Why deleting a protected variable does not remove its effect
- 07
Anti-classification, classification parity and calibration
- 08
Incompatibility at unequal base rates, and the fairness and accuracy trade-off
Two audits, both correct, one undeclared choice
- 3Name the criterion each team measured.
- 3Explain why both results can hold at once.
- 3State what the two reports jointly establish, and whose decision it is.
Key terms
- Built-in Bias
- Bias introduced while a system is being made, chiefly through the choice of which variables count as evidence and how data is collected and labelled.
- Operational Bias
- Bias arising from where and how outputs are used, so that a modest model flaw is amplified by a high-stakes context.
- Historical Bias
- Inequity already encoded in past practice, which a model learns as though it were a description of the world rather than of a history.
- Base Rate
- The underlying incidence of the predicted outcome in a population. Differences in base rates between groups are what make the fairness criteria incompatible.
- Feedback Loop
- A pattern in which a model directs attention to certain cases, the resulting observations are fed back as training data, and the model's confidence rises while the observed population narrows.
- Proxy Variable
- A feature that correlates with a protected attribute closely enough to carry its effect, which is why deleting the attribute itself rarely removes it.
- Error Rate Balance
- The component of classification parity requiring that the rate of false alarms and the rate of misses come out the same for each group.
Bias, Fairness and Justice FAQ
Can an algorithm be made fair?
Not in every sense at once. Three criteria are in circulation, and where the underlying rate of the predicted outcome differs between groups they are mathematically incompatible, so satisfying one means failing another. There are also trade-offs against accuracy, since excluding every proxy for a protected attribute removes much of the predictive signal.
What can be done is to choose a criterion, state it, defend it and record who bears the residual error, which is why the course treats this as a political question.
Why does removing gender or ethnicity from the training data not solve the problem?
Because the effects of those attributes survive in features that correlate with them, such as income, education and location, so the pattern persists without the label. Stripping every correlate would also remove much of what made the system useful, which is a real cost rather than an excuse.
A claim to have eliminated bias by deleting a column is therefore a claim about the column and not about the model, and the honest version reports what was removed and what was measured afterwards.
How do I recognise a feedback loop in a deployment?
Look for rising model confidence alongside a narrowing set of observations. The tell is that evidence is no longer being gathered from the places the system stopped directing resources to, which makes the model unfalsifiable rather than accurate.
That is a monitoring failure rather than a training-data failure, and the diagnostic question is simple enough to ask in an exam answer: who is still collecting data from the cases the system now ignores?
Exam move
Write the three fairness criteria from memory, then take any scoring tool in the news and say which one its defenders are using and which one its critics are using. Most public disputes about algorithmic fairness are exactly this disagreement left unstated, and being able to name it in two sentences is worth more than being able to recite the definitions.
Working through Bias, Fairness and Justice in PHIL7002? Sia is AskSia’s AI Arts and Humanities tutor — ask any PHIL7002 Bias, Fairness and Justice question and get a clear, step-by-step explanation grounded in how PHIL7002 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
