INFO 202 Chap.10 Algorithmic Bias, Equity and Inclusion
Algorithmic Bias, Equity and Inclusion
The focus of Week 10
This week asks what algorithmic bias is and why it matters for both efficiency and equity. The syllabus directs attention to the way design decisions inside AI systems carry values, so that systems can repeat and even magnify existing patterns of unfairness, and to the remedies that might help.
The assigned reading is a 2019 explainer by Karen Hao in MIT Technology Review that walks through how bias arises in deep learning and why engineers struggle to remove it.
Bias is more than bad data
A popular explanation says biased AI simply reflects biased training data. Hao argues that this is only part of the story.
Before any data exists, someone decides what the model should predict, and that decision usually reflects commercial priorities. Data can then fail by leaving groups underrepresented or by recording past discrimination faithfully.
Finally, engineers decide which attributes the model may use, a choice whose effect on accuracy is easy to test but whose effect on fairness is much harder to see.
Why fixing it is difficult
Hao identifies four obstacles. Harms often appear only after a system is in use, when their origin is hard to trace.
Standard testing draws evaluation data from the same pool as training data, so it cannot reveal a shared skew. Engineering culture values tools that can be reused across settings, while fairness depends on local context and differs between communities and domains.
And fairness has several mathematical definitions that cannot all be satisfied together, which forces a choice that the field tends to treat as permanent.
From explanation to evaluation
For exams and discussion sections, the useful skill is turning this account into questions you can ask of any automated decision system. Who set the goal, and what did they optimize? Whose history does the data record?
Which inputs could act as proxies for protected characteristics? Would the testing process have noticed a problem? Which idea of fairness was chosen, and could it be revisited? Answering these moves an argument from general worry to specific diagnosis.
It also keeps you from accepting a single reassuring statistic, such as strong accuracy on held-out data, as proof that a system treats people fairly, since accuracy and fairness are measured differently.
Equity and inclusion
The equity dimension of the week is that the costs of a biased system fall unevenly, often on groups already disadvantaged in hiring, lending or policing.
Hao ends by noting active work on detection tools, accountability mechanisms and debates over definitions, while stressing that the problem calls for ongoing attention rather than a one-time technical patch.
What this chapter covers
- 01
How the syllabus connects algorithmic bias to efficiency and equity
- 02
Why biased training data is only part of the explanation
- 03
Problem framing as the first point where bias enters
- 04
Unrepresentative and prejudiced data collection
- 05
Attribute selection in data preparation
- 06
Four reasons bias is hard to fix
- 07
The portability trap and social context
- 08
Competing mathematical definitions of fairness
- 09
An audit procedure for automated decision systems
Worked example · free
Diagnose bias in an automated hiring screen
- 2Identify at least two stages where bias could have entered this model.
- 2Explain why removing gender is unlikely to be sufficient.
- 2Explain why the retest does not support the fairness claim.
Key terms
- Algorithmic Bias
- Systematic unfairness in the outputs of an automated system that disadvantages some groups, arising from choices about goals, data, attributes or evaluation rather than from a single faulty input.
- Problem Framing
- The first stage of building a model, in which a vague goal such as creditworthiness is translated into a computable target, often for business rather than fairness reasons.
- Training Data
- The examples a machine learning model learns patterns from. Bias appears when this data underrepresents some groups or faithfully records past discrimination.
- Data Preparation
- The stage of selecting which attributes a model may use, such as age or income; its effect on accuracy is measurable, while its effect on bias is harder to detect.
- Proxy Variable
- An input that correlates with a protected characteristic, allowing a model to reproduce bias against a group even after the explicit characteristic has been removed.
- Validation Split
- The common practice of randomly dividing one dataset into training and test portions, which means test data carries the same biases and cannot reveal them.
- Portability Trap
- Andrew Selbst's term for the error of reusing a system across communities or domains, ignoring that each setting holds its own understanding of what fairness requires.
- Fairness Definition
- A mathematical statement of what an unbiased outcome means, such as equal score rates across groups; several reasonable definitions conflict and cannot all hold together.
Algorithmic Bias, Equity and Inclusion FAQ
How does bias get into AI systems?
According to Karen Hao's account, bias can enter at three main stages: when the problem is framed and a goal chosen, when training data is collected, and when data is prepared by selecting attributes. It can therefore appear before any data exists. Treating bias as purely a data quality issue misses the value judgments built into goals and feature choices.
Why is AI bias so hard to fix?
Hao gives four reasons. The effects of a design choice may only show up after deployment, making the cause hard to trace. Standard testing uses data drawn from the same source as training, so it inherits the same skew. Engineering favors portable systems that ignore local context. And competing mathematical definitions of fairness cannot all be satisfied at once.
Does removing race or gender from a model make it fair?
Usually not. Other inputs can act as proxies for the removed characteristic. When Amazon's engineers told an internal recruiting tool to ignore explicitly gendered words, it continued to favor men through other terms that were more common on men's applications. Fairness requires checking outcomes and the choices made at each stage, not just deleting a column.
What is the portability trap in machine learning?
It is the problem, named by the scholar Andrew Selbst, of designing a system to be reused across different settings. Engineers see reusability as good practice, but fairness depends on context: communities differ in what they consider fair, and the standards suited to criminal justice differ from those suited to employment decisions.
Exam move
Draw the six-stage pipeline from memory and mark the three entry points Hao highlights, then write one real-world example next to each. Beside the validation stage, explain in a sentence why a random split hides bias. Finally, practice the five-check audit on a system you encounter, such as a feed ranking or a loan tool, writing one question per check in under ten minutes.
Working through Algorithmic Bias, Equity and Inclusion in INFO 202? Sia is AskSia’s AI Information Technology tutor — ask any INFO 202 Algorithmic Bias, Equity and Inclusion question and get a clear, step-by-step explanation grounded in how INFO 202 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
