MAIE5103 Chap.2 How Bias Enters the Data Lifecycle
How Bias Enters the Data Lifecycle
Five checkpoints, one loop
The lecture's map of the data lifecycle has five stages, and each one is an audit checkpoint with its own question. At collection, ask who was sampled, measured and asked for consent. At labelling, ask whose judgement became ground truth. At modelling, ask about the objective, the features, the capacity and which slices were used to evaluate.
At deployment, ask about the threshold, the override rules and the population actually affected. Then comes the loop, where logged outcomes become the training data for the next version, so a harm that appears late is also evidence about how the data was gathered.
Measurement, representation and labels
The most consequential failures happen before a model exists.
Measurement bias is the gap between a construct, such as creditworthiness or need, and the instrument that stands in for it, such as an arrest record or a spending total. Representation fails when some groups never appear, appear only as outliers, or appear in worse-quality data, as the face-analysis benchmarks did.
Labels are value judgements made by particular raters, and a majority vote can erase a minority reading, which is why the lecture prefers to keep disagreement visible.
Proxies, missing data and loops
Deleting a protected attribute rarely removes it, because districts, names, languages and browsing patterns carry the same signal, and deleting it also makes the gap harder to audit.
Missing data matters most when the gap depends on the missing value itself, as with income, diagnosis or arrests. Feedback loops in policing, credit and ranking turn yesterday's decisions into tomorrow's evidence. The chapter traces all of these through the resume-ranking case and ends with a test for proxies that any audit can run.
What this chapter covers
- 01
Collect, label, model, deploy and loop as audit checkpoints
- 02
Model, institution and people as one sociotechnical frame
- 03
Historical bias and why an accurate predictor can still be unfair
- 04
Protected attributes, proxies and the failure of unawareness
- 05
Three mechanisms of missing data
- 06
Coverage, density and quality in representation
- 07
Rater disagreement and the problem with a single gold label
- 08
Runaway feedback loops in policing, credit and ranking
Worked example · free
Find the stage where a translation tool went wrong
- 2Locate at least two stages where the gap could enter.
- 2Name the evidence that would confirm or rule out each.
- 2Explain why an architecture claim does not answer a data question.
Key terms
- Measurement Bias
- The gap between a construct such as need or risk and the instrument used to measure it, such as spending or arrest records.
- Historical Bias
- Inequality in past access to jobs, credit, housing or policing that enters a model as labels, features and base rates even when every record is accurate.
- Unawareness
- The practice of excluding a protected attribute from a model, which fails when proxies remain and also blocks the audit of disparities.
- Missing Not at Random
- A pattern in which whether a value is missing depends on the unobserved value itself, so that the gap in the data is itself a signal.
- Intersectional Slice
- A subgroup defined by two or more attributes at once, such as skin type and gender, used because single-attribute averages can hide the worst-served cell.
- Rater Disagreement
- Variation among annotators labelling the same item, which carries information about contested categories and is lost when a single majority label is forced.
How Bias Enters the Data Lifecycle FAQ
Why does removing race or gender from the data not remove bias?
Because other inputs carry the same information. Districts encode residential segregation, names correlate with gender and ethnicity, and language patterns track communities, so a model can rebuild the attribute's effect without ever seeing it. Removing the attribute also makes it harder to measure the gap. The lecture's check is to train a second model to predict the attribute from the remaining inputs.
What is the difference between MCAR, MAR and MNAR?
They describe why data is missing. Missing completely at random is unrelated to anything and is rare in social data. Missing at random depends only on things you observe, so careful imputation can help. Missing not at random depends on the missing value itself, such as people with low income skipping an income question, and it is the case the lecture treats as a fairness failure.
What does intersectionality mean in a fairness audit?
It means reporting results for combinations of attributes, not only for each attribute separately. In the face-analysis benchmarks, averages by gender and by skin type each looked better than the combination of darker skin and female, where error reached 34.7%. The lecture credits the idea to Crenshaw and asks for the full cross of attributes first, followed by caveats about small cells.
Assessment move
Draw the five-stage lifecycle from memory and, under each stage, write one question you would ask and one case from the lecture that failed there. Then take a system you know and walk it through the five questions in order. Most written answers lose marks by naming bias without saying where it entered; naming the stage is what turns a complaint into something an engineer can fix.
Working through How Bias Enters the Data Lifecycle in MAIE5103? Sia is AskSia’s AI Arts and Humanities tutor — ask any MAIE5103 How Bias Enters the Data Lifecycle question and get a clear, step-by-step explanation grounded in how MAIE5103 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.