The Hong Kong University of Science and Technology · FACULTY OF ARTS & HUMANITIES

MAIE5103 Chap.2 How Bias Enters the Data Lifecycle

- one subject, every graph, every model, every mark
8 Chapters5-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 2 of 14 · MAIE5103

How Bias Enters the Data Lifecycle

Five checkpoints, one loop

The lecture's map of the data lifecycle has five stages, and each one is an audit checkpoint with its own question. At collection, ask who was sampled, measured and asked for consent. At labelling, ask whose judgement became ground truth. At modelling, ask about the objective, the features, the capacity and which slices were used to evaluate.

At deployment, ask about the threshold, the override rules and the population actually affected. Then comes the loop, where logged outcomes become the training data for the next version, so a harm that appears late is also evidence about how the data was gathered.

Measurement, representation and labels

The most consequential failures happen before a model exists.

Measurement bias is the gap between a construct, such as creditworthiness or need, and the instrument that stands in for it, such as an arrest record or a spending total. Representation fails when some groups never appear, appear only as outliers, or appear in worse-quality data, as the face-analysis benchmarks did.

Labels are value judgements made by particular raters, and a majority vote can erase a minority reading, which is why the lecture prefers to keep disagreement visible.

Proxies, missing data and loops

Deleting a protected attribute rarely removes it, because districts, names, languages and browsing patterns carry the same signal, and deleting it also makes the gap harder to audit.

Missing data matters most when the gap depends on the missing value itself, as with income, diagnosis or arrests. Feedback loops in policing, credit and ranking turn yesterday's decisions into tomorrow's evidence. The chapter traces all of these through the resume-ranking case and ends with a test for proxies that any audit can run.

In this chapter

What this chapter covers

  • 01

    Collect, label, model, deploy and loop as audit checkpoints

  • 02

    Model, institution and people as one sociotechnical frame

  • 03

    Historical bias and why an accurate predictor can still be unfair

  • 04

    Protected attributes, proxies and the failure of unawareness

  • 05

    Three mechanisms of missing data

  • 06

    Coverage, density and quality in representation

  • 07

    Rater disagreement and the problem with a single gold label

  • 08

    Runaway feedback loops in policing, credit and ranking

Worked example · free

Find the stage where a translation tool went wrong

Q [6 marks]. AskSia-authored practice. A hospital deploys a speech-to-text tool for clinic notes. Error rates are 4% for standard English speakers and 19% for patients who switch between Cantonese and English mid-sentence. The vendor says the model architecture is state of the art. Identify the lifecycle stages where this gap could have entered, the evidence you would ask for at each, and why the vendor's answer is beside the point. The marks shown are an AskSia study allocation, not a University marking scheme.
Each box poses the question an auditor asks at that stage.Each box poses the question an auditor asks at that stage. The dashed return path is the reason a bias found after deployment is also evidence about collection, since logged outcomes become the next training set.CollectWho is sampled?LabelWhose judgement?ModelWhat is optimised?DeployWhich threshold?LoopWhat is logged?outcomes retrain the score
Each box poses the question an auditor asks at that stage. The dashed return path is the reason a bias found after deployment is also evidence about collection, since logged outcomes become the next training set.
  • 2Locate at least two stages where the gap could enter.
  • 2Name the evidence that would confirm or rule out each.
  • 2Explain why an architecture claim does not answer a data question.
Collection is the first suspect: if the training recordings contained little code-switched speech, the model fitted the majority mode, and the evidence needed is the composition of the training data by language pattern. Labelling is the second: transcribers unfamiliar with mixed-language speech may have produced noisy reference transcripts, so ask for agreement rates on code-switched clips. Evaluation is a third, since a benchmark without code-switched speech would never have shown the gap before release. The vendor's claim is about model capacity, but nothing about the architecture can supply speech patterns the data never contained, so it answers a question nobody asked.
Sia tip — When a group's error rate is several times another's, ask first what share of the training and evaluation data that group made up; architecture is rarely the first answer.
Glossary

Key terms

Measurement Bias
The gap between a construct such as need or risk and the instrument used to measure it, such as spending or arrest records.
Historical Bias
Inequality in past access to jobs, credit, housing or policing that enters a model as labels, features and base rates even when every record is accurate.
Unawareness
The practice of excluding a protected attribute from a model, which fails when proxies remain and also blocks the audit of disparities.
Missing Not at Random
A pattern in which whether a value is missing depends on the unobserved value itself, so that the gap in the data is itself a signal.
Intersectional Slice
A subgroup defined by two or more attributes at once, such as skin type and gender, used because single-attribute averages can hide the worst-served cell.
Rater Disagreement
Variation among annotators labelling the same item, which carries information about contested categories and is lost when a single majority label is forced.
FAQ

How Bias Enters the Data Lifecycle FAQ

Why does removing race or gender from the data not remove bias?

Because other inputs carry the same information. Districts encode residential segregation, names correlate with gender and ethnicity, and language patterns track communities, so a model can rebuild the attribute's effect without ever seeing it. Removing the attribute also makes it harder to measure the gap. The lecture's check is to train a second model to predict the attribute from the remaining inputs.

What is the difference between MCAR, MAR and MNAR?

They describe why data is missing. Missing completely at random is unrelated to anything and is rare in social data. Missing at random depends only on things you observe, so careful imputation can help. Missing not at random depends on the missing value itself, such as people with low income skipping an income question, and it is the case the lecture treats as a fairness failure.

What does intersectionality mean in a fairness audit?

It means reporting results for combinations of attributes, not only for each attribute separately. In the face-analysis benchmarks, averages by gender and by skin type each looked better than the combination of darker skin and female, where error reached 34.7%. The lecture credits the idea to Crenshaw and asks for the full cross of attributes first, followed by caveats about small cells.

Study strategy

Assessment move

Draw the five-stage lifecycle from memory and, under each stage, write one question you would ask and one case from the lecture that failed there. Then take a system you know and walk it through the five questions in order. Most written answers lose marks by naming bias without saying where it entered; naming the stage is what turns a complaint into something an engineer can fix.

Working through How Bias Enters the Data Lifecycle in MAIE5103? Sia is AskSia’s AI Arts and Humanities tutor — ask any MAIE5103 How Bias Enters the Data Lifecycle question and get a clear, step-by-step explanation grounded in how MAIE5103 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 13 of your subjects at The Hong Kong University of Science and Technology, plus 1,000+ more Bibles across the AskSia library.
Sia - your MAIE5103 tutor, unlimited, worked the way the exam marks it
The full 5-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works