MAIE5103 Chap.7 Bias and Safety in Generative AI
Bias and Safety in Generative AI
Old harms in new shapes
Generative systems carry the familiar fairness problems in three forms the lecture names. Stereotypes appear when occupations, dialects and regions default to a majority prototype in generated text and images. Omission appears when some languages, legal systems and histories are simply absent, which is why the lecturer's group built a model aimed at Hong Kong's multilingual and legal context.
Allocation appears when generated advice, rankings or code end up denying someone an opportunity, so audits examine both the prompts people send and the outputs they get.
Whose preferences become the reward
Preference training turns human judgements into a reward signal, and the lecture asks whose judgements those are.
Helpfulness and harmlessness are different goals, and collapsing them into one scalar reward hides the trade-off. Rater pools, rating instructions and severity labels are therefore fairness decisions.
Retrieval raises the same issue one step earlier: a generator can only be as fair as the documents it retrieves, so index coverage by language, jurisdiction and publisher is part of the audit.
Bias is one failure among several
Drawing on the NIST generative AI profile, the Beyond Bias lecture lists confabulation, privacy and security failures, over-reliance on fluent answers, and value-chain and environmental risks beside bias.
It argues that an application's permissions and workflow decide how a model error becomes a harm, and it recommends defence in depth: controls at the inputs, the evidence, the output, the decision boundary and any action a system can take. Red-teaming finds failures but cannot prove their absence, and paired-prompt tests are probes rather than verdicts.
What this chapter covers
- 01
Stereotype, omission and allocation in generated outputs
- 02
Preference data and the single scalar reward
- 03
Bias in vision models and multimodal safety data
- 04
Retrieval coverage as a fairness control
- 05
Tensions between fairness, safety and privacy
- 06
Confabulation, reliance and the NIST risk list
- 07
Defence in depth across an application
- 08
Paired-prompt probes and the limits of red-teaming
Worked example · free
Decide what a red-team result can claim
- 2Say what a red-team exercise can and cannot establish.
- 2Identify two weaknesses in this exercise.
- 2Rewrite the website claim so it matches the evidence.
Key terms
- Confabulation
- A generative system producing confident but false facts, citations or advice.
- Scalar Reward
- A single number used to train a model on human preferences, which can hide the trade-off between helpfulness and harmlessness.
- Retrieval Coverage
- How fully a system's document index represents the languages, jurisdictions and publishers its users need.
- Paired-Prompt Probe
- A test that sends matched prompts differing in one identity cue and compares the outputs under fixed settings and blind scoring.
- Red-Teaming
- Structured adversarial testing under a written threat model, which finds failures but cannot prove their absence.
- Defence in Depth
- Layered controls on inputs, evidence, outputs, decision boundaries and actions, each with its own failure test.
Bias and Safety in Generative AI FAQ
How can a large language model be biased if it is not making decisions?
Its outputs can still stereotype, omit and allocate. Generated text and images can default to majority prototypes, leave out whole languages or legal systems, and give advice or rankings that steer people away from opportunities. The lecture also points out that application permissions decide whether an output becomes an action, so an assistant that can send, book or reject can turn a biased answer into a consequential one.
Does retrieval-augmented generation make answers accurate?
Not on its own. Grounding answers in retrieved documents can reduce some errors, but the lecture warns that it is not a truth guarantee: documents can be stale, conflicting or malicious, and the generator can only be as fair as the index. Check that cited sources support the claim, audit coverage by language and jurisdiction, and test how the system recovers when retrieval fails.
What should a bias test for a chatbot look like?
A paired-prompt probe: matched prompts that differ in one identity cue, with the model version, system prompt, retrieval, sampling settings and language held fixed, many templates and runs, and evaluators who cannot see the cue scoring the answers on a rubric. Report the variation, not just the average, and state the limits, since a name is an ambiguous proxy and one pair cannot represent a group.
Assessment move
Pick one generative application you use and draw its chain from input to action, then write one failure and one control at each hop. Next, write a short paired-prompt protocol for it in five lines. These two exercises cover most of what a reading response or case study on generative systems will ask, and they keep you from the common error of auditing only the model.
Working through Bias and Safety in Generative AI in MAIE5103? Sia is AskSia’s AI Arts and Humanities tutor — ask any MAIE5103 Bias and Safety in Generative AI question and get a clear, step-by-step explanation grounded in how MAIE5103 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.