The Hong Kong University of Science and Technology · FACULTY OF ARTS & HUMANITIES

MAIE5103 Chap.7 Bias and Safety in Generative AI

- one subject, every graph, every model, every mark
8 Chapters3-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 7 of 14 · MAIE5103

Bias and Safety in Generative AI

Old harms in new shapes

Generative systems carry the familiar fairness problems in three forms the lecture names. Stereotypes appear when occupations, dialects and regions default to a majority prototype in generated text and images. Omission appears when some languages, legal systems and histories are simply absent, which is why the lecturer's group built a model aimed at Hong Kong's multilingual and legal context.

Allocation appears when generated advice, rankings or code end up denying someone an opportunity, so audits examine both the prompts people send and the outputs they get.

Whose preferences become the reward

Preference training turns human judgements into a reward signal, and the lecture asks whose judgements those are.

Helpfulness and harmlessness are different goals, and collapsing them into one scalar reward hides the trade-off. Rater pools, rating instructions and severity labels are therefore fairness decisions.

Retrieval raises the same issue one step earlier: a generator can only be as fair as the documents it retrieves, so index coverage by language, jurisdiction and publisher is part of the audit.

Bias is one failure among several

Drawing on the NIST generative AI profile, the Beyond Bias lecture lists confabulation, privacy and security failures, over-reliance on fluent answers, and value-chain and environmental risks beside bias.

It argues that an application's permissions and workflow decide how a model error becomes a harm, and it recommends defence in depth: controls at the inputs, the evidence, the output, the decision boundary and any action a system can take. Red-teaming finds failures but cannot prove their absence, and paired-prompt tests are probes rather than verdicts.

In this chapter

What this chapter covers

  • 01

    Stereotype, omission and allocation in generated outputs

  • 02

    Preference data and the single scalar reward

  • 03

    Bias in vision models and multimodal safety data

  • 04

    Retrieval coverage as a fairness control

  • 05

    Tensions between fairness, safety and privacy

  • 06

    Confabulation, reliance and the NIST risk list

  • 07

    Defence in depth across an application

  • 08

    Paired-prompt probes and the limits of red-teaming

Worked example · free

Decide what a red-team result can claim

Q [6 marks]. AskSia-authored practice. A start-up red-teams its legal-information chatbot with 300 adversarial prompts written by its own engineers, finds 12 unsafe answers, fixes them, reruns the same 300 prompts and finds none. Its website now says the chatbot is 'proven safe for legal questions'. Assess the claim using the lecture's account of red-teaming. The marks shown are an AskSia study allocation, not a University marking scheme.
An application is a chain, and each hop admits its own failure.An application is a chain, and each hop admits its own failure. A model error becomes a consequential harm only at the last hop, when the workflow lets an answer become an action that nobody reviews.CV and promptunneededpersonal dataRetrievaluntrusted instructionsModel and policystereotyped adviceAnswerinvented requirementsActionunreviewedconsequence
An application is a chain, and each hop admits its own failure. A model error becomes a consequential harm only at the last hop, when the workflow lets an answer become an action that nobody reviews.
  • 2Say what a red-team exercise can and cannot establish.
  • 2Identify two weaknesses in this exercise.
  • 2Rewrite the website claim so it matches the evidence.
Red-teaming samples failures; it cannot prove that none remain, so 'proven safe' overstates what any such exercise could show. This one also falls short of the lecture's process in two ways. The attackers were all the company's own engineers, not a diverse group, so whole categories of misuse may never have been tried. And passing a retest on the same 300 prompts shows only that those twelve failures were fixed, not that similar prompts would now be handled. An honest claim would read: twelve unsafe answers found in an internal test of 300 prompts were fixed, and further testing by outside reviewers is planned.
Sia tip — Treat any red-team number as a count of failures found, never as a safety rate; write 'found and fixed', not 'proven safe'.
Glossary

Key terms

Confabulation
A generative system producing confident but false facts, citations or advice.
Scalar Reward
A single number used to train a model on human preferences, which can hide the trade-off between helpfulness and harmlessness.
Retrieval Coverage
How fully a system's document index represents the languages, jurisdictions and publishers its users need.
Paired-Prompt Probe
A test that sends matched prompts differing in one identity cue and compares the outputs under fixed settings and blind scoring.
Red-Teaming
Structured adversarial testing under a written threat model, which finds failures but cannot prove their absence.
Defence in Depth
Layered controls on inputs, evidence, outputs, decision boundaries and actions, each with its own failure test.
FAQ

Bias and Safety in Generative AI FAQ

How can a large language model be biased if it is not making decisions?

Its outputs can still stereotype, omit and allocate. Generated text and images can default to majority prototypes, leave out whole languages or legal systems, and give advice or rankings that steer people away from opportunities. The lecture also points out that application permissions decide whether an output becomes an action, so an assistant that can send, book or reject can turn a biased answer into a consequential one.

Does retrieval-augmented generation make answers accurate?

Not on its own. Grounding answers in retrieved documents can reduce some errors, but the lecture warns that it is not a truth guarantee: documents can be stale, conflicting or malicious, and the generator can only be as fair as the index. Check that cited sources support the claim, audit coverage by language and jurisdiction, and test how the system recovers when retrieval fails.

What should a bias test for a chatbot look like?

A paired-prompt probe: matched prompts that differ in one identity cue, with the model version, system prompt, retrieval, sampling settings and language held fixed, many templates and runs, and evaluators who cannot see the cue scoring the answers on a rubric. Report the variation, not just the average, and state the limits, since a name is an ambiguous proxy and one pair cannot represent a group.

Study strategy

Assessment move

Pick one generative application you use and draw its chain from input to action, then write one failure and one control at each hop. Next, write a short paired-prompt protocol for it in five lines. These two exercises cover most of what a reading response or case study on generative systems will ask, and they keep you from the common error of auditing only the model.

Working through Bias and Safety in Generative AI in MAIE5103? Sia is AskSia’s AI Arts and Humanities tutor — ask any MAIE5103 Bias and Safety in Generative AI question and get a clear, step-by-step explanation grounded in how MAIE5103 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 13 of your subjects at The Hong Kong University of Science and Technology, plus 1,000+ more Bibles across the AskSia library.
Sia - your MAIE5103 tutor, unlimited, worked the way the exam marks it
The full 3-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works