University College London · FACULTY OF EDUCATION

EDPS0007 Chap.8 Evaluating Teachers: Bias and Accountability

- one subject, every graph, every model, every mark
7 Chapters4-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 8 of 10 · EDPS0007

Evaluating Teachers: Bias and Accountability

Evaluating Teachers: Bias and Accountability examines how the question "How good is this teacher?" is answered and why the answers can mislead. Evaluations may be formative, improving teaching, or summative, informing promotion or dismissal.

A French study found male students rating male instructors more highly although their students did slightly worse on a common anonymous exam, and a US online-module study showed the same teacher rated higher when presumed male, even on automated feedback timing. A randomised trial found that free chocolate cookies raised course evaluations.

Le Grand's distinction between knights and knaves frames the accountability debate: post-war trust in public servants gave way to a neoliberal turn of markets, reduced trust and measured performance. The chapter separates stereotyping, prejudice and discrimination, uses a false-name job application study to show how bias is measured, and asks whether teachers should be paid by results.

The material feeds the teacher half of the second exam question directly, and its research designs are among the clearest examples in the module of how bias can be measured.

In this chapter

What this chapter covers

  • 01

    Formative and summative evaluation

  • 02

    Gender bias in student ratings

  • 03

    The cookie randomised trial

  • 04

    Treatment and measured variables

  • 05

    Knights, knaves and the neoliberal turn

  • 06

    Stereotyping, prejudice and discrimination

  • 07

    Paying teachers by results

Worked example · free

Explain why student ratings may track believed gender rather than teaching

Q [4 marks]. In the online-module study, students never saw their lecturer, and the gender students believed their teacher had was varied. Ratings rose when a teacher was presumed male and fell when presumed female. Explain what this design rules out and what it implies for using ratings to answer "How good is this teacher?" The marks shown are a practice allocation, not an official UCL scheme.
Rows give the lecturer's actual gender and columns the gender students believed: ratings follow the believed gender, so the same teacher scores higher when presRows give the lecturer's actual gender and columns the gender students believed: ratings follow the believed gender, so the same teacher scores higher when presumed male and lower when presumed female.Seen as maleSeen as femaleActually femalehigher than as femalelower ratingsActually malehigher ratingslower than as male
Rows give the lecturer's actual gender and columns the gender students believed: ratings follow the believed gender, so the same teacher scores higher when presumed male and lower when presumed female.
  • 1The same teacher taught both groups, so differences in teaching quality are ruled out.
  • 1Only the believed gender changed, so it is the treatment variable driving the ratings.
  • 1Even an objective item, on-time feedback that was automated for everyone, was rated lower for presumed women.
  • 1Ratings therefore partly measure perceptions, so they should not decide promotion or pay without other evidence.
Because the teaching was identical and only the believed gender varied, the rating gap reflects bias in perception. Ratings remain useful for formative improvement but are weak grounds for summative decisions on their own.
Sia tip — Name the treatment variable and what was held constant; that is what turns a correlation into evidence of bias.
Glossary

Key terms

Summative evaluation
An evaluation used to make decisions such as continuing a programme or promoting a person.
Knave
Someone driven mainly by their own interests, financial or not.
Implicit bias
Biased actions or judgements that occur without conscious prejudice.
FAQ

Evaluating Teachers: Bias and Accountability FAQ

Are student evaluations of teaching biased?

Lecture 5 presents evidence that they can be: ratings followed presumed gender in an online module, male instructors were rated higher despite their students learning slightly less, and free cookies raised scores in a randomised trial.

What does knights and knaves mean?

It is Le Grand's contrast between public servants as self-sacrificing, trustworthy experts and as self-interested actors who need incentives and penalties. The lecture concludes that every role mixes both motives.

Should teachers be paid by their students' exam results?

The module poses this as a debate. Incentives might counter coasting and bias, but results also reflect intake, income and other factors, so a fair scheme would need valid, controlled measures.

How do formative and summative evaluation differ?

Formative evaluation gathers feedback in order to improve teaching, during a module or before it runs again. Summative evaluation feeds decisions such as continuing a programme or promoting a member of staff, so any bias it contains carries far higher stakes.

Study strategy

Exam move

Organise this chapter around one question: what does a rating measure? List the three evaluation studies from the lecture and, for each, note the design, what was held constant and the finding. Then write a balanced paragraph on knights and knaves that includes one argument for accountability, one against and the lecture's conclusion that roles mix both motives.

Learn the three-way distinction between stereotyping, prejudice and discrimination so you can use the terms precisely. For the exam question on teachers, plan an issues, challenges and recommendation structure: biased ratings as the issue, validity and stakes as the challenge, and a combination of formative use and multiple measures as a recommendation with its rationale.

Remember that you rate each UCL module you take, so your own experience of module surveys can serve as clearly labelled personal reflection. Close by drafting a two-sentence answer to the policy question of paying teachers by results, one sentence for the case and one for its validity problem, and keep it ready for the teacher essay.

Working through Evaluating Teachers: Bias and Accountability in EDPS0007? Sia is AskSia’s AI Education tutor — ask any EDPS0007 Evaluating Teachers: Bias and Accountability question and get a clear, step-by-step explanation grounded in how EDPS0007 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 32 of your subjects at University College London, plus 1,000+ more Bibles across the AskSia library.
Sia - your EDPS0007 tutor, unlimited, worked the way the exam marks it
The full 4-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works
EDPS0007 · Understanding Education Research: Numbers, Narratives, Knowledge and Nonsense - independent study guide on the AskSia Library. More University College London subjects · Microeconomics across all universities