University College London · FACULTY OF EDUCATION

EDPS0007 Chap.4 Conceptualisation, Validity and Reliability

- one subject, every graph, every model, every mark
7 Chapters5-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 4 of 10 · EDPS0007

Conceptualisation, Validity and Reliability

Conceptualisation, Validity and Reliability explains why numbers about education are never automatically objective. Conceptualising means defining and clarifying an idea, which is easy for concrete concepts such as height and hard for abstract ones such as a great student or a successful school.

The PISA reading test shows the stakes: reading comprehension is split into retrieving, interpreting and reflecting, and changing the share of questions in each part reorders countries, moving Denmark from 19th to 1st and Canada from 4th to below average. Validity asks whether a tool measures what it claims to measure; reliability asks whether it gives the same result twice, which the lecture calls the hardest part.

The chapter adds self-report problems, the multiple questions used to measure loneliness and wellbeing, and a step-by-step routine for judging any measure, from a book count at home to take-home essays in the age of AI.

In this chapter

What this chapter covers

  • 01

    Concrete and abstract concepts

  • 02

    PISA and the definition of reading comprehension

  • 03

    Validity as accuracy

  • 04

    Reliability as consistency

  • 05

    Self-report and social desirability

  • 06

    Measuring loneliness and wellbeing with several questions

  • 07

    A routine for judging any measure

Worked example · free

Decide whether a consistent wellbeing measure is valid or merely reliable

Q [4 marks]. A school measures pupil wellbeing by asking every pupil, with teachers watching, "Are you happy at school?" Results are almost identical each term. Is the measure reliable, valid, both or neither? Explain why consistently similar results can still be consistently wrong. The mark allocation here is our own practice weighting, not an official UCL scheme.
Each target shows six attempts to measure the same thing: tight clusters are reliable, clusters centred on the bullseye are valid, and a tight cluster far from Each target shows six attempts to measure the same thing: tight clusters are reliable, clusters centred on the bullseye are valid, and a tight cluster far from the centre is consistently wrong.Reliable and validReliable, not validValid on average, unreliableNeither
Each target shows six attempts to measure the same thing: tight clusters are reliable, clusters centred on the bullseye are valid, and a tight cluster far from the centre is consistently wrong.
  • 1Reliability: results repeat closely across terms, so the measure looks reliable.
  • 1Validity: one yes or no item captures only a sliver of an abstract concept, and pupils watched by teachers may answer as expected.
  • 1Link the two: a measure can cluster tightly away from the true value, consistently wrong rather than accurate.
  • 1Improve it: use several anonymous questions on a 0 to 10 scale, as wellbeing and loneliness measures do.
The measure is reliable but probably not valid. Its stable results may reflect social pressure and a narrow definition rather than wellbeing, so it is consistently wrong; several anonymous items would cover more of the concept.
Sia tip — Test validity before reliability: a stable number is only useful if it measures the concept you defined.
Glossary

Key terms

Conceptualisation
Defining and clarifying an idea so that it can be studied and measured.
Validity
The extent to which a tool accurately measures what it is supposed to measure.
Reliability
The consistency of a measure, so that different markers or occasions give the same result.
Functional literacy
Reading, writing and basic calculation sufficient for everyday tasks in a community and workplace.
FAQ

Conceptualisation, Validity and Reliability FAQ

Why does PISA's definition of reading matter?

Because the result depends on it. Lecture 4 shows that shifting the mix of retrieving, interpreting and reflecting questions reorders countries, and there is no overwhelmingly strong reason for PISA's chosen mix, so a subjective decision shapes seemingly objective rankings.

Can a measure be reliable but not valid?

Yes. Reliability means you keep getting the same answer; validity means the answer is accurate. Self-report questions on sensitive topics can be very consistent and still be consistently wrong, for example when people underestimate how much alcohol they drink.

Why use four questions to measure loneliness?

People understand the word loneliness differently, and some are reluctant to admit it. Several questions capture different aspects of the concept, which is why the national statistics guidance combines three scale items with one direct question.

What separates a concrete concept from an abstract one?

Concrete concepts, such as height, can be defined and measured directly with little dispute. Abstract ones, such as wealth, a great student or a successful school, are contested, so researchers must choose a definition, and different choices can produce different results.

Study strategy

Exam move

Make the five-step routine automatic: name the concept, find the definition used, test validity, test reliability and ask what was left out. Practise it on four measures the lecture invites you to judge: print books at home as a measure of the home learning environment, the SAT as a predictor of university success, IQ tests as a measure of intelligence and take-home essays in the age of AI.

For each, write one sentence on which step fails first and how a better study would fix it. Learn the PISA example precisely, including the 7, 14 and 7 split and how a different split changes the ranking, because it is a compact, memorable case for the student question.

When you criticise a study in an essay, name the weak step rather than calling the research simply flawed; that precision is what distinguishes analysis from dismissal. Pair each measure with a better alternative, such as several anonymous wellbeing items instead of one supervised question, so that every criticism you make arrives with a constructive suggestion attached.

Working through Conceptualisation, Validity and Reliability in EDPS0007? Sia is AskSia’s AI Education tutor — ask any EDPS0007 Conceptualisation, Validity and Reliability question and get a clear, step-by-step explanation grounded in how EDPS0007 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 32 of your subjects at University College London, plus 1,000+ more Bibles across the AskSia library.
Sia - your EDPS0007 tutor, unlimited, worked the way the exam marks it
The full 5-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works
EDPS0007 · Understanding Education Research: Numbers, Narratives, Knowledge and Nonsense - independent study guide on the AskSia Library. More University College London subjects · Microeconomics across all universities