POLS1009 Chap.10 Operationalisation, Validity and Reliability
Operationalisation, Validity and Reliability
Week 7 asks how political scientists turn ideas they cannot see into things they can record, and how to judge whether the result is any good. Operationalisation runs from a concept through its attributes to observable indicators and then to variables, and several indicators can be combined into an index whose weighting choices shape every score it produces.
Two questions then test any measure: validity, which is about accuracy against the concept, and reliability, which is about consistency on repetition. The chapter separates five kinds of validity and three reliability checks, explains why the two properties are independent, and shows how measurement error comes in a random form that cancels and a systematic form that shifts the answer.
It closes with the trustworthiness criteria used for qualitative evidence and the four survey response formats, from dichotomous to continuous. Measurement and paragraphs open the second half of the semester, so this vocabulary sits squarely inside the end-of-semester exam, and the habit it builds, asking how a study measured what it claims, carries into every reading you meet afterwards.
What this chapter covers
- 01
Concept, attribute, indicator and variable as one chain
- 02
Building an index and declaring its weights
- 03
Validity as accuracy and reliability as consistency
- 04
Construct, face, content, convergent and discriminant validity
- 05
Test and retest, inter-coder agreement and internal consistency
- 06
Random error against systematic error
- 07
Credibility, transferability, dependability and confirmability
- 08
Dichotomous, categorical, ordinal and continuous response formats
Worked example · free
Placing four measures on the reliability and validity targets
- 1Measure 1. Official returns record the quantity itself and a second team reproduces the count, so the measure is both reliable and valid.
- 1Measure 2. Re-coding gives the same numbers, so the score is consistent, but newspaper counts track population size rather than press freedom: reliable, not valid.
- 1Measure 3. The estimates are right on average yet scattered case by case, so they are valid but not reliable for any single protest.
- 1Measure 4. Visitor numbers do not capture trust, a construct problem, and inconsistent counting rules make them unreliable too: neither.
Key terms
- Indicator
- An observable sign chosen to stand for one attribute of a concept, such as whether a council publishes its budget online as a sign of transparency.
- Index
- A single variable built by combining several indicators of a multi-part concept, whose score depends on which indicators are included and how each is weighted.
- Construct validity
- The degree to which a set of indicators really reflects the underlying quality it claims to capture, a test that matters most when the concept cannot be observed directly.
- Systematic error
- Bias that moves every recorded value in the same direction away from the truth, so averaging more observations does not remove it.
- Trustworthiness
- The standard Lincoln and Guba propose for qualitative research, assessed through credibility, transferability, dependability and confirmability.
Operationalisation, Validity and Reliability FAQ
What is the difference between validity and reliability?
Validity is about accuracy: whether a measure captures the concept you intend rather than something else. Reliability is about consistency: whether repeating the measure across time, coders or groups gives the same result. The two are independent, so a measure can be consistent and still wrong, and a measure that is right on average can still be too noisy for one case.
Can a measure be reliable but not valid?
Yes. A measure that is off by the same amount every time is perfectly consistent and still inaccurate, so it is reliable without being valid. This is why reliability is described as necessary but not sufficient: agreement between repeated measurements says nothing about whether they track the concept you care about.
What are the kinds of validity in political research?
The Week 7 lecture names five. Construct validity asks whether indicators reflect the intended quality, face validity whether the measure looks plausible, content validity whether it covers every dimension, convergent validity whether measures of one concept agree, and discriminant validity whether measures of different concepts diverge as expected.
How does random error differ from systematic error?
Random error is noise that varies unpredictably from case to case and tends to cancel out on average, but it makes estimates less precise and real relationships harder to detect. Systematic error pushes values the same way every time, so it does not cancel and can lead a study to report a wrong answer with confidence.
How is quality judged in qualitative research?
Through trustworthiness, following Lincoln and Guba. Credibility replaces internal validity, transferability replaces external validity, dependability replaces reliability and confirmability replaces objectivity. Researchers support these by showing how closely findings fit the setting studied, describing that setting fully, documenting their design and keeping an audit trail of analytical decisions.
Exam move
Treat this chapter as two grids to memorise and one habit to practise. The first grid pairs each kind of validity with a one-line test and an example you invent yourself. The second pairs each qualitative trustworthiness criterion with the quantitative test it replaces.
The habit is reading any measure in four steps: name the concept, name the indicator, ask whether repetition would give the same value, and ask which way any error would push the result. Practise on survey items and indices you meet in readings, classifying each response format as you go.
When two terms feel interchangeable, write a single sentence that only one of them could complete, and keep that sentence on your revision sheet.
Working through Operationalisation, Validity and Reliability in POLS1009? Sia is AskSia’s AI Arts and Humanities tutor — ask any POLS1009 Operationalisation, Validity and Reliability question and get a clear, step-by-step explanation grounded in how POLS1009 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.