University of Birmingham · FACULTY OF PSYCHOLOGY

13344 Chap.3 Measuring Mental Health: Validity and Reliability

- one subject, every graph, every model, every mark
6 Chapters4-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 3 of 10 · 13344

Measuring Mental Health: Validity and Reliability

Accuracy and stability

The chapter defines validity and reliability, explains why validity needs reliability but not the reverse, and compares categorical and continuous measures and their statistical power.

Properties and screening

It names six kinds of validity and three kinds of reliability, sets out how existing measures are chosen and new ones built, and works through sensitivity, specificity and predictive values, showing why prevalence changes what a positive screen means.

Calculations to practise

Two worked tables build the screening measures from counts, one in the community and one in a specialist clinic, and a curve shows how the positive predictive value of a fixed screen rises with prevalence.

Practise until you can set up the table without looking.

In this chapter

What this chapter covers

  • 01

    Validity and reliability

  • 02

    Categorical and continuous data

  • 03

    Six kinds of validity

  • 04

    Test-retest, inter-rater and internal consistency

  • 05

    Choosing or building a measure

  • 06

    Sensitivity, specificity, PPV and NPV

Worked example · free

Predictive value of a screen at 5% prevalence

Q [4 marks]. AskSia-authored practice. A screen with sensitivity 0.85 and specificity 0.90 is used on 2,000 people, 5% of whom have the condition. Find the positive predictive value. The marks shown are an AskSia study allocation, not a University marking scheme.
One screen, many predictive values.One screen, many predictive values. The curve plots positive predictive value against prevalence for a screen with sensitivity 0.85 and specificity 0.90; the marked point shows that at 10% prevalence only about half of positive screens are true cases. 0% 25% 50% 0 0.5 1.0 Prevalence in the screened group PPV 10%: PPV 0.49
One screen, many predictive values. The curve plots positive predictive value against prevalence for a screen with sensitivity 0.85 and specificity 0.90; the marked point shows that at 10% prevalence only about half of positive screens are true cases.
  • 1Cases = 5% of 2,000 = 100; true positives = 0.85 × 100 = 85.
  • 1Non-cases = 1,900; false positives = 0.10 × 1,900 = 190.
  • 1Positive screens = 85 + 190 = 275.
  • 1PPV = 85 ÷ 275 ≈ 0.31.
Only about 31% of positive screens are true cases at 5% prevalence, even though sensitivity and specificity are high. This matches the point at 5% on the predictive value curve and shows why rare conditions call for a confirmatory second phase. In a specialist clinic where half of referrals are cases, the same screen would give a positive predictive value close to 0.9.
Sia tip — Build the two by two table from the prevalence first, then divide by the column you need.
Glossary

Key terms

Validity
Validity is a measure's accuracy as a measure of the thing it is supposed to measure.
Cronbach's Alpha
Cronbach's Alpha is the statistic used to measure internal consistency, whether items reflect the total score.
Sensitivity
Sensitivity is the proportion of true cases that a screen detects.
Specificity
Specificity is the proportion of true non-cases that a screen correctly identifies.
Predictive Validity
Predictive Validity is how well a measure predicts later events connected to the construct.
Inter-Rater Reliability
Inter-Rater Reliability is the extent to which different observers or interviewers arrive at the same score.
FAQ

Measuring Mental Health: Validity and Reliability FAQ

Can a measure be reliable but not valid?

Yes. Reliability means a measure gives stable, repeatable scores; validity means it measures the intended construct. A scale can consistently capture something related, such as threat instead of anxiety, so validity needs reliability but reliability alone is not enough.

Why do predictive values change between settings?

Positive and negative predictive values depend heavily on prevalence. The same screen gives many false positives where a condition is rare and far fewer where it is common, which is why a positive result means more in a specialist clinic than in the community.

Should I prefer continuous or categorical measures?

The teaching favours continuous measures of dimensional traits because they offer more statistical power, while warning that psychiatric scales rarely behave arithmetically. Categorical measures still suit diagnoses and where a clear threshold for caseness is needed.

How is a new measure developed?

The module sets out seven steps: define the construct, review the definition with experts and potential participants, draft items, have them reviewed, run alpha testing for reliability and item reduction, run beta testing for validity and factor structure, and then let use in other settings settle its validity.

What is a two-phase survey?

A survey that first screens everyone with a short measure and then gives likely cases a full clinical assessment. It is used for rarer conditions, because a single screen would produce many false positives and a full interview for everyone would be too costly.

What should I check before choosing an existing measure?

The module's checklist asks whether the measure is conceptually relevant, comparable with other studies, comprehensible to the people completing it, including in translation, feasible alongside your other measures, and validated with published reliability and validity evidence. Participant burden matters too, especially for people who are acutely unwell.

Study strategy

Assessment move

Make flash cards pairing each validity and reliability type with the question it answers, and practise one screening table a week, changing only the prevalence to see how the predictive values move. When you cite a measure in an essay, name one validity property and one reliability property that has been tested, rather than calling it valid and reliable.

Working through Measuring Mental Health: Validity and Reliability in 13344? Sia is AskSia’s AI Psychology tutor — ask any 13344 Measuring Mental Health: Validity and Reliability question and get a clear, step-by-step explanation grounded in how 13344 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 17 of your subjects at University of Birmingham, plus 1,000+ more Bibles across the AskSia library.
Sia - your 13344 tutor, unlimited, worked the way the exam marks it
The full 4-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works