The University of Manchester · FACULTY OF STATISTICS

SOST10021 Chap.6 Survey Design, Sampling and Inference

- one subject, every graph, every model, every mark
7 Chapters4-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 6 of 10 · SOST10021

Survey Design, Sampling and Inference

Survey Design, Sampling and Inference follows evidence from a target population through a sampling frame, invited sample, respondents and analysed cases. Each transition introduces a different risk. Undercoverage excludes eligible units from selection, probability sampling creates quantifiable variation, and non-response can reshape the achieved sample.

Weighting can restore unequal selection chances or align known margins but cannot guarantee correction for unmeasured differences. Confidence intervals describe sampling uncertainty under assumptions rather than every source of survey error. Statistical tests must be read alongside effect size, measurement quality and research design.

In this chapter

What this chapter covers

  • 01

    Target populations

  • 02

    Sampling frames

  • 03

    Probability sampling

  • 04

    Non-response

  • 05

    Survey weighting

  • 06

    Confidence intervals

  • 07

    Hypothesis testing

Worked example · free

Read a sampling funnel without merging its losses

Q [7 marks]. A target population contains 10,000 people, the frame lists 9,200, a probability sample invites 1,000 and 620 respond. Calculate the response rate and identify a separate coverage problem. The mark allocation shown is an original study device, not a University assessment scheme.
The funnel begins with 10,000 target members, loses 800 outside the frame, selects 1,000 people and ends with 620 respondents after 380 non-responses.The funnel begins with 10,000 target members, loses 800 outside the frame, selects 1,000 people and ends with 620 respondents after 380 non-responses.target populationN = 10,000sampling frame9,200 listedinvited sample1,000 selectedrespondents620 repliescoverage loss: 800sampling variationnon-response: 380
The funnel begins with 10,000 target members, loses 800 outside the frame, selects 1,000 people and ends with 620 respondents after 380 non-responses.
  • 2Calculate response as 620 divided by 1,000, or 62%.
  • 2Identify 800 target members absent from the frame, a separate coverage loss.
  • 2Explain that non-response bias depends on differences between respondents and non-respondents, not the rate alone.
  • 1Keep sampling, coverage and response uncertainty distinct. The weighting is an original revision aid.
The response rate is 62%. The frame covers 9,200 of 10,000 target members, leaving 800 outside selection before sampling begins. The response rate does not diagnose either coverage bias or outcome differences among non-respondents.
Sia tip — Use a separate denominator for coverage, selection and response; combining them hides where representativeness was lost.
Glossary

Key terms

Probability sample
A probability sample gives each eligible unit a known nonzero chance of selection under the design.
Coverage error
Coverage error arises when the sampling mechanism omits eligible units or includes ineligible ones.
Survey weight
A survey weight represents selection and adjustment information used to estimate a target population quantity.
FAQ

Survey Design, Sampling and Inference FAQ

Does a large sample guarantee representativeness?

No. A large sample can estimate the characteristics of selected participants precisely while coverage, volunteer selection or non-response remains systematic. Representativeness depends on the path from target population to analysed cases, not the observation count alone.

What does a response rate reveal?

A response rate reports participation among eligible invited cases under a stated rule. Bias depends on whether respondents and non-respondents differ on relevant outcomes after adjustment. Compare frame information, contact histories and weighted estimates rather than judging from one percentage.

What does a confidence interval leave out?

A conventional confidence interval represents sampling variation under its model and design. It may not capture undercoverage, leading wording, measurement error, unmeasured confounding or model misspecification. Interpret interval width together with the full evidence process.

What can survey weighting correct?

Weights can adjust unequal selection probabilities and align respondents with known population margins when the required variables are observed and models are credible. They cannot recover people missing on unmeasured characteristics or repair faulty questions. Report the weighting basis, design effect and sensitivity of substantive conclusions.

Study strategy

Exam move

Draw the sampling funnel for every survey example: target population, frame, selected sample, respondents and analytic cases. Write the relevant count and loss mechanism at each transition. Recalculate coverage and response rates with their own denominators, then explain why neither percentage alone establishes bias.

Compare simple random, stratified and clustered selection by asking how inclusion probabilities and uncertainty change. Practise interpreting confidence intervals procedurally and separate their sampling meaning from total survey error. For hypothesis tests, report the estimate, effect size and interval before any threshold label, then assess substantive importance.

Create counterexamples in which a very large volunteer sample is less representative than a smaller probability sample. Add weighting and record what known imbalance it addresses plus what remains unmeasured. When design information is absent, narrow the claim to observed participants rather than inventing population coverage.

Maintain error categories for frame undercoverage, unknown selection, non-response, incorrect standard errors and significance treated as causation. Finish each exercise by matching remedy to failure: improve the frame for coverage, contact for response, question testing for measurement and design-aware estimation for clustering. This keeps statistical inference connected to how the data were produced.

Add design sketches for stratified and clustered samples, noting where independence assumptions change. Use the same point estimate under two effective sample sizes and compare interval width. Create a response-bias scenario in which respondents and non-respondents have different outcomes, then show why response rate alone cannot determine the direction.

Test a weighting adjustment with known population margins and explain the unmeasured difference it cannot fix. For each hypothesis result, write a substantive interpretation without the word significant, then add the formal test meaning. Compare exploratory subgroup analysis with a planned primary comparison and identify the need for confirmation.

Before the exam, practise one-minute diagnoses: name target, frame, design, response, weight and uncertainty from a short study description. End with the one survey error not represented in the confidence interval.

Working through Survey Design, Sampling and Inference in SOST10021? Sia is AskSia’s AI Statistics tutor — ask any SOST10021 Survey Design, Sampling and Inference question and get a clear, step-by-step explanation grounded in how SOST10021 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 41 of your subjects at The University of Manchester, plus 1,000+ more Bibles across the AskSia library.
Sia - your SOST10021 tutor, unlimited, worked the way the exam marks it
The full 4-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works