SOST10021 Chap.6 Survey Design, Sampling and Inference
Survey Design, Sampling and Inference
Survey Design, Sampling and Inference follows evidence from a target population through a sampling frame, invited sample, respondents and analysed cases. Each transition introduces a different risk. Undercoverage excludes eligible units from selection, probability sampling creates quantifiable variation, and non-response can reshape the achieved sample.
Weighting can restore unequal selection chances or align known margins but cannot guarantee correction for unmeasured differences. Confidence intervals describe sampling uncertainty under assumptions rather than every source of survey error. Statistical tests must be read alongside effect size, measurement quality and research design.
What this chapter covers
- 01
Target populations
- 02
Sampling frames
- 03
Probability sampling
- 04
Non-response
- 05
Survey weighting
- 06
Confidence intervals
- 07
Hypothesis testing
Worked example · free
Read a sampling funnel without merging its losses
- 2Calculate response as 620 divided by 1,000, or 62%.
- 2Identify 800 target members absent from the frame, a separate coverage loss.
- 2Explain that non-response bias depends on differences between respondents and non-respondents, not the rate alone.
- 1Keep sampling, coverage and response uncertainty distinct. The weighting is an original revision aid.
Key terms
- Probability sample
- A probability sample gives each eligible unit a known nonzero chance of selection under the design.
- Coverage error
- Coverage error arises when the sampling mechanism omits eligible units or includes ineligible ones.
- Survey weight
- A survey weight represents selection and adjustment information used to estimate a target population quantity.
Survey Design, Sampling and Inference FAQ
Does a large sample guarantee representativeness?
No. A large sample can estimate the characteristics of selected participants precisely while coverage, volunteer selection or non-response remains systematic. Representativeness depends on the path from target population to analysed cases, not the observation count alone.
What does a response rate reveal?
A response rate reports participation among eligible invited cases under a stated rule. Bias depends on whether respondents and non-respondents differ on relevant outcomes after adjustment. Compare frame information, contact histories and weighted estimates rather than judging from one percentage.
What does a confidence interval leave out?
A conventional confidence interval represents sampling variation under its model and design. It may not capture undercoverage, leading wording, measurement error, unmeasured confounding or model misspecification. Interpret interval width together with the full evidence process.
What can survey weighting correct?
Weights can adjust unequal selection probabilities and align respondents with known population margins when the required variables are observed and models are credible. They cannot recover people missing on unmeasured characteristics or repair faulty questions. Report the weighting basis, design effect and sensitivity of substantive conclusions.
Exam move
Draw the sampling funnel for every survey example: target population, frame, selected sample, respondents and analytic cases. Write the relevant count and loss mechanism at each transition. Recalculate coverage and response rates with their own denominators, then explain why neither percentage alone establishes bias.
Compare simple random, stratified and clustered selection by asking how inclusion probabilities and uncertainty change. Practise interpreting confidence intervals procedurally and separate their sampling meaning from total survey error. For hypothesis tests, report the estimate, effect size and interval before any threshold label, then assess substantive importance.
Create counterexamples in which a very large volunteer sample is less representative than a smaller probability sample. Add weighting and record what known imbalance it addresses plus what remains unmeasured. When design information is absent, narrow the claim to observed participants rather than inventing population coverage.
Maintain error categories for frame undercoverage, unknown selection, non-response, incorrect standard errors and significance treated as causation. Finish each exercise by matching remedy to failure: improve the frame for coverage, contact for response, question testing for measurement and design-aware estimation for clustering. This keeps statistical inference connected to how the data were produced.
Add design sketches for stratified and clustered samples, noting where independence assumptions change. Use the same point estimate under two effective sample sizes and compare interval width. Create a response-bias scenario in which respondents and non-respondents have different outcomes, then show why response rate alone cannot determine the direction.
Test a weighting adjustment with known population margins and explain the unmeasured difference it cannot fix. For each hypothesis result, write a substantive interpretation without the word significant, then add the formal test meaning. Compare exploratory subgroup analysis with a planned primary comparison and identify the need for confirmation.
Before the exam, practise one-minute diagnoses: name target, frame, design, response, weight and uncertainty from a short study description. End with the one survey error not represented in the confidence interval.
Working through Survey Design, Sampling and Inference in SOST10021? Sia is AskSia’s AI Statistics tutor — ask any SOST10021 Survey Design, Sampling and Inference question and get a clear, step-by-step explanation grounded in how SOST10021 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.