DATA1001 Chap.1 Study Design
Study Design
Before calculation, study design determines what a result can support. Start by identifying the target population, sampled units, exposure or treatment, response and any grouping variables. Variable meaning determines whether a measurement is categorical or quantitative, even when categories are stored as numbers. Random sampling and random assignment do different jobs.
Random sampling supports generalisation from a sample to a population when the frame and response process are credible. Random assignment tends to balance known and unknown confounders, supporting a causal comparison between treatments. An observational association does not become causal merely because it is strong, statistically significant or based on many observations.
A confounder is present when a variable is related to both exposure and outcome through a route that can distort their observed association. Selection, nonresponse and measurement bias can move an estimate systematically away from its target, and increasing sample size does not repair that direction error. Canvas Topic 1, Design Experiments, organises this material through its Imagine and Discover resources.
Across the supplied papers, the topic appears in conceptual multiple choice and in extended critiques involving privacy, confounding, representativeness and proposed experiments. A complete exam answer names the design, states the strongest legal conclusion, identifies a specific threat and recommends a repair.
Useful repairs include probability sampling for population reach, random assignment for causal reach, blinding for measurement quality and consistent instruments for fair comparison. Treat ethical reasoning as part of design: explain consent, privacy, fairness, harm and transparent communication in the facts of the study rather than listing abstract principles.
What this chapter covers
- 01
Topic 1: Design Experiments
- 02
Classify variables from their meaning
- 03
Separate random sampling from random assignment
- 04
Identify confounding and causal limits
- 05
Diagnose selection, nonresponse and measurement bias
- 06
Apply privacy, consent, fairness and transparency
Worked example · free
Read the design before the percentage
- 1The sample is voluntary and the exposure is not randomly assigned.
- 1The 68 percent describes responders, not necessarily all users.
- 1Motivation may affect both reminder use and reported sleep, so it is a plausible confounder.
- 1Use a probability sample for user prevalence and a randomised reminder offer for a causal comparison.
Key terms
- Observational study
- A study that observes exposure rather than assigning it randomly, so confounding limits causal claims.
- Randomised experiment
- A study that assigns treatment by chance so comparison groups tend to balance confounders.
- Confounder
- A variable associated with both exposure and outcome that can distort their observed relationship.
- Selection bias
- Systematic error caused when included units differ from the target population in a relevant way.
- Nonresponse bias
- Systematic error caused when people who do not respond differ meaningfully from those who do.
- Measurement bias
- Systematic error introduced by an instrument, question or recording process that pushes values away from truth.
Study Design FAQ
When may I use causal language?
Use causal language when treatment was randomly assigned and the comparison was otherwise conducted appropriately. Observational evidence can support association but leaves confounding open. A small p value does not upgrade an observational design into an experiment, so name the design before choosing words such as causes or changes.
Does a large sample remove bias?
No. A large sample usually reduces chance variation in a statistic, but it does not repair a sampling frame or measurement process that is systematically wrong. Ask who could enter the sample and who actually responded before taking precision as evidence of population validity.
How do random sampling and random assignment differ?
Random sampling concerns who enters the study and supports population generalisation. Random assignment concerns which treatment enrolled units receive and supports causal comparison. A study can have one without the other, so state the population reach and causal reach separately in an exam answer.
How do I identify a confounder?
Propose a variable that is plausibly related to the exposure and also related to the outcome through another route. Explain both links in the story. Merely naming an extra variable is not enough, and a variable caused by the exposure is not serving the same confounding role.
How does ethics enter an exam answer?
Connect the principle to the data practice. Explain what privacy could be lost, whether consent covers the intended use, who might be harmed or excluded, and how uncertainty should be communicated. Specific consequences are stronger than a list of ethical words without application.
Exam move
Use a four line template on every study story. First, identify the target population, sample, exposure and response. Second, name the design and say whether random sampling or random assignment occurred. Third, state the strongest supported conclusion, separating population generalisation from causation. Fourth, identify one specific threat and one feasible repair.
Practise converting headlines that say causes into accurate association language, then design a randomised comparison that would justify a stronger claim. Keep random sampling and random assignment on separate flashcards because they solve different problems. For bias items, classify the mechanism as selection, nonresponse or measurement and explain the direction it could push the result when possible.
For ethics items, write one sentence each on privacy, consent, fairness, harm and transparent reporting in the actual setting. Finish by doing the two chapter multiple choice questions without looking at the answer, then rewrite the extended response in fewer than five precise sentences.