GSOE9011 Chap.8 Experimental Design and Statistical Thinking
Experimental Design and Statistical Thinking
The course introduces statistical ways of thinking and formal experimental design rather than teaching statistical mathematics. The key task is to match a research claim with observations that could challenge it.
Students define experimental units, factors, levels, responses and controls; use randomisation to limit systematic assignment or order effects; use independent replication to estimate variability; and plan analysis and uncertainty language before seeing outcomes. Repeated measurements improve precision but do not automatically create new independent units.
What this chapter covers
- 01
Experimental units
- 02
Factors and levels
- 03
Response variables
- 04
Controls and blocking
- 05
Randomisation
- 06
Independent replication
- 07
Factorial interactions
- 08
Uncertainty and inference limits
Worked example · free
Finding the experimental unit in repeated measurements
- +1Identify the specimen as the independently treated experimental unit and the readings as repeated observations within it.
- +1Randomly assign coating and measurement order where feasible and preserve specimen identity in the data.
- +1Summarise repeated readings within specimen or use an analysis that represents their dependence.
- +1Base uncertainty on independent specimens and record nuisance factors such as preparation batch and temperature.
Key terms
- Experimental unit
- The smallest independent unit assigned to a treatment or condition for the inference.
- Factor
- An input deliberately varied to study its effect on a response.
- Response variable
- The measured outcome used to evaluate a research question.
- Randomisation
- Chance-based allocation or order used to reduce systematic bias.
- Replication
- Independent repetition that supports estimation of variability and repeatability.
- Interaction
- A pattern in which one factor's effect changes across levels of another factor.
Experimental Design and Statistical Thinking FAQ
What is the difference between replication and repeated measurement?
Replication uses independent experimental units. Repeated measurements observe the same unit several times. They may improve precision or describe change over time, but they share dependence and cannot be counted as new units.
Why is randomisation important?
Randomisation reduces the chance that treatment or order is systematically linked with another influence. It protects against allocation bias but does not repair poor measurement, inadequate controls or a mismatched response.
What does failure to reject a null hypothesis mean?
It means the data and analysis did not provide sufficient evidence against the null claim under the chosen procedure. It does not prove that no effect exists, especially when the design has low precision or narrow scope.
When should I use a factorial design?
Use it when more than one factor matters and interactions are scientifically or operationally important. The proposal must plan how factor combinations are assigned, replicated and interpreted rather than reporting only average effects.
Assessment move
Draw the design before collecting data. Label experimental units, treatment assignment, control, factor levels, response timing and independent replication. List nuisance influences and decide which can be controlled, blocked, randomised or measured. Write inclusion, exclusion and stopping rules before outcomes are known. Choose the comparison and uncertainty description that answer the research question.
Practise distinguishing repeated observations from independent units and explaining interactions in words. After data collection, preserve the design structure in analysis and reporting. State practical meaning alongside statistical uncertainty and connect every limitation to the conclusion it restricts. Build the diagram from the intended inference backwards.
Write the comparison or relationship in plain language, identify the unit to which a condition is independently assigned or observed, and distinguish that unit from subsamples and repeated readings. For every factor, define operational levels and justify their range. For the response, record units, timing, instrument, calibration and any aggregation that occurs before analysis.
List nuisance variables and choose a response deliberately: hold constant, block, randomise, balance, measure or acknowledge. Simulate the data structure with identifiers before collection and verify that the planned comparison respects dependence. Ask what would happen if one unit is lost, measurements drift over time, assignment is unbalanced or an interaction reverses the average pattern.
Predefine primary comparisons, inclusion and exclusion rules, stopping logic and the practical threshold that would change an engineering decision. Exploratory analyses can remain possible, but label them so they do not masquerade as preplanned confirmation. Conduct a pilot for procedure and measurement range rather than treating early favourable outcomes as proof.
Inspect whether the pilot reveals ceiling effects, excessive noise, access constraints or unsafe workload. After collection, compare the realised design with the plan and document deviations before interpreting results. Report estimated magnitude and uncertainty in natural units, then state whether the interval includes differences that matter in practice.
A weakly estimated near-zero result is not evidence of equivalence unless the study was designed for that question. Link each limitation to a specific claim: precision, causal attribution, measurement validity or transfer beyond tested conditions. Finish by writing two conclusions, one for the observed direction and one for a plausible reversal.
If only the preferred version sounds acceptable, the interpretation has become advocacy.
