DSC6008 Chap.8 Power, Sample Size and Allocation
Power, Sample Size and Allocation
Plan for an effect that matters
Power is the probability of rejecting the null under a specified alternative. It depends on the smallest effect worth detecting, outcome variability, significance level, sample size and allocation.
A sample-size calculation therefore documents scientific and operational choices rather than producing a universal number.
Name the alternative before reporting power
A test has different power against different true effects. Small effects are harder to distinguish from noise, while large effects move the statistic farther into the rejection region.
Reporting 80% power without the effect, variance, alpha and sidedness leaves the statement incomplete. The target effect should come from the practical decision, not from a value chosen to make recruitment convenient.
Keep effect and variability on compatible scales
A raw mean difference and standard deviation must use the same outcome units.
Their ratio gives a standardized effect, which is helpful for formulas but can hide practical meaning. Report both scales.
An effect equal to half a standard deviation may be important or trivial depending on whether the original outcome is minutes, kilograms or a subjective score.
Count independent units
For a balanced two-group comparison, greater noise increases the required group size, while a larger target effect reduces it.
Repeated visits and instrument readings do not become new experimental units simply because they create more records. Simulation repetitions improve the precision of a power estimate, but only independently assigned physical or observational units change the study's information.
Compare allocation through information
Balance is usually most efficient when per-unit costs and variances are similar.
Unequal allocation can be sensible when one condition is expensive, scarce or ethically limited, or when variances differ. The decision should compare the resulting standard error or information rather than group counts alone.
A three-to-one split may require a larger total to recover the precision of equal groups.
Round and inflate in the correct order
A fractional required sample is rounded upward because rounding down misses the target under the planning assumptions. If attrition or unusable runs are expected, first calculate the analysable independent units and then inflate recruitment using a defended retention rate.
Keep recruited, assigned and analysed counts distinct so losses do not disappear inside one headline number.
Stress-test uncertain inputs
Variance estimates from small pilots and retention assumptions are uncertain. Recalculate the sample under plausible high and low values while holding the scientific effect fixed. A plan that works only under the most optimistic row is fragile.
Sensitivity analysis identifies whether better measurement, a pilot or a narrower question would improve feasibility more than simply changing the target power.
Protect any mid-course update
An internal pilot can sometimes update a nuisance variance without examining treatment differences.
The update rule, timing and masking should be specified in advance because unplanned looks at the effect can change error rates. The final plan records every input, rounding choice, allocation ratio and revision so another analyst can reproduce the target.
What this chapter covers
- 01
Statistical power
- 02
Target effects
- 03
Variance planning
- 04
Significance levels
- 05
Balanced allocation
- 06
Sample-size sensitivity
Worked example · free
Round a two-group requirement in the safe direction
- 1Add the critical values to obtain 2.80.
- 3Compute 2[(2.80)(6)/5]^2 = 22.58.
- 2Round upward to 23 per group and 46 overall.
Key terms
- Statistical Power
- The probability of rejecting the null under a specified alternative effect.
- Target Effect
- The smallest scientifically meaningful difference used for design planning.
- Balanced Allocation
- Assignment of equal numbers of independent units to comparison groups.
Power, Sample Size and Allocation FAQ
Why must a power calculation specify an effect?
Power is not a property of a test alone. The chance of detection varies across alternatives, so planning needs a particular difference considered important enough to find. Choosing an unrealistically large effect can make the proposed sample too small for the scientific question.
What happens when variability is underestimated?
The experiment is likely to have less power than planned because the treatment contrast is noisier relative to its standard error. Sensitivity calculations across plausible standard deviations make this risk visible and can guide a conservative design or an internal pilot strategy.
Why is equal allocation often efficient?
With equal per-unit costs and similar variances, a fixed total sample gives the smallest standard error for a two-group mean difference when the groups are balanced. Different costs, variances, ethical constraints or limited treatment availability can justify another allocation, but the tradeoff should be calculated.
Assessment move
Memorise the direction of each input before memorising a formula: required sample size rises with noise and stricter evidence demands, but falls when the target effect is larger. Recalculate every plan under at least two plausible variance assumptions and verify that the counted observations are independently assigned units.
Working through Power, Sample Size and Allocation in DSC6008? Sia is AskSia’s AI Statistics tutor — ask any DSC6008 Power, Sample Size and Allocation question and get a clear, step-by-step explanation grounded in how DSC6008 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
