BIOS6902 Chap.1 Models, Estimators and the Bias-Variance Trade-off
Models, Estimators and the Bias-Variance Trade-off
What the first lecture is actually setting up
Inference, as this course defines it, is what you do when observations are used to work out something about whatever produced them. The definition sounds mild, and it fixes the shape of every question in the course. There is a distribution you cannot see.
There is one number you want from it: a prevalence, a mean blood pressure, an event rate, a difference between two treatments. You get one sample. From that sample you construct a number that is supposed to stand in for the one you wanted, and the entire remaining syllabus is an account of how far apart those two numbers can be.
Two ways of describing what you cannot see run through the whole course.
The first commits the distribution to a family labelled by a small number of quantities: Bernoulli for a yes-or-no outcome, Normal for a biomarker reading, Poisson for how many events occurred, Gamma for how long survival lasted. The second commits to far less, perhaps only continuity, or finite spread. The first route buys precision with an assumption.
The second declines the assumption and settles the bill in width.
Functionals, and why naming one first saves marks
The target is written as a functional: any real-valued quantity the distribution determines. The mean, the variance, the median and a regression function are all functionals.
Deciding which one the question asks for, before reaching for any formula, is the single highest-return habit in this subject.
A question about a median does not become a question about a mean merely because a mean is easier to put an interval around, and an answer that estimates the wrong functional is wrong no matter how carefully the arithmetic is done.
Once you have an estimator, three quantities describe its performance. Bias is how far it sits from the target on average across repeated samples.
Variance is how much it moves from sample to sample.
Mean squared error combines the two as the squared bias plus the variance, and it is the budget that actually matters: you are permitted to be off-centre, and permitted to be jumpy, but the sum of squared off-centring and jumpiness is what you pay.
The pair of estimators this lecture keeps returning to
The sample mean is unbiased at every sample size and its variance falls in proportion to the sample size, so its mean squared error goes to zero and it is consistent.
The largest observation from a uniform distribution on an interval starting at zero can never exceed the upper endpoint, so it is guaranteed to underestimate; its bias is not zero at any finite sample size, yet the bias shrinks and the estimator is consistent all the same.
Between them the two cases separate a statement about one sample size from a statement about the limit, and that separation is examined in almost every offering of a course like this one.
The chapter closes on confidence intervals, where the most common error in the subject lives. A ninety-five per cent interval is a random interval that captures the fixed parameter in ninety-five per cent of repeated studies.
Once the data are in, the interval is a pair of fixed numbers and the parameter is a fixed number, so there is nothing random left to carry a probability. What is ninety-five per cent is the procedure, and saying so precisely is worth real marks.
What this chapter covers
- 01
Parametric against nonparametric models, with the four biostatistical families
- 02
Statistical functionals: mean, variance, median, regression function
- 03
Bias, variance and mean squared error as one budget
- 04
Consistency, and why a shrinking bias is compatible with it
- 05
Hoeffding and normal-based confidence intervals for a prevalence
- 06
The correct and incorrect ways to read a ninety-five per cent interval
Worked example · free
Following one estimator all the way to its mean squared error
- 1State the target as a functional and the estimator as a function of the data.
- 2Use linearity of expectation to obtain the bias, with no approximation.
- 2Use independence to remove the covariance terms and obtain the variance.
- 1Combine the two into the mean squared error and read off consistency.
Key terms
- Statistical Functional
- Any real-valued quantity determined by the whole distribution, such as a mean, a variance, a median or a regression function. It is the target an estimator aims at.
- Bias
- The difference between the average value of an estimator across repeated samples and the quantity it is estimating. Zero bias is a statement about one sample size.
- Mean Squared Error
- The expected squared distance between an estimator and its target, equal to the squared bias plus the variance. It is the budget that decides which estimator wins.
- Consistency
- The property that an estimator converges to its target as the sample grows. A mean squared error tending to zero is sufficient for it.
- Coverage Probability
- The long-run proportion of repeated studies in which an interval procedure captures the fixed parameter. It is a property of the procedure, not of one computed interval.
- Parametric Model
- A description that commits the distribution to a family labelled by a small number of quantities, trading an assumption for a narrower answer.
Models, Estimators and the Bias-Variance Trade-off FAQ
Should an estimator centred on its target always be preferred?
No, and the question is raised in the opening lecture precisely because the intuitive answer is wrong. What matters is mean squared error, which adds the squared bias to the variance. An estimator that sits slightly off-centre but barely moves from sample to sample can easily beat an unbiased estimator that swings widely.
The largest observation from a uniform distribution is the standard illustration: it is biased at every sample size and still converges to the truth.
Why can I not say there is a ninety-five per cent chance the parameter is in my interval?
Because after the data arrive nothing in that sentence is random. The interval is two fixed numbers and the parameter is one fixed number, so the parameter either is or is not inside and no probability attaches to the pair. The ninety-five per cent describes the procedure across repeated samples: build an interval this way many times and that proportion of them will contain the parameter.
What does a distribution-free interval buy compared with the usual one?
It buys a guarantee that holds at every sample size and for every true parameter value, without appealing to a large-sample approximation. The cost is width: because it never reads the spread off the data, it cannot narrow when the data turn out to be informative. Choose it when the sample is small or the consequence of a false claim is serious.
Exam move
Before touching a formula, finish this sentence: the number I want is the ____ of the distribution that generated these data. Then decide the estimator, then the standard error, then the assumption carrying the answer. Most avoidable errors in this course are a failure to finish that first sentence honestly.
Working through Models, Estimators and the Bias-Variance Trade-off in BIOS6902? Sia is AskSia’s AI Statistics tutor — ask any BIOS6902 Models, Estimators and the Bias-Variance Trade-off question and get a clear, step-by-step explanation grounded in how BIOS6902 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
