ECMT1010 · Introduction To Economic Statistics
Covariance & the Theory of Estimators
Week 13 is the lecture-only capstone that explains why the earlier methods work. It defines covariance cov(X, Y) = σ_XY and the population correlation ρ, gives the covariance and variance rules, and draws the key distinction between an estimator X̄ (a random variable before sampling) and an estimate x̄ (a realised number after sampling) — leading to E(X̄) = μ and Var(X̄) = σ²/n, the algebraic root of the CLT's standard error. It is examined as short-answer 'apply the variance rules' and conceptual questions, and it bridges to ECMT1020.
What this chapter covers
- 011. Population covariance cov(X, Y) = σ_XY = E[(X − μ_X)(Y − μ_Y)] and its sign
- 022. Population correlation ρ_XY = σ_XY/(σ_X·σ_Y), the unit-free version of covariance
- 033. Independence ⇒ cov = 0 (and why the converse need not hold)
- 044. Covariance rules: cov(X, V + W) = cov(X, V) + cov(X, W); cov(X, bZ) = b·cov(X, Z); cov(X, b) = 0
- 055. Variance rules: Var(V + W) = Var(V) + Var(W) + 2cov(V, W); Var(bZ) = b²Var(Z); Var(b) = 0
- 066. Estimator vs estimate: X̄ as a random variable (before) vs x̄ as a number (after)
- 077. iid sampling and the key results E(X̄) = μ (unbiased) and Var(X̄) = σ²/n
- 088. SD(X̄) = σ/√n as the algebraic root of the CLT's SE, and the bridge to ECMT1020
Variance of a sum and the sampling distribution of the estimator X̄
- 2 marks(a) Independence means cov(A, B) = 0, so the cross term drops out: Var(A + B) = Var(A) + Var(B) = 25 + 4 = 29.
- 2 marksFor 3A − B: use Var(bZ) = b²Var(Z). Var(3A) = 3²·25 = 225 and Var(−B) = (−1)²·4 = 4; independence ⇒ no covariance term, so Var(3A − B) = 225 + 4 = 229.
- 1 mark(b) The estimator X̄ is unbiased, so E(X̄) = μ = 40.
- 1 markIts standard deviation is SD(X̄) = σ/√n = 16/√64 = 16/8 = 2.
Key terms
- Covariance
- cov(X, Y) = σ_XY = E[(X − μ_X)(Y − μ_Y)], a measure of how two variables move together. It is positive when they tend to move in the same direction, negative when in opposite directions, and depends on the units of X and Y.
- Population correlation (ρ)
- ρ_XY = σ_XY/(σ_X·σ_Y), the unit-free standardisation of covariance with −1 ≤ ρ ≤ 1. It is the population analogue of the sample correlation r from the descriptive-statistics chapter.
- Variance rules
- Var(V + W) = Var(V) + Var(W) + 2cov(V, W); Var(bZ) = b²Var(Z); Var(b) = 0; Var(V + b) = Var(V). The cross term vanishes when V and W are independent, leaving Var(V + W) = Var(V) + Var(W).
- Independence ⇒ cov = 0
- If X and Y are independent then their covariance (and correlation) is zero. The converse is not guaranteed — a zero covariance means no linear association, but a non-linear relationship can still exist.
- Estimator vs estimate
- Before you sample, each observation is a random variable and X̄ = (1/n)ΣXᵢ is an estimator (itself random, with a distribution); after you sample, the realised number x̄ is the estimate. The distinction is the heart of the Week-13 theory.
- Sampling results for X̄
- For an iid sample, E(X̄) = μ (the sample mean is unbiased) and Var(X̄) = σ²/n, so SD(X̄) = σ/√n. This last result is the algebraic root of the standard error used throughout the unit.
Covariance & the Theory of Estimators FAQ
What is the difference between an estimator and an estimate?
An estimator is a rule or formula applied to a sample — for the mean it is X̄ = (1/n)ΣXᵢ — and before you collect data it is a random variable with its own distribution, mean and variance. An estimate is the single number you get once you plug in your actual sample, written with a lowercase x̄. The estimator is the random recipe; the estimate is the realised dish. This distinction is what lets us talk about the estimator being unbiased (E(X̄) = μ) and having variance σ²/n even before any data are collected.
Where does the standard error σ/√n actually come from?
It comes straight from the variance rules. For an iid sample, X̄ = (1/n)(X₁ + … + Xₙ); applying Var(bZ) = b²Var(Z) and adding independent variances gives Var(X̄) = (1/n²)·n·σ² = σ²/n, so SD(X̄) = σ/√n. That is exactly the standard error you used for confidence intervals and tests of a mean — Week 13 derives the formula you had been applying on faith, which is why it is the algebraic root of the CLT's SE.
When do I include the 2·cov term in the variance of a sum?
Whenever the two variables are NOT independent. The general rule is Var(V + W) = Var(V) + Var(W) + 2cov(V, W). If V and W are independent their covariance is 0 and the cross term disappears, leaving Var(V + W) = Var(V) + Var(W). So check independence first: if the problem states the variables are independent you can drop the cross term; otherwise you must compute and include it.
Does zero covariance mean the variables are independent?
Not necessarily. Independence always implies zero covariance, but the reverse is not guaranteed. Covariance only captures LINEAR association, so two variables can have a perfect non-linear relationship (for example Y = X² over a symmetric range) and still have covariance zero. So 'cov = 0' tells you there is no linear relationship, not that the variables are unrelated. Independence is the stronger condition.
Exam move
This capstone is examined more on understanding and rule-application than on heavy arithmetic, so prioritise the concepts. Be able to state crisply the estimator-versus-estimate distinction (random recipe X̄ vs realised number x̄) and the two headline results E(X̄) = μ and Var(X̄) = σ²/n, and be ready to derive SD(X̄) = σ/√n from the variance rules — examiners love to ask 'where does the standard error come from?'. Memorise the covariance and variance rules as a small toolkit and practise applying them to linear combinations like Var(aX + bY), always asking first whether the variables are independent so you know whether to include the 2·cov cross term. Keep clear that independence implies zero covariance but not the reverse. Treat the week as the unit's 'why it works' chapter and the on-ramp to ECMT1020.
