STAT8003 Chap.2 Moments, Covariance and Random Walks
Moments, Covariance and Random Walks
Describe level, spread and dependence
This chapter builds the mean, variance, autocovariance and autocorrelation functions used throughout the course.
The mean can vary with time, variance is the autocovariance on its diagonal, and autocorrelation standardises covariance so dependence can be compared without measurement units.
Use a random walk to expose memory
A random walk shows how independent innovations can produce highly dependent levels and time-varying variance. Each later level contains every earlier shock plus new ones.
Its variance therefore grows with the number of accumulated innovations, while covariance between two times counts the shocks shared by both levels.
Consecutive late levels can be strongly correlated even though the next increment is unpredictable.
Align innovations before calculating
The main computational device is overlap: expand two process values, retain shocks with matching subscripts and use independence to remove unmatched covariance terms. Coefficient products determine the sign and size of each surviving contribution.
Only after both marginal variances are known should covariance be standardised. Units then provide a final audit: covariance retains squared units, while correlation is unit-free and bounded.
What this chapter covers
- 01
Mean function
- 02
Variance function
- 03
Autocovariance function
- 04
Autocorrelation function
- 05
Random-walk moments
- 06
Innovation overlap
Worked example · free
Calculate covariance from shared shocks
- 3Count innovations entering each variance.
- 3Count the shocks shared by the two levels.
- 2Standardise the covariance.
Key terms
- Mean Function
- The expected process value at each time index. It can vary with time in a general process and becomes one constant only after the relevant stationary condition has been established.
- Autocovariance Function
- The covariance between two values of the same process at specified times. It keeps squared measurement units and records the signed dependence created by shared innovations or other linked structure.
- Random Walk
- A process whose current level equals its previous level plus a new innovation. Its accumulated shocks make the variance grow with time and create strong dependence between nearby levels.
Moments, Covariance and Random Walks FAQ
Why are random-walk levels dependent if the shocks are independent?
Each later level contains all earlier shocks. Two levels therefore share an accumulated block of innovations, and that overlap creates covariance even though distinct innovations are mutually independent.
What units does autocorrelation have?
Autocorrelation is unit-free because covariance is divided by the product of the two relevant standard deviations. Autocovariance retains squared measurement units, so confusing the two can often be detected by checking dimensions.
Why does random-walk variance depend on time?
The value at time t contains t independent innovations. Their variances add, giving t times the innovation variance. Because this changes with t, the random-walk level violates weak stationarity even though its increments have stable behaviour.
What is the safest way to derive a lagged covariance?
Write both process values as innovation sums, align identical subscripts and multiply their coefficients. Independence removes every unmatched pair, while each shared shock contributes its variance times the coefficient product. Calculate the two marginal variances separately before converting covariance to correlation.
This route preserves signs, exposes symmetry and works for both cumulative processes and finite moving averages without relying on a memorised special-case numerator.
How can units catch a moment error?
A mean retains the original measurement unit, while variance and covariance carry its square. Correlation is unit-free. If a reported covariance has no units or a correlation still contains the innovation variance scale, the standardisation or denominator should be checked again.
Exam move
Practise drawing two aligned innovation lists. Use the overlap to calculate covariance, then compute each marginal variance before standardising. For a random walk, retain t and s until the final line so the square-root correlation formula follows from the correct time-specific variances. Check every answer for units and for the required correlation bounds.
Create three versions of each calculation: a full double-sum expansion, an overlap diagram and a short verbal explanation. The three should agree. In the full expansion, keep covariance symmetry visible and remove a term only after naming the independence that makes it zero. In the diagram, align equal innovation subscripts rather than equal positions on the page.
In the explanation, say how many shocks are shared and why later shocks contribute nothing. Then vary the innovation variance to verify that covariance changes but correlation does not. For the random walk, compare early and late consecutive levels and explain why their correlation approaches one even though the next innovation remains unpredictable.
For a finite moving average, change one coefficient sign and predict the direction of lag-one association before calculating it. Keep a unit checklist beside the work: mean has the original unit, variance and covariance have squared units, and correlation is unit-free. Use the bound from minus one to one as a final diagnostic rather than as a substitute for the denominator.
If a result fails the bound, revisit the marginal variances first. End revision by deriving the minimum-of-times form of random-walk covariance, since it states symmetry more clearly than a formula written only for ordered times.
Working through Moments, Covariance and Random Walks in STAT8003? Sia is AskSia’s AI Statistics tutor — ask any STAT8003 Moments, Covariance and Random Walks question and get a clear, step-by-step explanation grounded in how STAT8003 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.