The Hong Kong Polytechnic University · FACULTY OF STATISTICS

RS517 Chap.8 Correlation and Regression

- one subject, every graph, every model, every mark
6 Chapters3-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 8 of 10 · RS517

Correlation and Regression

Association and prediction

Pearson correlation summarises the strength and direction of a linear relation and is symmetric in its two variables.

Regression is directional: it models an expected outcome from predictors and gives coefficients in outcome units.

Coefficients are conditional

A multiple-regression coefficient describes an expected outcome difference for a one-unit predictor change while included predictors are held constant.

That conditioning answers a useful question only when variable roles and model form match the scientific aim.

Diagnostics protect the boundary

Scatterplots reveal nonlinearity, restricted range and influential observations. Residual plots assess model shape and spread, while independence follows from study design.

Predictions outside the observed range are extrapolations even when software returns a number.

Separate association, prediction and causation

Correlation summarises the direction and strength of a linear relationship, not agreement, causation or the whole shape of the data. A scatterplot can reveal curvature, clusters, range restriction and influential observations that the coefficient conceals.

Regression adds an explicit outcome and can adjust for included predictors, but each coefficient remains conditional on the model, coding and selected sample. Causal interpretation requires a design and adjustment set justified independently of model fit.

Read coefficients on their actual scales

In simple linear regression, the slope is the expected outcome change for one predictor unit.

With several predictors, it is the conditional change holding other included terms constant. Categorical predictors require a named reference group. Interactions make a coefficient depend on another variable's level, while transformations change the unit of interpretation.

The intercept may be mathematically necessary yet scientifically meaningless when zero lies outside the observed range; centring can create a more useful reference without changing fitted values.

Diagnose and validate the model

Residuals should be examined for nonlinearity, unequal spread, dependence and unusual influence.

A high coefficient of determination does not establish accurate individual predictions, calibration in a new population or lack of bias. Internal validation estimates optimism through resampling or held-out data; external validation tests transport to another setting or time.

Refitting all coefficients in the new cohort is new model development rather than validation of the original prediction rule.

Revision drill

Draw one nonlinear scatterplot with strong correlation and one linear plot with wide prediction error. Explain why the same coefficient can support different decisions.

Interpret a slope with units, reference category and interaction, then calculate one prediction and mark whether it is interpolation or extrapolation. End with a validation plan that separates discrimination, calibration and error on the intended target scale.

In this chapter

What this chapter covers

  • 01

    Linear association and scatterplots

  • 02

    Pearson correlation and restricted range

  • 03

    Regression equations and predictions

  • 04

    Conditional coefficient interpretation

  • 05

    Residuals, variance and influence

  • 06

    Extrapolation and model validation

Worked example · free

Compute a conditional prediction

Q [8 marks]. Practice note: the marks used here are not part of the university's published assessment scheme. Use predicted distance = 520 - 2.5×age + 6×strength for age 60 and strength 20, then interpret coefficients.
The fitted line gives predicted outcomes while each vertical residual records the observed value minus its prediction at that predictor level.The fitted line gives predicted outcomes while each vertical residual records the observed value minus its prediction at that predictor level.vertical gaps are residuals
The fitted line gives predicted outcomes while each vertical residual records the observed value minus its prediction at that predictor level.
  • 3Compute the prediction correctly.
  • 5Interpret coefficients conditionally with units.
The prediction is 490 metres. Holding strength constant, an additional year of age is associated with 2.5 metres lower expected distance. Holding age constant, one strength unit is associated with 6 metres higher expected distance. These adjusted associations are not automatically causal.
Sia tip — Say holding the other included predictors constant before giving a multiple-regression coefficient meaning.
Glossary

Key terms

Correlation
A unitless summary of the direction and strength of linear association.
Regression Coefficient
The expected outcome difference for a one-unit predictor change under the fitted model.
Residual
The observed outcome minus the outcome predicted by the fitted model.
Influence
The degree to which an observation changes fitted coefficients or predictions.
Extrapolation
Prediction beyond the predictor range represented in the fitted data.
FAQ

Correlation and Regression FAQ

Does strong correlation mean two measures agree?

No. Two measures can rank people almost perfectly while one is consistently higher. Agreement requires examination of paired differences, systematic bias and spread across the measurement range. Correlation alone does not establish that the measures can be used interchangeably.

How should a regression coefficient be written?

Name the outcome units, the predictor unit change, the direction and the conditioning variables. Add the interval and target population. Avoid causal verbs unless the design and assumptions justify intervention interpretation rather than adjusted association.

Why is extrapolation risky?

The fitted relationship is supported only over the observed predictor range. A linear trend may bend or reverse outside it, and the target population may differ. Software can calculate an external prediction, but arithmetic availability is not empirical validation.

Study strategy

Exam move

For every regression output, write the equation, calculate one prediction and translate each coefficient into a conditional sentence with units. Inspect a scatterplot and residual plot before the table. Mark the observed predictor range so that extrapolation becomes visually obvious.

Working through Correlation and Regression in RS517? Sia is AskSia’s AI Statistics tutor — ask any RS517 Correlation and Regression question and get a clear, step-by-step explanation grounded in how RS517 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 24 of your subjects at The Hong Kong Polytechnic University, plus 1,000+ more Bibles across the AskSia library.
Sia - your RS517 tutor, unlimited, worked the way the exam marks it
The full 3-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works