The University of Manchester · FACULTY OF COMPUTER SCIENCE

SOST10031 Chap.3 Data Sources, Provenance and Data Quality

- one subject, every graph, every model, every mark
7 Chapters5-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 3 of 10 · SOST10031

Data Sources, Provenance and Data Quality

Data Sources, Provenance and Data Quality asks whether data suit the conclusion you want to draw, even when the code is correct. SOST10031 defines quality as fitness for purpose: evidence suits its intended use and its limitations are understood, so the same file may serve one question and fail another.

The chapter separates a data source, where information is first recorded, from the curated teaching extract students actually open, and follows a value's provenance through UNDP tables into the analysis. Five checks organise the week: meaning, provenance, timing, coverage and comparability.

Timing turns on three different dates, because a column suffix names a selected series rather than the year every observation was collected. Coverage and completeness are kept apart, and the chapter quantifies completeness for the overall index across the teaching file. Missing values recorded as NA are distinguished from zeros, and the effect of removing them in a calculation is spelled out.

Every limitation is then turned into a decision: identify, check, assess and respond.

In this chapter

What this chapter covers

  • 01

    Quality as fitness for purpose

  • 02

    Data sources compared with teaching datasets

  • 03

    Provenance from original record to conclusion

  • 04

    Publication, reference and observation dates

  • 05

    Coverage compared with completeness

  • 06

    Missing values recorded as NA, not zero

  • 07

    Comparability and turning a limitation into a decision

Worked example · free

Count available and missing rows before averaging an index

Q [4 marks]. An invented practice file has 12 rows. Nine hold index values that sum to 3.15 and three are NA. Report how many rows are available and missing, compute the average among available rows, show what replacing NA with 0 would give, and write the correct sentence. Marks used across the steps are AskSia's study aid, not an official university breakdown.
The horizontal line is calendar time; the three dots mark the dates attached to one value, so the 27.2% seat share sits in the 2023 series and a 2025 report tabThe horizontal line is calendar time; the three dots mark the dates attached to one value, so the 27.2% seat share sits in the 2023 series and a 2025 report table yet was observed in 2021.Observation yearReference seriesPublication year202120232025seat share 27.2%column suffix _20232025 report tableone value, three dates
The horizontal line is calendar time; the three dots mark the dates attached to one value, so the 27.2% seat share sits in the 2023 series and a 2025 report table yet was observed in 2021.
  • 1Available rows: 9; missing rows: 12 minus 9, which is 3.
  • 1Mean among available rows: 3.15 divided by 9 equals 0.35.
  • 1Replacing NA with 0 divides the same 3.15 by 12, giving 0.2625, an average lowered by values nobody measured.
  • 1Sentence: among the nine countries with an available value, the average index is 0.35; three are excluded because their value is missing.
Nine rows are available and three are missing; the defensible average is 0.35 among the nine, while zero-filling would wrongly report 0.2625.
Sia tip — Write the group size into the sentence that reports any average, so the reader knows which rows it describes.
Glossary

Key terms

Data provenance
The recorded route a value takes, from its first measurement to the number used in an analysis.
Fitness for purpose
The standard that evidence is of good quality when it suits its intended use and its limitations are understood.
Missing value
An entry unavailable in the dataset, shown in R as NA and never equivalent to a recorded zero.
FAQ

Data Sources, Provenance and Data Quality FAQ

Why can't I just replace NA with zero?

Zero is an observed result, while NA marks information that is unavailable. Filling gaps with zeros invents measurements and can distort averages, rankings, charts and relationships, usually by pulling summary figures towards values that were never recorded.

Does a column ending in _2023 mean every value is from 2023?

No. The suffix names the reference-year series chosen for the teaching file. Individual values may come from earlier years, shown by country footnotes in UNDP's tables, and maternal mortality carries an estimate from 2020.

Is a dataset with no visible errors high quality?

Not necessarily. The module judges quality by suitability for the intended use. A tidy file can still measure something other than your concept, omit important countries or mix reference periods, which would make it unfit for that particular question.

Study strategy

Assessment move

Pick one indicator from the data dictionary and build a short quality card for it. Write its definition and unit, trace its provenance from original records to the teaching file, note its reference series and any observation-year caveat, and record how many rows hold a value. Then add one comparability concern for a cross-country comparison you might make.

Practise the NA habit with invented sets of six to twelve values: count missing entries, compute the mean without them, compute the zero-filled mean, and explain the difference in one sentence. Rewrite a careless claim about a footnoted value so that it names the publication, measure, value and timing note. Last of all, convert one limitation into a decision using identify, check, assess and respond.

These cards feed directly into the check and limitation the formative plan asks for.

Working through Data Sources, Provenance and Data Quality in SOST10031? Sia is AskSia’s AI Computer Science tutor — ask any SOST10031 Data Sources, Provenance and Data Quality question and get a clear, step-by-step explanation grounded in how SOST10031 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 41 of your subjects at The University of Manchester, plus 1,000+ more Bibles across the AskSia library.
Sia - your SOST10031 tutor, unlimited, worked the way the exam marks it
The full 5-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works