CMCE10002 Chap.12 Predictive Analytics, Stored Data and APIs
Predictive Analytics, Stored Data and APIs
Three topics, one shared job
The last three teaching weeks name using data to form and evaluate forecasts, storing and using existing data, and collecting structured and unstructured data from the interfaces services publish. The matching workshops apply them to workforce planning, auction records and economic series.
Together they cover the cases where the data is not already in front of you.
A forecast is judged on rows it has never seen
A descriptive summary is judged by whether it computes what it claims. A forecast is judged by whether it is right about something unseen, so the evaluation cannot reuse the rows the rule was built from.
Hold part of the data back, fit on the rest, score on the part held back, and where the data runs in time hold back the later rows rather than a random selection.
Live data is dated data
The same request issued next month returns different rows, so an analysis built on a live service is reproducible only if the request and its date are recorded and the response is saved beside the code.
A score means nothing on its own
Whether an average error is good depends entirely on what the alternative would have produced, and the cheapest alternative is always available: predict each period to equal the one before it.
Reporting the two side by side turns an unreadable number into a decision about whether the rule is worth maintaining. A rule that fails to beat that alternative is a finding rather than a failed exercise, and reporting it plainly is stronger than re-fitting until a better number appears.
What this chapter covers
- 01
Forming a rule and evaluating it as two separate jobs
- 02
Holding back later rows when the data runs in time
- 03
Comparing every forecast against a trivial alternative
- 04
Describing the slice taken from a shared store precisely
- 05
Recording the request and saving the response so live data stays reproducible
Evaluating a staffing forecast honestly
- 1Ask which rows the rule was scored on, and say what the answer implies.
- 1Give the correct evaluation and the reason for the split direction.
- 1Name the comparison that makes the score meaningful.
- 1State the three figures the report must carry.
Key terms
- Held-back data
- Rows deliberately excluded while a rule is built, kept aside to score it honestly.
- Baseline comparison
- A deliberately trivial alternative used to judge whether a more elaborate rule is worth its cost.
- Overfitting
- Matching the accidental detail of the rows a rule was built from, which does not repeat on new data.
- Structured data
- Data that already arrives in a rectangular shape of rows and columns.
- Unstructured data
- Data that arrives nested or text-heavy and needs reshaping before it becomes a table.
Predictive Analytics, Stored Data and APIs FAQ
Why can a forecast not be scored on the data it was built from?
Because any sufficiently flexible rule can be made to trace rows it has already seen. Scoring on those rows measures how closely it reproduces the past, not whether it will be right about a period nobody has observed yet.
How should data be split when it runs in time?
Hold back the later periods rather than a random selection. In production the rule will be predicting forwards, so a random split lets it learn from periods that come after the ones it is being asked to predict.
What makes an analysis built on a live data service reproducible?
Recording the request as code, noting the date it was issued, and saving the response as a file beside the analysis. The code shows how the data was obtained and the saved file fixes what was actually reported.
Exam move
These weeks carry no assessment of their own and still appear in the final paper. Give them one deliberate pass in revision, focused on the three-part forecast report and the reproducibility of a live data request.
Working through Predictive Analytics, Stored Data and APIs in CMCE10002? Sia is AskSia’s AI Statistics tutor — ask any CMCE10002 Predictive Analytics, Stored Data and APIs question and get a clear, step-by-step explanation grounded in how CMCE10002 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
