MAIE6000C Chap.4 Data Persistence and Schema Design
Data Persistence and Schema Design
Data design is not a storage choice
The module states the claim directly and it is worth taking literally: the shape of the data governs how interfaces look, how legible the workflow is, whether anything can be audited, what the background step can do, and how dependable the result is, and a poor schema tends to generate engineering trouble at every later stage.
That is a prediction about where your Week 9 and Week 10 problems will come from, not a remark about tidiness.
Two questions, asked in the right order
What records must exist, and how do they change over time. The recurring candidates the course names are a submitted item, a processing job, an AI result, a review decision and, where the workflow has one, an alert or status record.
The common state sequence runs from submitted through queued and processing to completed, reviewed or error. Answering the second question first produces a list of nouns with no states, which is the schema that has to be rewritten once the worker path arrives.
The four state questions
What exists immediately after submission. What exists after background processing. What exists after human review.
And which states a reader can see once something has broken. The fourth is the one teams skip and it is the most expensive omission in the course, because a lifecycle with no error state cannot distinguish a job that died from one that is still running.
From outside the system those two look identical, which means neither a user nor a reviewer can tell them apart.
What belongs in the store and what belongs in a log
Part of schema thinking is being disciplined about what belongs in the database against what belongs in logs or derived views. The working test is whether anybody will need to query the fact later.
A month after the semester ends somebody might reasonably ask how often the supervisor disagreed with the proposal; nobody will ask which container handled the third retry.
The first is a stored fact with a column and the second is a log line, and confusing them produces either an unqueryable evaluation or a log file carrying content it should not hold.
Three facts, not one
What the system received, what the system proposed and what a person finally decided are three separate facts with three different authors.
Storing only the third gives a working product and destroys the evaluation, because the proposal a human corrected is the most valuable single record your system will produce: it is the only direct measurement of whether the AI step helps.
Overwriting it costs nothing on the day and makes the final technical brief's evidence-based evaluation unavailable in Week 13.
Migrations are a collaboration mechanism
The module lists migrations and schema evolution among its topics, and the starter carries a migrations folder whose initial revision runs at startup. The reason the course insists on them is social rather than technical.
A schema change applied by hand works on one machine and nowhere else, so the next teammate to pull gets a system that starts and then fails on the first query, with an error pointing at the application rather than at the missing column. That is a reproducibility failure wearing the costume of a bug, and it costs the team a day each time.
What this chapter covers
- 01
Why the module claims a weak schema damages everything downstream
- 02
The two questions that produce entities, in the order that works
- 03
Four candidate records and what each one is for
- 04
The submitted to reviewed lifecycle, and the error state teams omit
- 05
The sorting rule between the store and the logs
- 06
Received, proposed and decided as three separate facts
- 07
What overwriting a proposal costs the final technical brief
- 08
Migrations as a collaboration mechanism rather than paperwork
A schema that cannot answer its own questions
- 4Walk the four state questions against the proposed table.
- 3Name the facts that are being conflated.
- 3Give the repaired record set and say what each record answers.
Key terms
- Candidate Record
- A persistent entity named during decomposition with its purpose and likely attributes, before any table, field type or migration is written.
- State Transition
- A change from one lifecycle status to another, which is what brings most records into existence and what a demonstration has to be able to show.
- Error State
- A lifecycle status distinguishing work that failed from work still in progress, without which a dead job and a slow job look identical from outside.
- Auditability
- The property of being able to reconstruct what the system received, what it proposed and what a person decided, from stored facts rather than from memory.
- Migration
- A recorded, repeatable schema change applied by the system rather than by hand, which is what lets a teammate reach the same database shape from a clean start.
- Override Rate
- How often a human changed a proposal, computable only when the proposal and the decision are stored as separate facts.
- Derived View
- A result computed from stored facts rather than stored itself, which is usually the right home for a summary that can be recomputed on demand.
Data Persistence and Schema Design FAQ
What records does a project like this normally need?
Four recur across almost every workflow the course teaches: a submitted item holding what arrived, a processing job tracking work that should happen with its own status and timestamps, an AI result holding the category, score or explanation that was produced, and a review decision holding what a person concluded. Some workflows add an alert or status record.
The important structural point is that they are separate records rather than columns on one table, because each answers a different question and each is written by a different actor at a different moment.
Why should I not just update the row with the final answer?
Because it destroys the only direct measurement of whether the AI step is worth having. The proposal a human corrected is the most valuable record a system of this shape produces: it lets you compute how often the workflow disagreed with the model, which cases it disagreed about and whether the pattern changed over the semester.
A schema that overwrites the proposal with the decision still runs the workflow perfectly, but the final technical brief asks for an evidence-based evaluation against stated goals, and that evaluation is unavailable. The cost of keeping both is two columns.
Do I really need migrations for a semester project?
Yes, and the reason is collaboration rather than rigour. A schema change made by hand exists on the machine where it was made, so a teammate who pulls your code gets a service that starts and then fails on its first query with an error pointing at the application rather than at the missing column. That costs a day each time it happens.
It also breaks the clean-clone test the deployment week requires, because the reviewer's database begins empty. The starter already applies its initial revision at startup, so the pattern is available from the first day.
Assessment move
Take your current schema and answer the four state questions out loud against it, writing one sentence each. The question that produces a vague answer is where your Week 10 debugging time will go. Doing this before the Week 4 proposal costs half an hour and is most of what the proposal's data model criterion is reading for.
Working through Data Persistence and Schema Design in MAIE6000C? Sia is AskSia’s AI Computer Science tutor — ask any MAIE6000C Data Persistence and Schema Design question and get a clear, step-by-step explanation grounded in how MAIE6000C is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
