MAIE6000C Chap.6 Background Jobs and Worker Based Processing
Background Jobs and Worker Based Processing
Background execution is an architectural choice
The module says so directly: background execution is not just a performance optimisation, it is frequently the right structural decision for work that takes a while, breaks often or deserves a second attempt, and that ought to stay visible and recoverable. Those adjectives are the specification.
Work that can fail for reasons the caller cannot fix belongs outside the request cycle even when it is fast, because inside it the failure becomes the user's problem and outside it becomes a retry.
Give the work a record before you give it a worker
A job is a stored row representing work that should happen, with a state of its own that is not the state of the item it acts on.
That record is what makes waiting work visible, which a background thread does not.
It is also what lets a reviewer see the difference between a system that is busy and one that is stuck, and it supplies the operational signal the midterm demonstration requires alongside the asynchronous step itself.
The lifecycle, and the two edges teams forget
The observable sequence in the starter run is concrete: the case is created, the job is claimed, the AI service is called, the triage completes, the job completes.
Two edges are missing from most first designs. The retry edge returns a transiently failed job to the queue while attempts remain.
The expiry edge returns a job that has been processing for too long, because a worker that died between claiming and writing leaves a row nobody will ever pick up again, since it no longer looks queued.
Claiming has to be a state change
The design most teams write first reads the oldest queued row and then updates it to processing. Between those two operations another worker can read the same row.
On one laptop with one worker this never happens and the design looks correct for ten weeks. Making the claim itself the transition means only one caller can move a given row out of queued and the loser simply finds nothing to do.
This is the most common concurrency defect in a semester project and it is invisible until the day two workers run together.
Retries need a ceiling, a record and a safe write
An unlimited retry turns a permanent failure into an infinite one, and a retry with no recorded attempt count cannot be distinguished from a job that has only just started.
Retrying is also only a repair if repeating the work is harmless: a worker that appends rather than sets leaves three results after three attempts, and one that notifies as part of the same step sends three notifications.
The rule from the interface chapter applies inside the worker, and the failure rule the course states for the lab exercise is explicit: record the job as failed and put it where a person will see it.
Two numbers make the design legible
How many jobs are waiting, and how old the oldest waiting job is.
Together they distinguish a busy system from a stuck one, which neither number does alone, and they cost one query each. They are also exactly the sort of operational signal the midterm asks for, and they turn a backlog during a demonstration from a surprise into something you can point at and explain.
What this chapter covers
- 01
What belongs outside the request cycle, and why duration is not the only test
- 02
The job record as the thing that makes waiting work visible
- 03
The lifecycle sequence the starter emits, event by event
- 04
The retry edge and the claim-expiry edge
- 05
Why claiming must be the state change rather than precede it
- 06
Attempt counts, failure reasons and safe repeated writes
- 07
Queue depth and oldest-queued age as the two legibility numbers
- 08
The five-line worker specification, written before any code
Three symptoms, one lifecycle to repair
- 3Explain the duplicate processing and name its repair.
- 3Explain the permanently stuck row and name its repair.
- 3Say why the pile-up is not a defect, and what it needs.
Key terms
- Job Claim
- The act of a worker taking ownership of queued work, which has to be the state transition itself so that only one worker can win it.
- Claim Expiry
- A rule returning work held in processing beyond a stated period to the queue, which is what prevents a dead worker from removing work silently.
- Attempt Count
- The number of times a job has been tried, stored so that a retry ceiling can exist and a repeatedly failing job can be distinguished from a new one.
- Queue Depth
- How many jobs are waiting, which together with the age of the oldest waiting job separates a busy system from a stuck one.
- Terminal State
- A job status from which no further transition occurs, such as completed or failed, which is what makes progress measurable.
- Safe Repeated Write
- A write that produces the same stored result however many times it runs, which is what makes a retry a repair rather than a multiplier.
- Worker Process
- A separate running process that takes work from the queue, performs it and records the outcome, independent of any request in flight.
Background Jobs and Worker Based Processing FAQ
What work should leave the request cycle?
Anything long-running, failure-prone or retryable, which is the module's own phrasing. Duration is the obvious test but not the only one. A call that takes two hundred milliseconds and fails for reasons the caller cannot fix still belongs outside, because inside the request the failure becomes the user's problem while outside it becomes a retry with a recorded reason.
There is also a structural answer: the course lists a worker or asynchronous path among the ten elements every passing project must contain, so a system with none has not met the contract however fast it is.
What happens if a worker dies while processing a job?
In a design with no claim expiry, the row stays in processing forever and nothing will pick it up again, because it no longer looks queued. From outside, that item is indistinguishable from one still being worked on, so nobody reports a problem and the work silently disappears.
The repair is a claim timestamp and a rule that returns a row held beyond a stated period to the queue, incrementing the attempt count so the return is bounded. It is worth testing deliberately by stopping the worker mid-job, because it is a failure nobody encounters until a demonstration.
Is catching every exception in the worker good error handling?
No, it is error hiding with a paper trail. If the worker catches, logs and moves on, the item keeps whatever state it had before, so from outside it looks like an item nobody has reached yet. The log line exists, but nobody reads logs to answer a question about one item.
The repair is to write the failure to the job record with the stage and the reason, keep the log line for diagnosis, and surface the failed state through the read endpoint. The course states the rule for its own lab exercise: record the job as failed and put it where a person will see it.
Assessment move
Fill the five-line worker specification for your own project before writing any worker code: name the background task, say what sets it going, what it consumes, what it records, and what happens when it breaks. Teams who can complete all five have an asynchronous design; teams who can complete only the first three have a function they intend to call later, and the difference shows up at the midterm.
Working through Background Jobs and Worker Based Processing in MAIE6000C? Sia is AskSia’s AI Computer Science tutor — ask any MAIE6000C Background Jobs and Worker Based Processing question and get a clear, step-by-step explanation grounded in how MAIE6000C is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
