The Hong Kong University of Science and Technology · FACULTY OF ARTS & HUMANITIES

MAIE5103 Chap.13 Automation, Work and Productivity Evidence

- one subject, every graph, every model, every mark
8 Chapters3-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 13 of 14 · MAIE5103

Automation, Work and Productivity Evidence

Capability, adoption and job loss are three different things

The labour section of the automation lecture starts with a 2014 video argument that machines will replace human workers as they replaced horses, and treats it as a provocation to examine rather than a forecast.

Its central warning is that a machine being able to do a task does not mean it will be adopted, and adoption does not mean a job disappears: demand, costs and institutional choices sit in between.

An ILO working paper of May 2025 makes the same distinction with numbers, estimating that about one in four workers worldwide has some exposure to generative AI and 3.3% of employment is in the highest-exposure group, while judging transformation more likely than replacement.

Reading a controlled experiment

The coding study by Peng and colleagues is a careful experiment with a narrow scope.

In 2022, 95 professional programmers were split into an assisted group of 45 and a control group of 50, and each built one JavaScript HTTP server from skeleton code. Among those who finished, the assisted group was 55.8% faster, with an interval of 21% to 89%, but the gain in task success was not significant.

The lecture lists what the study could not measure: code quality, security, learning, maintenance, wages or headcount.

Reading a company's claims

The Klarna case shows how to audit a company announcement. Its 2024 release reported conversation counts, a workload equal to 700 full-time agents, faster resolution and steady satisfaction, all company-reported and unaudited.

The lecture separates what each claim establishes from what it must not be stretched to mean, uses Autor's task framework to explain why no net-employment conclusion follows, and notes that a stated human option is not a tested hand-off.

In this chapter

What this chapter covers

  • 01

    The substitution argument and its hidden assumptions

  • 02

    Occupational exposure to generative AI

  • 03

    Task exposure, adoption and job loss

  • 04

    The 2022 coding-assistant experiment and its limits

  • 05

    Intervals, completers and non-significant results

  • 06

    Auditing a company claim package

  • 07

    Substitution, complementarity and output demand

  • 08

    A human option against a documented hand-off

Worked example · free

Bound the claims a productivity study can support

Q [6 marks]. AskSia-authored practice. A firm runs a four-week trial in which 40 accountants use an AI drafting tool and 40 do not. Assisted accountants close month-end reports 30% faster on average, with a 95% interval of 5% to 55%; error rates in the reports are not measured. The firm's newsletter says the tool 'will let us run finance with a third fewer staff'. Say what the trial supports and what it does not. The marks shown are an AskSia study allocation, not a University marking scheme.
Two results from the same coding experiment, each with its 95% interval.Two results from the same coding experiment, each with its 95% interval. Completion was 55.8% faster among completers, interval 21 to 89, while the 7-point gain in task success has an interval crossing zero.050100Completion time55.8Task success7percent improvement
Two results from the same coding experiment, each with its 95% interval. Completion was 55.8% faster among completers, interval 21 to 89, while the 7-point gain in task success has an interval crossing zero.
  • 2State the result the trial supports, with its interval.
  • 2Name two outcomes the trial did not measure.
  • 2Explain why the staffing claim does not follow.
The trial supports a statement that, over four weeks, assisted accountants completed month-end reports about 30% faster on average, with the true effect plausibly anywhere from 5% to 55%. It did not measure report accuracy, so faster could mean more errors, and it did not measure anything beyond one task over one month, such as review burden, learning or other duties. The staffing claim does not follow, because speed on one task is not a measure of a whole job, and the task framework predicts that some work will be changed rather than removed while demand for output may also shift.
Sia tip — Rewrite any productivity headline as 'on this task, for these people, over this period, measured this way' before you quote it; whatever does not fit the sentence is a claim the study did not make.
Glossary

Key terms

Task Exposure
The extent to which the tasks in an occupation could be performed or changed by a technology, which is not the same as adoption or job loss.
Workload Equivalence
An estimate expressing automated output as the number of full-time staff it would take, which is not a count of jobs removed.
Claim Package
The set of figures a company publishes about its own deployment, each of which needs attribution and a statement of what it does not establish.
Task Framework
Autor's account of automation as substituting for some tasks, complementing others and changing demand for output, so that net employment cannot be read from one channel.
Hand-off Route
The documented path by which a person moves from an automated service to a human, including routing rules, availability, wait times and outcomes.
FAQ

Automation, Work and Productivity Evidence FAQ

Does exposure to generative AI mean jobs will be lost?

No. Exposure measures how much of an occupation's tasks a technology could affect, not whether it will be adopted or whether anyone will lose work. The ILO working paper the lecture cites estimates that about one in four workers globally has some exposure and 3.3% of employment is in the highest-exposure group, and it judges transformation of jobs more likely than wholesale replacement.

What did the GitHub Copilot productivity study actually find?

In a 2022 experiment with 95 programmers, those using the completion assistant finished one standardised JavaScript task 55.8% faster among completers, with a 95% interval of 21% to 89%, and the result held when four slow control outliers were removed. The difference in task success was not significant. It did not measure code quality, security, learning or employment.

How should a company's AI productivity claims be evaluated?

Attribute each figure to the company, say what kind of measurement it is, and state what it does not establish. Ask for counting rules, denominators, time windows, task mix, score methods and a causal comparison, and do not convert a workload-equivalence figure into a count of jobs removed.

To claim shared gains, compare outcomes for contacts the AI closed alone, contacts where it supported a person, and contacts people resolved themselves.

Study strategy

Assessment move

Keep a one-page ledger for any productivity or labour claim you meet: the figure, who reported it, what kind of measurement it is, and what it does not establish. Fill it for the coding study and the Klarna release from memory, then check against this chapter. The ledger is the fastest way to write the evidence paragraph of a news presentation without overstating the source.

Working through Automation, Work and Productivity Evidence in MAIE5103? Sia is AskSia’s AI Arts and Humanities tutor — ask any MAIE5103 Automation, Work and Productivity Evidence question and get a clear, step-by-step explanation grounded in how MAIE5103 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 13 of your subjects at The Hong Kong University of Science and Technology, plus 1,000+ more Bibles across the AskSia library.
Sia - your MAIE5103 tutor, unlimited, worked the way the exam marks it
The full 3-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works