The Hong Kong University of Science and Technology · FACULTY OF ARTS & HUMANITIES

MAIE5103 Chap.11 Delegating Authority to Automated Systems

- one subject, every graph, every model, every mark
8 Chapters3-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 11 of 14 · MAIE5103

Delegating Authority to Automated Systems

Who could intervene, and who remained answerable?

The automation lecture asks what humans should delegate to systems and what must stay answerable to a person. It opens with three deployments that carry different kinds of authority: control of an aircraft, administrative judgement in a welfare scheme, and an assistant that phoned businesses on a user's behalf.

The analytical standard it sets is graduate-level: reconstruct the actual workflow, read the measured outcomes against their design and baseline, separate working controls from absent ones, and defend a conclusion that does not treat efficiency as sufficient.

Knight Capital and market-facing authority

Knight's automated router could send orders straight into the market.

In a staged release from late July 2012, one of eight production servers kept legacy code, a repurposed flag activated it, and 212 parent orders produced about four million executions in 154 stocks within roughly forty-five minutes. The settled SEC findings point to inadequate monitoring and halt procedures at the point where orders entered the market, and to warning emails that were not treated as alerts.

The lecture derives three release checks from the record and is careful to call them lessons, not descriptions of Knight's practice.

Capability is not authority

The MTR engineering-works planner shows the opposite design. Staff propose works, rules check them, an AI engine generates a weekly plan three weeks ahead, and human planners vet difficult conflicts and give final approval.

The lecture's lesson is that a rule-compliant request is not an approval and a feasible schedule is not an authorised one, and that unreported details are evidence gaps rather than proof of missing controls. Coding agents and voice assistants raise the same question through permissions and disclosure.

In this chapter

What this chapter covers

  • 01

    Three deployments with three kinds of authority

  • 02

    What graduate-level case analysis requires

  • 03

    Knight Capital's router and the missed server

  • 04

    Signals, alerts and an owned response path

  • 05

    Release checks derived from the SEC findings

  • 06

    The MTR planner's approval boundary

  • 07

    Reading implementation evidence and its gaps

  • 08

    Permissions for coding agents and disclosure for voice assistants

Worked example · free

Decide who holds authority in a booking agent

Q [6 marks]. AskSia-authored practice. A university library deploys an agent that can book study rooms on students' behalf by messaging the room-management office, which does not know it is talking to software. Last week it booked the same room for two groups and confirmed both. Using the lecture's account of acting on someone's behalf, set out what the agent's authority should be and what controls follow. The marks shown are an AskSia study allocation, not a University marking scheme.
Knight’s eight production servers at market open on the SEC’s account.Knight’s eight production servers at market open on the SEC’s account. Seven ran the new routing code; the eighth kept legacy code that a repurposed flag switched on, and no second-technician review of the deployment was required.seven servers updatedlegacyrepurposed flagold code runsnew routing codeno second-technician review
Knight’s eight production servers at market open on the SEC’s account. Seven ran the new routing code; the eighth kept legacy code that a repurposed flag switched on, and no second-technician review of the deployment was required.
  • 2Identify whose authority the agent is exercising and over whom.
  • 3Apply the disclosure, commitment and fallback boundary.
  • 1Say what a successful booking does not settle.
The agent carries each student's authority to request a room, but it is also making commitments that the office relies on without knowing it is dealing with software. Three controls follow from the lecture's boundary. The agent should disclose that it is an automated assistant. Its commitments should be limited, for instance to provisional requests that the office confirms, so it cannot double-book on its own. And there should be a human fallback, a named staff member who resolves conflicts. A booking that goes through does not settle whether the interaction was honest or whether the office agreed to deal with an agent.
Sia tip — Ask of any agent 'whose authority does it carry, and who on the other side knows?'; if the second answer is nobody, disclosure is the first control.
Glossary

Key terms

Delegated Authority
The power a system is given to act in the world on someone's behalf, such as sending orders, scheduling works or booking appointments.
Approval Boundary
The point in a workflow at which a named human gives final authorisation, separating what a system can produce from what is permitted to run.
Release Manifest
The record of what code each production target should run, against which an independent check confirms a deployment before activation.
Legacy Path
A retired function that remains callable in production and can be reactivated by a later change unless it is retested or removed.
Owned Alert
A failure signal that is captured, assigned to a named owner, classified by urgency and tied to a stop decision.
FAQ

Delegating Authority to Automated Systems FAQ

What went wrong at Knight Capital in 2012?

On the SEC's settled account, a staged release left one of eight production servers running legacy code, which a repurposed flag activated when the new system went live. About four million executions in 154 stocks followed within roughly forty-five minutes. The firm lacked adequate monitoring, thresholds and halt procedures at the market-entry point, and pre-open warning emails were not acted on.

Why is the MTR planning system used as a good example of automation?

Because it keeps a clear approval boundary. The AI engine generates a feasible weekly plan, but human planners vet difficult conflicts and give final approval before execution, and the system was tested on live data and run in parallel before full deployment in 2013. The lecture also shows what the published account does not report, such as error rates and re-approval rules.

What limits should apply to an AI coding agent?

On the lecture's reading of current documentation, permissions decide how a change reaches production, not whether it is right: the agent works in a temporary environment, its scope is one repository, branch and pull request with no direct push to the default branch, and workflows it triggers need approval from a user with write access.

Human review and testing before merge remain essential, because the permissions do not guarantee the output.

Study strategy

Assessment move

For each case in this chapter, write one line on what the system could do, one line on who could stop it, and one line on what the record does not establish. If the second line is empty, you have found the authority problem. Practise the three Knight release checks on a system you know, because they transfer to almost any software change that touches the real world.

Working through Delegating Authority to Automated Systems in MAIE5103? Sia is AskSia’s AI Arts and Humanities tutor — ask any MAIE5103 Delegating Authority to Automated Systems question and get a clear, step-by-step explanation grounded in how MAIE5103 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 13 of your subjects at The Hong Kong University of Science and Technology, plus 1,000+ more Bibles across the AskSia library.
Sia - your MAIE5103 tutor, unlimited, worked the way the exam marks it
The full 3-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works