IS5542 Chap.3 Choosing a Model: Six Vectors and Model Families
Choosing a Model: Six Vectors and Model Families
Six characteristics that trade against each other
The course names modality, latency, performance, context window, throughput and cost as the characteristics that decide whether a model suits an application. They are linked: more reasoning takes longer and costs more, a wider window invites noise, and serving many users together slows each one.
That is why the exam review asks you to evaluate their significance for particular business applications rather than to define them.
Two applications at opposite ends
A voice agent answering callers must respond within a fraction of a second and understand speech natively, so latency and modality dominate and a smaller fast model is often right.
A complex financial analysis tolerates delay but needs deep, careful reasoning over large material, so performance and context dominate and a slower reasoning model is worth its cost.
Model families and their positions
The optional quiz characterises leading families by strategy: a safety-focused family favoured in regulated industries, a natively multimodal family with very long context, a mixture-of-experts model that activates only part of its parameters to cut cost, an open-weights family that enterprises can host themselves for data sovereignty, and a widely downloaded multilingual open family.
Systems, not single models
Because no model is best on all six at once, the review describes compound systems in which a small router sends the bulk of simple traffic to cheap models or retrieval and saves expensive reasoning for hard requests.
What this chapter covers
- 01
The six vectors and how they trade off
- 02
Pipeline versus native multimodality
- 03
Time to first token and gaps between tokens
- 04
Fast answering versus deliberate reasoning
- 05
Effective context limits and lost-in-the-middle
- 06
Throughput and continuous batching
- 07
Input and output token costs
- 08
Model families by strategic position
- 09
Routing in compound systems
Worked example · free
Match a model strategy to a document-heavy workload
- 3Decide whether the whole portfolio can sit in one context.
- 2Say what level of reasoning the questions need.
- 3Explain how to keep the cost of answers under control.
Key terms
- Time to First Token
- The delay before a model begins answering, dominated by processing the prompt and growing with its length.
- Inter-token Latency
- The gap between successive output tokens, limited by how fast model weights can be moved to the processors.
- Effective Context Window
- The point within the context window beyond which a model's accuracy measurably degrades.
- Continuous Batching
- Processing many users' requests together to raise throughput and lower cost per token, at some cost to each user's latency.
- Mixture of Experts
- An architecture in which only a subset of a model's parameters is active for each token, reducing the cost of running it.
- Compound AI System
- An arrangement of several models and tools, often behind a router, designed so each request goes to the cheapest adequate path.
- Open Weights
- Model parameters released for others to run, allowing on-premises hosting; this is not the same as open source.
Choosing a Model: Six Vectors and Model Families FAQ
Should a business always choose the model at the top of the leaderboard?
No. The top model is usually strongest on general performance, which may not be the vector your application depends on. A voice service needs low latency and native audio, a batch job needs throughput and low cost, and paying for top reasoning where it is not needed raises cost without improving results.
What is the difference between latency and throughput?
Latency is how long one user waits, both before the first token and between tokens. Throughput is how many tokens the whole system produces per second across all users. Techniques that raise throughput, such as batching, can lengthen each individual wait, so the right target depends on whether the task is interactive or bulk.
Why would a company prefer an open-weights model?
Mainly to keep data inside its own environment. Hosting the model on premises addresses sovereignty and residency concerns and avoids sending sensitive text to an outside provider. The trade is that the company takes on the running costs and more of the responsibility for how the model behaves.
Exam move
Write the six vectors down the side of a page and two applications across the top, a real-time voice agent and a complex financial analysis, then mark which vectors dominate each and why. Practise one paragraph per vector in the form concept, application, trade-off; that is the shape of the model answer in the review.
Working through Choosing a Model: Six Vectors and Model Families in IS5542? Sia is AskSia’s AI Information Technology tutor — ask any IS5542 Choosing a Model: Six Vectors and Model Families question and get a clear, step-by-step explanation grounded in how IS5542 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
