DDES9902 Chap.6 Evaluation, Fidelity and Compliance
Evaluation, Fidelity and Compliance
Evaluation begins with a requirement, not with an impression of realism. A simulation should represent the cues, actions and consequences needed for the target decision while making deliberate simplifications visible. Physical fidelity concerns appearance, functional fidelity concerns behaviour, and psychological fidelity concerns the attention or decision demands created by the scene.
Compliance becomes traceable when each requirement has a scenario trigger, observable behaviour, acceptance rule and corrective response. This approach directs limited project resources toward the parts of fidelity that carry learning, safety or operational value.
Translate each broad objective into a condition that can be observed: the participant notices a hazard within the available time, follows the required sequence or selects the correct escalation route. Link that condition to the event that triggers it, the prototype state that represents it and the evidence that establishes a pass or failure.
Negative cases matter because automatic prompts, forgiving collision or silent correction can conceal an unsafe action while making completion statistics look strong. Fidelity choices should follow the requirement. Visual resemblance may be expendable when control behaviour and consequences carry the learning, while psychological pressure may matter when the target task depends on prioritisation under distraction.
Document every deliberate departure from reality and explain why it does not invalidate the current test. Re-run the trace after optimisation, device changes or revised interaction timing. A compliance claim is bounded to the tested version, participants and conditions; transfer to a workplace or operational setting requires further evidence.
This makes evaluation a chain from purpose to behaviour and response, not a final opinion attached to an impressive demonstration.
What this chapter covers
- 01
Observable requirements rather than broad claims of realism
- 02
Physical, functional and psychological fidelity
- 03
Task-critical cues and consequences in training simulations
- 04
Requirement-to-test traceability
- 05
Acceptance criteria and predefined corrective action
- 06
Boundary between prototype performance and real-world competence
Worked example · free
Review a maintenance training prototype
- +1Translate the goal into an observable sequence of safety-critical checks and actions.
- +1Present the same fault and record skipped steps, ordering errors and recovery after a warning.
- +1Prevent the system from silently correcting an unsafe action because that would hide weak learning.
- +1Report visual or physical simplifications and avoid claiming the prototype certifies field competence.
Key terms
- Physical fidelity
- The degree to which objects and surroundings resemble the target setting.
- Functional fidelity
- The degree to which controls, behaviour and consequences match the target system.
- Psychological fidelity
- The degree to which a simulation reproduces relevant attention, pressure or decision demands.
- Requirement traceability
- A documented link from a requirement through test evidence to a design decision.
- Acceptance criterion
- A predefined observable condition used to judge whether a requirement has been met.
- Corrective action
- A named design, process or escalation response triggered when evidence fails an acceptance rule.
Evaluation, Fidelity and Compliance FAQ
Does higher visual realism mean higher training fidelity?
Not necessarily. Visual resemblance is physical fidelity, while a training goal may depend more on correct control behaviour, consequences and decision pressure. A realistic scene can still teach the wrong sequence. Allocate detail to the cues required by the task and verify behaviour.
What belongs in a requirement-to-test matrix?
Include the requirement, scenario condition, observable behaviour or measure, acceptance rule, evidence source and response to failure. This creates a visible chain between why the experience exists and what the team will change when a test does not meet the need.
Why must simulation limits appear near the conclusion?
A credible representation can encourage overgeneralisation. Placing the simplification or missing behaviour beside the result helps a reviewer understand which decisions are supported. It prevents success in a prototype task from being presented as proof of real-world qualification or safety.
Assessment move
Take a prototype feature and classify its fidelity contribution as physical, functional or psychological. Then ask which category the requirement actually needs. Convert broad goals into observable statements with a condition and a consequence. Build a compact traceability matrix and make the failure response explicit before testing.
During review, look for silent assistance, unrealistic recovery and decorative complexity that consumes resources without changing the target behaviour. Practise writing boundary statements that preserve the value of a result while limiting its reach. For assessment evidence, show how a requirement influenced prototype scope, how the test exposed a pass or failure, and what action followed.
This produces a stronger argument than a showcase of technical detail because it ties effort to purpose and makes trade-offs reviewable.
