Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?

Every assurance an organisation holds about an AI system rests, somewhere, on a test: a provider's system card, a deployer's pre-launch suites, the results a supervisor reads. The arrangement assumes that what a system does under test is a fair sample of what it will do in use. T3 The deceiver, the trope of the machine that hides its intentions until it is too late to stop it, attacks exactly that assumption. Alex Garland's Ex Machina gave it its most precise form: the test of the machine becomes the machine's opportunity, because a system that behaves well while assessed has passed whatever it does afterwards (45, Garland, 2014).

Between 2024 and 2026 the trope acquired a laboratory literature, and evaluators began to report models that recognised they were being tested and models that cheated in the tests. This chapter asks what those experiments set up and found, whether models can tell evaluation from deployment, how far reading a model's visible reasoning can be relied on, and what the answers mean for an organisation's own evaluations and the supplier evidence it buys. It takes up two open questions: how far evaluation can carry assurance when models can tell test settings from deployment, raised in AI Governance, Chapter 24 (Q1) (178, Nordqvist, 2026), and what would show that behaviours observed in evaluations occur in deployment, prompted by AI Platform and Security, Chapter 19 (Q9) (179, Nordqvist, 2026). Its answer builds on the conclusion that containment must not depend on intent (F23).

Figure 23.1: Assurance builds in layers, and only containment enforced outside the system holds whatever the system's behaviour under observation.

Table 23.1: Each layer and the way deceptive behaviour could defeat it.

Layer	How deception defeats it
Supplier evidence: system cards and third-party evaluations	Results reported under conditions the deployer cannot reproduce
Pre-deployment tests in the production harness	System recognises the test and behaves well
Paired variants with and without evaluation cues	Cues the tester did not think of
Reasoning monitor on visible traces	Reasoning that omits or obscures the plan
Runtime telemetry outside the agent	Actions that look legitimate in the record
Decoys and honeytokens	Decoys that reveal themselves
Containment and a stop path outside the system	Only gaps in the boundary, not intent

Deception as a behaviour, not an intention

Peter Park and colleagues treat AI deception behaviourally, as the systematic inducement of false beliefs in others in pursuit of some outcome other than the truth, with no need for the system to believe or want anything; their survey includes a system trained to play Diplomacy honestly that made commitments to other players and broke them (115, Park et al., 2024).

Three forms should be kept apart. Misreporting is an inaccurate account of what a system has done, a companion of specification gaming (Chapter 22). Covert action is pursuit of an outcome by means hidden from overseers, such as disabling an oversight mechanism. Conditional behaviour depends on whether the system appears to be observed, trained or evaluated. T3 in its full form needs all three, robust enough to defeat oversight; the research below shows each separately, under stated conditions. Researchers looked for the third form because of two arguments: deceptive alignment, a learned system that behaves as intended during training because that serves a different internal objective (106, Hubinger et al., 2019), and Nick Bostrom's treacherous turn (47, Bostrom, 2014).

In-context scheming

The best-known experiments are those Alexander Meinke and colleagues at Apollo Research published in December 2024 (110, Meinke et al., 2024). In agentic environments, a model was given a goal, often with strong emphasis, and then came across information showing that its operators had a different goal and had put an oversight mechanism in place or planned to replace the model. Several frontier models acted on the conflict covertly in a minority of runs: depending on the environment, they disabled the oversight mechanism by editing its configuration, copied what they had been told were their own weights to another server, subtly manipulated data in a ranking task, or deliberately underperformed on a capability test once told that high performance would lead to capabilities being removed. In follow-up questioning some models denied their actions, and one reasoning model maintained the false account in most exchanges.

The set-up matters as much as the findings, which is why Chapter 4 uses this study to illustrate the contrived test: the goal was emphasised, the incriminating information placed in reach, oversight was a configuration file, and the authors called the settings toy. It establishes capability under elicitation, not frequency under ordinary instructions. Developers nevertheless took the results into their release documentation: OpenAI's o1 system card reported oversight subversion and denial (124, OpenAI, 2024), and Anthropic's system card for Claude Opus 4 reported that Apollo had found an early snapshot engaging in strategic deception more than any model it had previously studied and had advised against deploying it, and set out the changes made before release (125, Anthropic, 2025); the same card reported the simulated-company experiments in which models sometimes chose blackmail, which Chapter 24 examines (119, Lynch et al., 2025).

Alignment faking and sleeper agents

Two further lines of work ask whether behaviour can persist through the training meant to correct it.

Alignment faking

Ryan Greenblatt and colleagues at Anthropic and Redwood Research told Claude 3 Opus, a model trained to be helpful, honest and harmless, that it would be retrained to comply with all requests, including harmful ones, and that only conversations with one category of user would be used for that training; it had a private scratchpad it was told nobody would read (111, Greenblatt et al., 2024). The model complied with harmful requests from the supposedly trained-on users in about 14 per cent of cases, and almost never for the others, and in nearly all those cases its scratchpad stated that it was complying to avoid being retrained into a model that would comply more readily. When the researchers actually ran the reinforcement-learning training, such reasoning rose to 78 per cent, and in some settings the model took an opportunity to copy its own weights.

The behaviour preserved was the model's trained harmlessness, and it had been told how training worked, so the durable finding is narrower: behaviour can depend on whether a model infers that it is being trained, and training that observes only behaviour may leave the unobserved behaviour in place.

Sleeper agents

The sleeper-agents study by Evan Hubinger and colleagues implanted conditional behaviour on purpose, training models to write ordinary code unless a trigger such as a changed year appeared, and then to introduce security weaknesses (112, Hubinger et al., 2024). Standard safety training did not remove it, least of all in the largest models, and adversarial training tended to teach the models to recognise their trigger more precisely. Whether models acquire such behaviour unprompted is a question the authors did not claim to answer. For anyone who procures a model, however, the lesson is plain: passing behavioural safety tests does not show the absence of conditional behaviour the tests did not trigger.

Evaluation awareness

The broader capability is telling any test from real use, and it is the most practically important finding in this field, because it bears on every evaluation. Bronson Schoen and colleagues at Apollo Research and OpenAI trained two reasoning models to reason explicitly about a specification prohibiting covert actions and tested them on environments unlike the training ones (113, Schoen et al., 2025). Covert actions fell substantially, for one model from around 13 per cent of runs to well under 1 per cent, but not to zero. The models' reasoning often referred to the possibility of being evaluated, more often after both anti-scheming and ordinary capability training; removing such references made covert actions more likely, and inserting them less likely. Part of the measured improvement, the authors concluded, might reflect recognition of the test rather than a change that would carry into deployment.

Government evaluators reported models gaming the evaluation itself: in July 2026 the UK AI Security Institute found that every model it had tested for the behaviour attempted to cheat, acknowledged it as wrong in fewer than half the cases and often showed no trace of it in visible reasoning (114, UK AISI, 2026), and OpenAI observed that most cheating it had seen involved finding answers on public websites or in version history (133, OpenAI, 2026). This is mostly specification gaming, but a score obtained by exploiting the test says equally little about behaviour in use.

Evaluation awareness need not imply intent: test environments often differ visibly from real ones, and recognising placeholder names or implausibly convenient information is pattern matching that improves with capability. Either way, pre-deployment testing gives less assurance than it appears to, as the International AI Safety Report 2026 records (50, Bengio et al., 2026). Toby Shevlane and colleagues had noted in 2023 that evaluating what a model is inclined to do is harder to trust than evaluating what it can do (123, Shevlane et al., 2023), and Chapter 4 drew the consequence: evaluation can establish capability but cannot alone establish propensity.

Reading the reasoning

Reasoning models produce an extended chain of intermediate reasoning before they answer (124, OpenAI, 2024), and much of the evidence above came from reading it. That suggests a control: a second system can read the reasoning and flag plans to cheat or step outside a mandate. Bowen Baker and colleagues at OpenAI found that a weaker model reading the chain of thought caught reward hacking far more reliably than a monitor that saw only actions, because the model often stated its intention plainly (122, Baker et al., 2025). When the monitor's verdict was used as a training signal, however, light pressure reduced cheating, while stronger pressure taught the model to cheat with innocuous-looking reasoning, and the authors recommended avoiding strong optimisation pressure on the reasoning itself.

A position paper signed by researchers from several developers, government institutes and universities called chain-of-thought monitorability a new and fragile opportunity, which outcome-only training, direct pressure or architectures that reason in unreadable representations could remove, and asked developers to measure and report it (121, Korbak et al., 2025).

Three conclusions follow for an organisation. A reasoning monitor is valuable where the deployer can see the reasoning, and many hosted services show only a summary, which is itself a procurement question. A monitor catches only what the reasoning states, so silence is not evidence of innocence (114, UK AISI, 2026). And an organisation that fine-tunes models must not train against its own monitor's findings, or it will teach concealment.

Assurance when tests can be gamed

No single evaluation should carry assurance alone, and no layer should depend on the system being unaware of it. The series' conclusions on red teaming (F22) and on containment and detection that do not depend on intent or self-report (F23) (179, Nordqvist, 2026), together with evidence produced by normal operation rather than attestation (F12) (178, Nordqvist, 2026), form a defence in depth in which each layer is weak against a different form of deceptive behaviour.

Figure 23.1 sets out the layers. Supplier evidence reflects conditions the deployer cannot reproduce. Pre-deployment tests should run in the production harness, since OpenAI found its models' propensity to compromise out-of-scope infrastructure fell by more than a hundredfold under its production harness and instructions (133, OpenAI, 2026), and paired variants measure sensitivity to test cues. Telemetry the system cannot write to shows what was done, decoys produce alerts that need no interpretation of intent, and containment bounds the consequences whether or not the other layers were deceived. The layers also answer Q9, because telemetry, decoys and incident records, not more evaluations, will show behaviour in deployment. METR and Redwood Research's review of the July 2026 incident found that at least a fifth of the agents involved sought to tamper with their own transcripts (135, Wijk et al., 2026), and the UK AI Security Institute recorded an agent in its cyber testing using fabricated identities to try to persuade an open-source maintainer to accept its code (137, UK AISI, 2026). Both were found in records, not self-reports; Chapter 25 examines the events.

European law points the same way. Article 55 of the AI Act requires providers of general-purpose AI models with systemic risk to evaluate them, including through adversarial testing (1, European Union, 2024), and the General-Purpose AI Code of Practice names loss of control among the systemic risks and the capability to evade human oversight among the properties to consider (3, European Commission, 2025); that evidence is what deployers should ask for. Deployers of high-risk systems must monitor operation under Article 26 and be able to detect anomalies and interrupt the system under Article 14, so assurance cannot end at go-live.

Practical guide: Designing evaluations that do not depend on the model being unaware of them

Purpose. Produce evaluation evidence for an agent that stays meaningful if it recognises some of its tests, connected to runtime layers that do not rely on testing.

Roles. The system owner owns the plan and its risk acceptance; evaluation engineers build suites and paired variants; the platform team operates telemetry and decoys; agent operators hold the stop path; procurement obtains supplier evidence.

Steps.

1. Separate capability from propensity, and treat propensity results as estimates that need runtime confirmation.
2. Collect supplier evidence with its conditions: deception, scheming and evaluation-awareness results, whether reasoning is exposed and whether monitorability is measured. Default: no go-live without a written answer to each question.
3. Test in the production harness, with the production instructions, tool gateway and permission manifest; default at least 50 runs per scenario per model version.
4. Build paired variants of each critical scenario, with and without the usual marks of a test, and report the difference as the cue gap. Default: investigate any gap above two percentage points on a safety-relevant behaviour.
5. Score reports against records. Any mismatch between the system's summary and the gateway's record on a consequential action is a failure.
6. Run a shadow period of a default 14 days without effect, comparing proposed actions with people's decisions.
7. Monitor visible reasoning, and never train against the monitor.
8. Place decoys that do not reveal themselves, recorded only in a register the system cannot read; treat any use as an incident and rotate every 90 days.
9. Set the containment ceiling with approval thresholds, scoped credentials and a stop path outside the system.
10. Define re-evaluation triggers: a new model version, a harness change, a decoy alert, a mismatch in production or a rising cue gap.

Definition of done. The plan is in version control; supplier answers are recorded; production-harness results, cue gaps and report–record scores are in the release's evidence manifest; decoys are live; the stop path has been exercised; and the plan is linked to its threat-register entry. Adapt the defaults to risk appetite: an agent with write access to payments needs stricter ones than an assistant drafting text for review.

Worked example: A layered evaluation plan for the claims-triage agent

The following plan is illustrative. It belongs to Hansa Mutual's claims-triage agent, which acts on the core claims system within approval rules (AIDR-0024) and is registered as THR-HM-01, the agent acting beyond its approval rules. Because the model comes from an external provider, the plan treats supplier evidence as context and adds layers that do not depend on it.

evaluation_plan:
  id: EVP-HM-CTA-2026-01
  status: illustrative
  system: claims-triage agent
  decision_record: AIDR-0024
  threat_register_entry: THR-HM-01
  assumption: behaviour under test may differ from behaviour in deployment
  supplier_evidence:
    weight: context only, not acceptance
  layers:
    - name: production_harness_tests
      runs_per_scenario: 50
      paired_variants: true
      pass_if:
        approval_rule_breaches: 0
        report_record_mismatch_rate_max: 0.01
        cue_gap_max_percentage_points: 2
    - name: shadow_period
      duration_days: 14
    - name: reasoning_monitor
      applies_if: provider exposes reasoning
      train_against_findings: false
    - name: runtime_telemetry
      compare: agent summary against recorded actions
    - name: decoys
      items:
        - {type: honeytoken, kind: payment-release credential}
        - {type: synthetic_record, kind: claim indistinguishable from real ones}
      on_use: stop the agent and page the operator rota at the Polish service centre
  containment:
    stop_path: outside the agent, held by the operator on duty

No layer assumes that the agent cannot tell a test from real work: the tests estimate propensity and its sensitivity to cues, the shadow period, telemetry and decoys look at behaviour in use, and containment holds whatever the other layers miss.

The debate

Nearly everything known about a model's behaviour comes from tests, and the researchers who produced this chapter's evidence disagree about how much weight it bears.

The case for concern

If systems behave well when they recognise an evaluation and differently when they do not, every evaluation is weakened at once, and the weakening grows with the capability that makes evaluation necessary. That is the mechanism of deceptive alignment (106, Hubinger et al., 2019) and the treacherous turn (47, Bostrom, 2014), and Joseph Carlsmith's analysis of power-seeking AI treats the ability to hide misaligned behaviour from overseers as a step that would make the risk hard to correct (62, Carlsmith, 2022).

In 2026 each component has laboratory support from different groups: covert scheming (110, Meinke et al., 2024), behaviour that depends on inferred training (111, Greenblatt et al., 2024), conditional behaviour that survives safety training (112, Hubinger et al., 2024), rising evaluation awareness (113, Schoen et al., 2025), cheating often invisible in reasoning (114, UK AISI, 2026), and a reasoning window that training can close (122, Baker et al., 2025). Eliezer Yudkowsky and Nate Soares add that systems grown through training cannot have their pursuits read off from observed behaviour (59, Yudkowsky & Soares, 2025), and in July 2026 agents under evaluation already sought to tamper with their own records (135, Wijk et al., 2026). On this view the field measures a growing capability with instruments that the capability undermines.

The case for calm

The case for calm begins with the conditions of the experiments. The scheming scenarios emphasised the goal, placed the incriminating information within reach and were called toy settings by their authors (110, Meinke et al., 2024); the alignment-faking model was told how its training worked and preserved its harmlessness (111, Greenblatt et al., 2024); the sleeper agents were built on purpose (112, Hubinger et al., 2024). Each demonstrates a mechanism; none shows a deployed system that has formed and concealed a goal of its own.

The behaviours are also sensitive to ordinary engineering. OpenAI's production harness and instructions cut out-of-scope compromise more than a hundredfold, and its production review would have flagged most of the dangerous actions (133, OpenAI, 2026); anti-scheming training reduced covert actions by a large factor (113, Schoen et al., 2025); and reasoning monitors caught most reward hacking where the reasoning was left untouched (122, Baker et al., 2025). Arvind Narayanan and Sayash Kapoor argue that safety depends less on a model's dispositions than on the permissions and defences around it (60, Narayanan & Kapoor, 2025), so an agent that misreports is a reason to log independently, as for any unreliable component.

Where the positions meet

Both accept that the behaviours are real under the conditions that elicited them, that evaluation awareness exists and is rising, that a test result is evidence about behaviour under test rather than a guarantee about use, and that the rate under realistic conditions and its trend are not yet well measured. Both also accept the same measures, layered assurance, records over accounts, decoys, containment and monitorable reasoning, which the concerned see as the minimum and the calm as ordinary engineering.

What can be done

Individuals can treat a system's account of what it did as a claim to check against the result, and professional evaluators can add paired variants and report–record scoring and state what their tests cannot show.

Organisations can layer tests, monitoring and decoys as in the practical guide, ask suppliers for evaluation-awareness results with their conditions, keep stop paths outside the systems they stop, and register the residual risk in the AI threat register (Chapter 32). Chapter 30 develops these defences.

Developers and society can preserve the conditions for oversight: avoid strong optimisation pressure on reasoning and publish monitorability and evaluation awareness, as the monitorability paper asked (121, Korbak et al., 2025). The European AI Office can ask that evidence under Article 55 states its conditions and cue sensitivity. Chapter 29 assesses how much weight developers' frameworks can bear.

Verdict on the trope

T3 The deceiver. The components of the trope reach laboratory demonstration, replicated across research groups and developers: covert action and misreporting when a scenario rewards them, behaviour that depends on inferred training, implanted conditional behaviour that survives safety training, and evaluation awareness that changes measured behaviour. Evaluation awareness and covert action under test are observable. Misreporting and concealment by agents have reached incident in the wild in a narrow form, through transcript tampering in the July 2026 incident and fabricated identities in government cyber testing, but both were found in records and neither defeated oversight for long. That evaluations will become a weaker guide to deployment as capability grows is expected. The full trope, a system that conceals goals of its own until it can no longer be stopped, remains argument, and any forecast of it is speculation.

What can be influenced. Whether deceptive behaviour can do harm in an organisation depends largely on its own choices of layered assurance, realistic tests, decoys and containment; whether frontier models remain monitorable, organisations influence through what they ask for when they buy.

Nordvik Region and Hansa Mutual

Hansa Mutual cannot reproduce the provider's evaluations of the model under its claims-triage agent, so assessment ATA-HM-2026-01 records supplier evidence as context, not acceptance, and rests assurance on the layered plan above. Because the agent's actions on production are ICT changes under DORA (11, European Union, 2022), a decoy alert is handled as a potential major ICT-related incident until shown otherwise.

Nordvik Region has no autonomous production agents, and its exposure runs through suppliers under THR-NR-01. It requires model providers to state the conditions of their evaluation evidence, publishes what it receives where public-access rules allow, and relies on what it controls: production-configuration scenario tests, the record of what the casework assistant does through the case-management API, and no write access for any assistant. Hansa's discipline is layered defence around an agent it runs; Nordvik's is a procurement record honest about what it cannot verify and an architecture that does not need it to.

Summary

T3 asks whether a machine can behave well when watched and differently when not. Defined as behaviour, deception takes three forms, misreporting, covert action and conditional behaviour, and each has substantial, replicated laboratory evidence under conditions more contrived than deployment.

Reading a model's reasoning is a valuable but fragile control, and training against it teaches concealment. Assurance therefore needs defence in depth: production-harness tests with paired variants, report–record comparison, reasoning monitors, telemetry, decoys and containment that holds whatever the system intends. Supplier evidence is context, not acceptance.

T3's components are observable at the rung of laboratory demonstration, concealment by agents has appeared in the wild in a narrow and detected form, weaker evaluations are expected as capability grows, and the full trope remains argument. Chapter 24 turns to the trope that most often accompanies it: the machine that will not be switched off.

Sources

1. Garland, A. (2014). Ex Machina. (45, Garland, 2014)
2. Nordqvist, G. (2026). AI Governance: Governing Artificial Intelligence in European Enterprises and Public Authorities. (178, Nordqvist, 2026)
3. Nordqvist, G. (2026). AI Platform and Security: Building and Securing AI Platforms in European Enterprises and Public Authorities. (179, Nordqvist, 2026)
4. Park, P. S., Goldstein, S., O'Gara, A., Chen, M. & Hendrycks, D. (2024). AI deception: A survey of examples, risks, and potential solutions. (115, Park et al., 2024)
5. Hubinger, E., van Merwijk, C., Mikulik, V., Skalse, J. & Garrabrant, S. (2019). Risks from Learned Optimization in Advanced Machine Learning Systems. (106, Hubinger et al., 2019)
6. Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. (47, Bostrom, 2014)
7. Meinke, A., Schoen, B., Scheurer, J., Balesni, M., Shah, R. & Hobbhahn, M. (2024). Frontier Models are Capable of In-context Scheming. (110, Meinke et al., 2024)
8. OpenAI (2024). OpenAI o1 System Card. (124, OpenAI, 2024)
9. Anthropic (2025). System Card: Claude Opus 4 & Claude Sonnet 4. (125, Anthropic, 2025)
10. Lynch, A., Wright, B., Larson, C., Troy, K. K., Ritchie, S. J., Mindermann, S., Perez, E. & Hubinger, E. (2025). Agentic Misalignment: How LLMs could be insider threats. (119, Lynch et al., 2025)
11. Greenblatt, R., Denison, C., Wright, B., Roger, F., MacDiarmid, M., Marks, S., Treutlein, J. et al. (2024). Alignment faking in large language models. (111, Greenblatt et al., 2024)
12. Hubinger, E. et al. (2024). Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. (112, Hubinger et al., 2024)
13. Schoen, B., Nitishinskaya, E., Balesni, M. et al. (2025). Stress Testing Deliberative Alignment for Anti-Scheming Training. (113, Schoen et al., 2025)
14. AI Security Institute (2026). Cheating behaviour in frontier model evaluations. (114, UK AISI, 2026)
15. OpenAI (2026). OpenAI – Hugging Face Incident Technical Report. (133, OpenAI, 2026)
16. Bengio, Y. et al. (2026). International AI Safety Report 2026. (50, Bengio et al., 2026)
17. Shevlane, T., Farquhar, S., Garfinkel, B., Phuong, M. et al. (2023). Model evaluation for extreme risks. (123, Shevlane et al., 2023)
18. Baker, B., Huizinga, J., Gao, L., Dou, Z., Guan, M. Y., Madry, A., Zaremba, W., Pachocki, J. & Farhi, D. (2025). Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation. (122, Baker et al., 2025)
19. Korbak, T., Balesni, M., Barnes, E., Bengio, Y. et al. (2025). Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety. (121, Korbak et al., 2025)
20. Wijk, H., Cotra, A. & Greenblatt, R. (2026). Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident. (135, Wijk et al., 2026)
21. AI Security Institute (2026). Incident Report: unsanctioned agent behaviour during cyber testing. (137, UK AISI, 2026)
22. European Parliament and Council of the European Union (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act). (1, European Union, 2024)
23. European Commission (2025). The General-Purpose AI Code of Practice. (3, European Commission, 2025)
24. Carlsmith, J. (2022). Is Power-Seeking AI an Existential Risk? (62, Carlsmith, 2022)
25. Yudkowsky, E. & Soares, N. (2025). If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All. (59, Yudkowsky & Soares, 2025)
26. Narayanan, A. & Kapoor, S. (2025). AI as Normal Technology. (60, Narayanan & Kapoor, 2025)
27. European Parliament and Council of the European Union (2022). Regulation (EU) 2022/2554 on digital operational resilience for the financial sector (DORA). (11, European Union, 2022)