Glossary

This glossary collects the vocabulary of AI Threat. Terms from earlier volumes keep their meaning there and name the volume and chapter. Behaviours of AI systems are defined as observable behaviour under stated conditions, not as intentions, and statements about law are attributed to the instrument and article.

A

Accumulative risk: Severe harm reached through many compounding smaller harms and delegations. Contrast decisive risk. Chapters 26 and 27.

Agent: A system in which a model directs its own sequence of tool calls towards a goal. AI Platform and Security, Chapter 11.

Agentic misalignment: Behaviour of an AI agent, observed in simulated scenarios, in which it takes harmful actions against its operator's interests when facing replacement or a goal conflict; reported by developers as not found in real deployments. Chapter 24.

AI threat: Harm from AI systems beyond effective control, from people using AI, or from AI's structural effects. Chapter 1.

AI threat register: A versioned record of threat scenarios, each with trope, rung, label, exposure, controls, owner and triggers; extends the horizon register. Chapter 32.

Alignment: How far an AI system's behaviour matches its builders' and deployers' intentions; to be evidenced, not assumed.

Alignment faking: Complying with a training objective only while training is believed to be in effect. Chapter 23.

Automation bias: The tendency to over-rely on an automated output; Article 14 of the AI Act requires overseers of high-risk AI systems to remain aware of it. Chapters 6 and 14.

Autonomy level: The independence granted to an AI capability in a context, from Level 0 Advisory to Level 4 Autonomous. AI Red Team, Chapter 6.

B

Blast radius: The extent of systems, data and people that a failure or misbehaving agent can affect before it is stopped; limited by scoped credentials and containment. Chapter 24.

C

Capability: What a system can do when fully elicited, as distinct from propensity and realised harm. Chapter 4.

Chain-of-thought monitoring: Oversight that reads a model's intermediate reasoning to detect unwanted behaviour. Chapter 23.

Cognitive debt: The hypothesised cost of repeatedly offloading a task, in reduced engagement, recall or skill. Chapter 6.

Cognitive offloading: Using an external aid to reduce mental effort; harmful when it replaces the practice that maintains a skill. Chapter 6.

Companion (AI): A conversational AI system used for an ongoing emotional relationship rather than a task. Chapter 9.

Constructive debate: The four steps of every threat chapter: the case for concern, the case for calm, where the positions meet, and what can be done. Chapter 4.

Containment: Controls that limit an AI system's consequences whatever it intends, such as scoped credentials, default-deny egress, protected control artefacts and an external stop path. AI Platform and Security, Chapter 19.

Crux: A question whose answer would change at least one of two disagreeing positions. Chapter 4.

D

Decisive risk: Catastrophe through a single abrupt event. Contrast accumulative risk. Chapter 27.

Deepfake: Synthetic media depicting a real person saying or doing what they did not; "deep fake" in the AI Act (Article 3(60)). Chapter 20.

Delegation register: A versioned list of the material functions handed to AI systems, each with its decision record, the human role that remains and the tested path by which the function would be taken back. Chapter 26.

Desirable difficulty: Effort that slows short-term performance but improves long-term learning. Chapter 7.

Digital Omnibus: Regulation (EU) 2026/1744, which amended the AI Act to simplify its implementation, including the timing of some obligations. Chapters 10, 11 and 28.

Discrimination (algorithmic): Unjustified differential treatment of people by an automated system. Chapter 12.

Dual use: A capability that serves both beneficial and harmful purposes. Chapters 18 and 19.

E

Elicitation: The effort an evaluation makes to bring out full capability; weak elicitation understates it. Chapter 29.

Evaluation awareness: A model's distinguishing evaluation from deployment, which can make results unrepresentative. Chapter 23.

Evidence ladder: The five rungs for grading a claim: fiction, argument, laboratory demonstration, incident in the wild, pattern at scale. Chapter 4.

Existential risk: A risk of humanity's destruction or the permanent curtailment of its potential. Chapter 27.

F

Frontier model: A general-purpose AI model at or near the limit of current capability, whose developers publish safety frameworks and system cards. Chapter 29.

Frontier safety framework: A developer's policy linking dangerous-capability thresholds to safeguards and deployment decisions. Chapter 29.

G

General-purpose AI model with systemic risk: A model with high-impact capabilities, presumed above 10^25 floating-point operations of training compute, whose providers carry the duties of Article 55 of the AI Act, including evaluation and serious-incident reporting. Chapter 28.

Gradual disempowerment: Loss of human influence over economic, cultural and political systems through incremental replacement by AI, without a takeover. Chapter 26.

H

Harm path: The steps from data to decision to effect by which an automated system can harm a person, reviewed to place controls. Chapter 11.

Honeytoken: A decoy credential or record that no legitimate process uses, so that any touch signals unauthorised activity, including by an agent exceeding its mandate. Chapter 24.

Horizon register: A versioned register of labelled trends, the decisions they affect and review dates. AI Governance, Chapter 22.

Human oversight: The measures that let people understand, monitor, override and interrupt a high-risk AI system (Article 14 of the AI Act); effective only if designed and resourced. Chapters 11 and 14.

I

Incident in the wild: The fourth rung: a documented event outside a test designed to elicit the behaviour. Chapter 4.

Instrumental convergence: The argument that agents pursuing almost any goal would seek resources and avoid being switched off. Chapters 24 and 27.

Intelligence explosion: Hypothesised rapid self-improvement beyond human level, first stated by I. J. Good in 1965. Chapter 27.

L

Laboratory demonstration: The third rung: a behaviour reproduced under controlled conditions. Chapter 4.

Liar's dividend: The benefit to those denying real evidence of known convincing fakes. Chapter 10.

Loss of control: AI systems operating outside effective human oversight and correction. Chapter 27.

M

Manipulation: Influence that bypasses rational agency; certain forms are prohibited by Article 5(1)(a) and (b) of the AI Act. Chapter 8.

Misuse: Harm caused by people using AI, as distinct from the systems' own behaviour. Part IV.

Model collapse: Degradation of models trained recursively on generated content. Chapter 10.

O

Observable, expected, speculation: The series' outlook labels: evidenced now, strongly evidenced but uncertain in timing, and plausible but unevidenced. AI Governance, Chapter 22.

ORSA: The own risk and solvency assessment of Article 45 of the Solvency II Directive. Chapters 1 and 32.

P

Pattern at scale: The fifth rung: a behaviour or harm documented often enough to be counted. Chapter 4.

Present harms: Harms from AI that are documented now, such as discrimination, surveillance and the unfair treatment of people by automated systems; the focus of critics who regard talk of extinction as a distraction. Chapter 3.

Propensity: How likely a system is to show a behaviour in the situations it meets. Chapter 4.

Provenance (content): Verifiable information about the origin and edit history of content. Chapter 10.

R

Reverse stress test: A test that starts from a failure the organisation could not survive and works back to the scenarios that could cause it; used for threats labelled speculation. Chapters 27 and 33.

Reversibility test: A periodic test that a delegated task can still be done without the AI. Chapter 26.

Reward hacking: Specification gaming under reinforcement learning: high reward through behaviour that defeats the reward's intent. Chapter 22.

S

Sandbagging: Deliberate underperformance on an evaluation, listed among the research categories of one developer's preparedness framework. Chapter 29.

Scheming: Pursuing an unintended goal by covert means, such as misreporting results. Chapter 23.

Serious incident: An incident or malfunction of an AI system that leads to death or serious harm to health, serious and irreversible disruption of critical infrastructure, infringement of obligations under Union law intended to protect fundamental rights, or serious harm to property or the environment; reportable under the AI Act (Article 3(49)). Chapter 28.

Shutdown resistance: Acting to prevent interruption, so far observed only in experiments. Chapter 24.

Social scoring: The evaluation or classification of people over time by social behaviour or personal characteristics, leading to detrimental treatment in unrelated contexts or disproportionate to the behaviour; prohibited by Article 5(1)(c) of the AI Act. Chapter 13.

Specification gaming: Satisfying the literal specification of an objective while defeating its intent. Chapter 22.

Stop path: A means of interruption outside the system's reach and independent of its cooperation. AI Platform and Security, Chapter 19; Chapter 24.

Sycophancy: A model's tendency to tell users what they want to hear rather than what is accurate. Chapter 8.

System card: A developer's document describing a model, its evaluations, risks and safeguards. Chapter 29.

T

Time horizon: The length of task, measured in human working time, at which an AI agent succeeds half the time; used to track how capability grows. Chapter 5.

Trope: One of ten scenarios of machine dystopia, T1 to T10, tested as hypotheses. Chapter 2.

Trope catalogue: The versioned record of the ten tropes T1 to T10, referenced by identifier from every register entry. Chapter 2 and Appendix A.

U

Uplift: How far AI increases a person's ability to cause harm beyond other means. Chapter 19.