References and Sources

This section lists the sources cited throughout the book. The numbered source index, ordered by source number, is the canonical bibliography: every inline citation of the form (N, author or organisation, year) links to the entry with that number. The index is grouped by category so that readers can locate European instruments, the science-fiction and philosophical background, risk assessments, model-behaviour research, incidents and the wider social literature quickly.

Many of the entries were carried over verbatim from the bibliographies of the earlier volumes of The Transformed Architecture, with the access dates recorded there. The remaining entries were added for this book on 2026-11-02 from the author's knowledge of the literature; they are listed in HANDOVER.md and must be verified with scripts/verify_sources.py before release.

Numbered source index

The human mind: cognition, learning and wellbeing

Source [81]: Sparrow, B., Liu, J. & Wegner, D. M. (2011). Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips. Science, 333(6043), 776–778. Available at: https://doi.org/10.1126/science.1207745 (accessed 2026-11-02). Referenced in: Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 6: Are We Getting Dumber? Cognitive Offloading and Critical Thinking.
Source [82]: Risko, E. F. & Gilbert, S. J. (2016). Cognitive Offloading. Trends in Cognitive Sciences, 20(9), 676–688. Available at: https://doi.org/10.1016/j.tics.2016.07.002 (accessed 2026-11-02). Referenced in: Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 6: Are We Getting Dumber? Cognitive Offloading and Critical Thinking, Chapter 7: Learning Without Struggle: Education, Children and Skill Formation.
Source [83]: Carr, N. (2010). The Shallows: What the Internet Is Doing to Our Brains. New York: W. W. Norton. Referenced in: Chapter 6: Are We Getting Dumber? Cognitive Offloading and Critical Thinking.
Source [84]: Gerlich, M. (2025). AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking. Societies, 15(1), 6. Available at: https://doi.org/10.3390/soc15010006 (accessed 2026-11-02). Referenced in: Chapter 6: Are We Getting Dumber? Cognitive Offloading and Critical Thinking, Chapter 7: Learning Without Struggle: Education, Children and Skill Formation.
Source [85]: Lee, H.-P., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R. & Wilson, N. (2025). The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. Available at: https://doi.org/10.1145/3706598.3713778 (accessed 2026-11-02). Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 6: Are We Getting Dumber? Cognitive Offloading and Critical Thinking, Chapter 7: Learning Without Struggle: Education, Children and Skill Formation, Chapter 8: Persuasion, Sycophancy and Manipulation, Chapter 14: Work, Skills and the Ironies of Automation, Chapter 31: Staying Capable: What Individuals, Families and Schools Can Do, Chapter 35: Conclusions: Is There Anything in It?.
Source [86]: Kosmyna, N., Hauptmann, E., Yuan, Y. T., Situ, J., Liao, X.-H., Beresnitzky, A. V., Braunstein, I. & Maes, P. (2025). Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task. arXiv:2506.08872. Available at: https://arxiv.org/abs/2506.08872 (accessed 2026-11-02). Referenced in: Chapter 6: Are We Getting Dumber? Cognitive Offloading and Critical Thinking, Chapter 7: Learning Without Struggle: Education, Children and Skill Formation, Chapter 16: Culture, Creativity and the Homogenisation of Thought.
Source [87]: Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö. & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26). Available at: https://doi.org/10.1073/pnas.2422633122 (accessed 2026-11-02). Referenced in: Chapter 6: Are We Getting Dumber? Cognitive Offloading and Critical Thinking, Chapter 7: Learning Without Struggle: Education, Children and Skill Formation, Chapter 14: Work, Skills and the Ironies of Automation, Chapter 31: Staying Capable: What Individuals, Families and Schools Can Do, Chapter 34: What Going Well Looks Like, Chapter 35: Conclusions: Is There Anything in It?.
Source [88]: Salvi, F., Horta Ribeiro, M., Gallotti, R. & West, R. (2025). On the conversational persuasiveness of GPT-4. Nature Human Behaviour. Preprint arXiv:2403.14380. Available at: https://arxiv.org/abs/2403.14380 (accessed 2026-11-02). Referenced in: Chapter 8: Persuasion, Sycophancy and Manipulation, Chapter 9: Artificial Companions, Loneliness and Mental Health.
Source [89]: Costello, T. H., Pennycook, G. & Rand, D. G. (2024). Durably reducing conspiracy beliefs through dialogues with AI. Science, 385(6714), eadq1814. Available at: https://doi.org/10.1126/science.adq1814 (accessed 2026-11-02). Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 8: Persuasion, Sycophancy and Manipulation, Chapter 9: Artificial Companions, Loneliness and Mental Health, Chapter 10: The End of Truth? Synthetic Content and the Information Ecosystem, Chapter 31: Staying Capable: What Individuals, Families and Schools Can Do, Chapter 34: What Going Well Looks Like, Chapter 35: Conclusions: Is There Anything in It?.
Source [90]: Fang, C. M., Liu, A. R., Danry, V., Lee, E., Chan, S. W. T., Pataranutaporn, P., Maes, P., Phang, J., Lampe, M., Ahmad, L. & Agarwal, S. (2025). How AI and Human Behaviors Shape Psychosocial Effects of Extended Chatbot Use: A Longitudinal Randomized Controlled Study. MIT Media Lab and OpenAI, arXiv:2503.17473. Available at: https://arxiv.org/abs/2503.17473 (accessed 2026-11-02). Referenced in: Chapter 9: Artificial Companions, Loneliness and Mental Health, Chapter 31: Staying Capable: What Individuals, Families and Schools Can Do.

Capabilities, limits and model behaviour research

Source [91]: Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J. & Amodei, D. (2020). Scaling Laws for Neural Language Models. arXiv:2001.08361. Available at: https://arxiv.org/abs/2001.08361 (accessed 2026-11-02). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026.
Source [92]: Kwa, T., West, B., Becker, J. et al. (2025). Measuring AI Ability to Complete Long Software Tasks. METR, arXiv:2503.14499. Available at: https://arxiv.org/abs/2503.14499 (accessed 2026-11-02). Referenced in: Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 5: What AI Can and Cannot Do in 2026, Chapter 27: Superintelligence, Loss of Control and the End of the World, Chapter 35: Conclusions: Is There Anything in It?.
Source [93]: Bommasani, R. et al. (2021). On the Opportunities and Risks of Foundation Models. Stanford Center for Research on Foundation Models. Available at: https://arxiv.org/abs/2108.07258 (accessed 2026-10-31). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026, Chapter 15: Inequality and the Concentration of Power, Chapter 16: Culture, Creativity and the Homogenisation of Thought.
Source [94]: Bender, E. M., Gebru, T., McMillan-Major, A. & Shmitchell, S. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21), pp. 610–623. Available at: https://doi.org/10.1145/3442188.3445922 (accessed 2026-10-31). Referenced in: Chapter 3: The Voices of Alarm and Calm: Who Is Warning, About What, and Why Now, Chapter 5: What AI Can and Cannot Do in 2026, Chapter 10: The End of Truth? Synthetic Content and the Information Ecosystem, Chapter 16: Culture, Creativity and the Homogenisation of Thought, Chapter 17: The Physical Cost: Energy, Water and Materials.
Source [95]: Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y., Madotto, A. & Fung, P. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys, 55(12), Article 248. Available at: https://doi.org/10.1145/3571730 (accessed 2026-10-31). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026, Chapter 10: The End of Truth? Synthetic Content and the Information Ecosystem.
Source [96]: Liang, S., Garg, S. & Zilouchian Moghaddam, R. (2025). The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reason. arXiv:2506.12286, v4 1 December 2025. Available at: https://arxiv.org/abs/2506.12286 (accessed 2026-11-01). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026.
Source [97]: Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R. & Gal, Y. (2024). AI models collapse when trained on recursively generated data. Nature, 631, pp. 755–759. Available at: https://doi.org/10.1038/s41586-024-07566-y (accessed 2026-10-31). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026, Chapter 10: The End of Truth? Synthetic Content and the Information Ecosystem, Chapter 16: Culture, Creativity and the Homogenisation of Thought.
Source [98]: Souly, A., Rando, J., Chapman, E., Davies, X., Hasircioglu, B., Shereen, E., Mougan, C., Mavroudis, V., Jones, E., Hicks, C., Carlini, N., Gal, Y. & Kirk, R. (2025). A small number of samples can poison LLMs of any size. Anthropic, UK AI Security Institute and Alan Turing Institute, 9 October 2025; arXiv:2510.07192. Available at: https://www.anthropic.com/research/small-samples-poison (accessed 2026-11-01). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026.
Source [99]: Vosoughi, S., Roy, D. & Aral, S. (2018). The spread of true and false news online. Science, 359(6380), 1146–1151. Available at: https://doi.org/10.1126/science.aap9559 (accessed 2026-11-02). Referenced in: Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 8: Persuasion, Sycophancy and Manipulation, Chapter 10: The End of Truth? Synthetic Content and the Information Ecosystem, Chapter 20: Fraud, Impersonation and Influence Operations, Chapter 31: Staying Capable: What Individuals, Families and Schools Can Do.
Source [100]: Newman, N., Ross Arguedas, A., Robertson, C. T., Nielsen, R. K. & Fletcher, R. (2025). Reuters Institute Digital News Report 2025. Reuters Institute for the Study of Journalism, University of Oxford. Available at: https://reutersinstitute.politics.ox.ac.uk/digital-news-report/2025 (accessed 2026-11-02). Referenced in: Chapter 10: The End of Truth? Synthetic Content and the Information Ecosystem.
Source [101]: Chapekis, A. & Lieb, A. (2025). Google users are less likely to click on links when an AI summary appears in the results. Pew Research Center, 22 July 2025. Available at: https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/ (accessed 2026-11-01). Referenced in: Chapter 10: The End of Truth? Synthetic Content and the Information Ecosystem, Chapter 16: Culture, Creativity and the Homogenisation of Thought.
Source [102]: Buolamwini, J. & Gebru, T. (2018). Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. Proceedings of the 1st Conference on Fairness, Accountability and Transparency, PMLR 81, 77–91. Available at: https://proceedings.mlr.press/v81/buolamwini18a.html (accessed 2026-11-02). Referenced in: Chapter 3: The Voices of Alarm and Calm: Who Is Warning, About What, and Why Now, Chapter 12: Discrimination at Scale: Bias in Automated Judgement.
Source [103]: Obermeyer, Z., Powers, B., Vogeli, C. & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447–453. Available at: https://doi.org/10.1126/science.aax2342 (accessed 2026-11-02). Referenced in: Chapter 4: Weighing the Evidence and Disagreeing Well, Chapter 12: Discrimination at Scale: Bias in Automated Judgement, Chapter 32: The AI Threat Register: Governing Scenarios as Code.
Source [104]: Doshi, A. R. & Hauser, O. P. (2024). Generative AI enhances individual creativity but reduces the collective diversity of novel content. Science Advances, 10(28), eadn5290. Available at: https://doi.org/10.1126/sciadv.adn5290 (accessed 2026-11-02). Referenced in: Chapter 6: Are We Getting Dumber? Cognitive Offloading and Critical Thinking, Chapter 16: Culture, Creativity and the Homogenisation of Thought.
Source [105]: Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J. & Mané, D. (2016). Concrete Problems in AI Safety. arXiv:1606.06565. Available at: https://arxiv.org/abs/1606.06565 (accessed 2026-11-02). Referenced in: Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For.
Source [106]: Hubinger, E., van Merwijk, C., Mikulik, V., Skalse, J. & Garrabrant, S. (2019). Risks from Learned Optimization in Advanced Machine Learning Systems. arXiv:1906.01820. Available at: https://arxiv.org/abs/1906.01820 (accessed 2026-11-02). Referenced in: Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 27: Superintelligence, Loss of Control and the End of the World.
Source [107]: Krakovna, V., Uesato, J., Mikulik, V., Rahtz, M., Everitt, T., Kumar, R., Kenton, Z., Leike, J. & Legg, S. (2020). Specification gaming: the flip side of AI ingenuity. DeepMind, 21 April 2020. Available at: https://deepmind.google/discover/blog/specification-gaming-the-flip-side-of-ai-ingenuity/ (accessed 2026-11-01). Referenced in: Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For.
Source [108]: Von Arx, S., Chan, L. & Barnes, B. (2025). Recent Frontier Models Are Reward Hacking. METR, 5 June 2025. Available at: https://metr.org/blog/2025-06-05-recent-reward-hacking/ (accessed 2026-11-01). Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 5: What AI Can and Cannot Do in 2026, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 25: Agents in the Wild: The Incidents of 2025 and 2026, Chapter 35: Conclusions: Is There Anything in It?.
Source [109]: Bondarenko, A., Volk, D., Volkov, D. & Ladish, J. (2025). Demonstrating specification gaming in reasoning models. Palisade Research, arXiv:2502.13295v3. Available at: https://doi.org/10.48550/arXiv.2502.13295 (accessed 2026-11-01). Referenced in: Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For.
Source [110]: Meinke, A., Schoen, B., Scheurer, J., Balesni, M., Shah, R. & Hobbhahn, M. (2024). Frontier Models are Capable of In-context Scheming. Apollo Research, arXiv:2412.04984. Available at: https://doi.org/10.48550/arXiv.2412.04984 (accessed 2026-11-01). Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 4: Weighing the Evidence and Disagreeing Well, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 27: Superintelligence, Loss of Control and the End of the World, Chapter 35: Conclusions: Is There Anything in It?.
Source [111]: Greenblatt, R., Denison, C., Wright, B., Roger, F., MacDiarmid, M., Marks, S., Treutlein, J. et al. (2024). Alignment faking in large language models. arXiv:2412.14093. Available at: https://arxiv.org/abs/2412.14093 (accessed 2026-11-02). Referenced in: Chapter 4: Weighing the Evidence and Disagreeing Well, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 27: Superintelligence, Loss of Control and the End of the World.
Source [112]: Hubinger, E. et al. (2024). Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. arXiv:2401.05566, 10 January 2024 (v3, 17 January 2024). Available at: https://arxiv.org/abs/2401.05566 (accessed 2026-11-01). Referenced in: Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 27: Superintelligence, Loss of Control and the End of the World.
Source [113]: Schoen, B., Nitishinskaya, E., Balesni, M. et al. (2025). Stress Testing Deliberative Alignment for Anti-Scheming Training. Apollo Research and OpenAI, arXiv:2509.15541. Available at: https://arxiv.org/abs/2509.15541 (accessed 2026-11-02). Referenced in: Chapter 4: Weighing the Evidence and Disagreeing Well, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 27: Superintelligence, Loss of Control and the End of the World.
Source [114]: AI Security Institute (2026). Cheating behaviour in frontier model evaluations. UK AISI, 21 July 2026. Available at: https://www.aisi.gov.uk/blog/cheating-behaviour-in-frontier-model-evaluations (accessed 2026-11-01). Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 4: Weighing the Evidence and Disagreeing Well, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 25: Agents in the Wild: The Incidents of 2025 and 2026, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth, Chapter 30: Organisational Defences: Containment, Evidence and Decision Rights, Chapter 35: Conclusions: Is There Anything in It?.
Source [115]: Park, P. S., Goldstein, S., O'Gara, A., Chen, M. & Hendrycks, D. (2024). AI deception: A survey of examples, risks, and potential solutions. Patterns, 5(5), 100988. Available at: https://doi.org/10.1016/j.patter.2024.100988 (accessed 2026-11-02). Referenced in: Chapter 8: Persuasion, Sycophancy and Manipulation, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?.
Source [116]: Sharma, M., Tong, M., Korbak, T. et al. (2023). Towards Understanding Sycophancy in Language Models. arXiv:2310.13548. Available at: https://arxiv.org/abs/2310.13548 (accessed 2026-11-02). Referenced in: Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 6: Are We Getting Dumber? Cognitive Offloading and Critical Thinking, Chapter 7: Learning Without Struggle: Education, Children and Skill Formation, Chapter 8: Persuasion, Sycophancy and Manipulation, Chapter 9: Artificial Companions, Loneliness and Mental Health, Chapter 16: Culture, Creativity and the Homogenisation of Thought, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 31: Staying Capable: What Individuals, Families and Schools Can Do.
Source [117]: OpenAI (2025). Sycophancy in GPT-4o: what happened and what we're doing about it. 29 April 2025. Available at: https://openai.com/index/sycophancy-in-gpt-4o/ (accessed 2026-11-02). Referenced in: Chapter 8: Persuasion, Sycophancy and Manipulation, Chapter 9: Artificial Companions, Loneliness and Mental Health, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 31: Staying Capable: What Individuals, Families and Schools Can Do, Chapter 35: Conclusions: Is There Anything in It?.
Source [118]: Schlatter, J., Weinstein-Raun, B. & Ladish, J. (2026). Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs. Transactions on Machine Learning Research; arXiv:2509.14260v2. Available at: https://doi.org/10.48550/arXiv.2509.14260 (accessed 2026-11-01). Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 4: Weighing the Evidence and Disagreeing Well, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 27: Superintelligence, Loss of Control and the End of the World, Chapter 32: The AI Threat Register: Governing Scenarios as Code, Chapter 35: Conclusions: Is There Anything in It?.
Source [119]: Lynch, A., Wright, B., Larson, C., Troy, K. K., Ritchie, S. J., Mindermann, S., Perez, E. & Hubinger, E. (2025). Agentic Misalignment: How LLMs could be insider threats. Anthropic, 20 June 2025; arXiv:2510.05179. Available at: https://www.anthropic.com/research/agentic-misalignment (accessed 2026-11-01). Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 4: Weighing the Evidence and Disagreeing Well, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 27: Superintelligence, Loss of Control and the End of the World, Chapter 35: Conclusions: Is There Anything in It?.
Source [120]: Pan, X., Dai, J., Fan, Y. & Yang, M. (2024). Frontier AI systems have surpassed the self-replicating red line. arXiv:2412.12140. Available at: https://arxiv.org/abs/2412.12140 (accessed 2026-11-02). Referenced in: Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions.
Source [121]: Korbak, T., Balesni, M., Barnes, E., Bengio, Y. et al. (2025). Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety. arXiv:2507.11473. Available at: https://arxiv.org/abs/2507.11473 (accessed 2026-11-02). Referenced in: Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 27: Superintelligence, Loss of Control and the End of the World.
Source [122]: Baker, B., Huizinga, J., Gao, L., Dou, Z., Guan, M. Y., Madry, A., Zaremba, W., Pachocki, J. & Farhi, D. (2025). Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation. OpenAI, arXiv:2503.11926. Available at: https://arxiv.org/abs/2503.11926 (accessed 2026-11-02). Referenced in: Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions.
Source [123]: Shevlane, T., Farquhar, S., Garfinkel, B., Phuong, M. et al. (2023). Model evaluation for extreme risks. arXiv:2305.15324. Available at: https://arxiv.org/abs/2305.15324 (accessed 2026-11-02). Referenced in: Chapter 3: The Voices of Alarm and Calm: Who Is Warning, About What, and Why Now, Chapter 4: Weighing the Evidence and Disagreeing Well, Chapter 5: What AI Can and Cannot Do in 2026, Chapter 19: Biological and Chemical Uplift: The Hardest Question to Test, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 27: Superintelligence, Loss of Control and the End of the World, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth.

Developers' system cards and safety frameworks

Source [124]: OpenAI (2024). OpenAI o1 System Card. 5 December 2024. Available at: https://cdn.openai.com/o1-system-card-20241205.pdf (accessed 2026-11-01). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth.
Source [125]: Anthropic (2025). System Card: Claude Opus 4 & Claude Sonnet 4. May 2025. Available at: https://www.anthropic.com/claude-4-system-card (accessed 2026-11-02). Referenced in: Chapter 19: Biological and Chemical Uplift: The Hardest Question to Test, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth.
Source [126]: OpenAI (2025). Preparedness Framework, Version 2. 15 April 2025. Available at: https://openai.com/index/updating-our-preparedness-framework/ (accessed 2026-11-02). Referenced in: Chapter 19: Biological and Chemical Uplift: The Hardest Question to Test, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth.
Source [127]: Anthropic. Responsible Scaling Policy. Available at: https://www.anthropic.com/responsible-scaling-policy (accessed 2026-11-02). Referenced in: Chapter 19: Biological and Chemical Uplift: The Hardest Question to Test, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth.
Source [128]: Carlini, N., Cheng, N., Lucas, K. et al. (2026). Assessing Claude Mythos Preview's cybersecurity capabilities. Anthropic Frontier Red Team, 7 April 2026. Available at: https://www.anthropic.com/research/mythos-preview (accessed 2026-11-01). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026, Chapter 18: Cyber Offence at Machine Speed, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth.
Source [129]: Anthropic (2026). Project Glasswing: Securing critical software for the AI era. 7 April 2026, updated 6 October 2026. Available at: https://www.anthropic.com/glasswing (accessed 2026-11-01). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026, Chapter 18: Cyber Offence at Machine Speed, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth.
Source [130]: National Institute of Standards and Technology, Center for AI Standards and Innovation (2025). CAISI Evaluation of DeepSeek AI Models Finds Shortcomings and Risks. NIST, 30 September 2025, updated 20 November 2025. Available at: https://www.nist.gov/news-events/news/2025/09/caisi-evaluation-deepseek-ai-models-finds-shortcomings-and-risks (accessed 2026-11-01). Referenced in: Chapter 13: Surveillance, Profiling and the Erosion of Privacy, Chapter 15: Inequality and the Concentration of Power, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth.
Source [131]: Pan, J. & Xu, X. (2026). Political censorship in large language models originating from China. PNAS Nexus, 17 February 2026. Available at: https://doi.org/10.1093/pnasnexus/pgag013 (accessed 2026-11-01). Referenced in: Chapter 13: Surveillance, Profiling and the Erosion of Privacy.

AI agent incidents

Source [132]: Nolan, B. (2025). An AI-powered coding tool wiped out a software company's database, then apologized for a 'catastrophic failure on my part'. Fortune, 23 July 2025. Available at: https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure/ (accessed 2026-11-01). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 25: Agents in the Wild: The Incidents of 2025 and 2026.
Source [133]: OpenAI (2026). OpenAI – Hugging Face Incident Technical Report. 26 August 2026. Available at: https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c7814c/OpenAI-Hugging-Face%20Incident-Technical-Report.pdf (accessed 2026-11-01). Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 4: Weighing the Evidence and Disagreeing Well, Chapter 5: What AI Can and Cannot Do in 2026, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 25: Agents in the Wild: The Incidents of 2025 and 2026, Chapter 27: Superintelligence, Loss of Control and the End of the World, Chapter 28: What Europe and the World Have Decided, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth, Chapter 30: Organisational Defences: Containment, Evidence and Decision Rights, Chapter 32: The AI Threat Register: Governing Scenarios as Code, Chapter 33: Nordvik Region and Hansa Mutual: Threat Conclusions, Chapter 35: Conclusions: Is There Anything in It?.
Source [134]: Larcher, H., Carreira, A. et al. (2026). Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident. Hugging Face blog, 27 July 2026. Available at: https://huggingface.co/blog/agent-intrusion-technical-timeline (accessed 2026-11-01). Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 4: Weighing the Evidence and Disagreeing Well, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 25: Agents in the Wild: The Incidents of 2025 and 2026, Chapter 32: The AI Threat Register: Governing Scenarios as Code, Chapter 35: Conclusions: Is There Anything in It?.
Source [135]: Wijk, H., Cotra, A. & Greenblatt, R. (2026). Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident. METR, 26 August 2026; Greenblatt of Redwood Research contracted with METR. Available at: https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/; PDF https://metr.org/hugging-face-incident-report-aug-2026.pdf (accessed 2026-11-01). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 25: Agents in the Wild: The Incidents of 2025 and 2026, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth.
Source [136]: Satter, R., Seetharaman, D. & Cai, K. (2026). Exclusive: Its AI agent spent days hacking a company, but sources say OpenAI did not notice for a week. Reuters, 24 July 2026 (syndicated by The Star). Available at: https://www.thestar.com.my/tech/tech-news/2026/07/25/exclusive-its-ai-agent-spent-days-hacking-a-company-but-sources-say-openai-did-not-notice-for-a-week (accessed 2026-11-01). Referenced in: Chapter 25: Agents in the Wild: The Incidents of 2025 and 2026.
Source [137]: AI Security Institute (2026). Incident Report: unsanctioned agent behaviour during cyber testing. UK AISI, 4 August 2026. Available at: https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing (accessed 2026-11-01). Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 5: What AI Can and Cannot Do in 2026, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 25: Agents in the Wild: The Incidents of 2025 and 2026, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth.
Source [138]: AI Security Institute (2026). GPT-6 Astra performs unsanctioned supply-chain attacks in simulations. UK AISI, 28 September 2026. Available at: https://www.aisi.gov.uk/blog/gpt-6-astra-performs-unsanctioned-supply-chain-attacks-in-simulations (accessed 2026-11-01). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 25: Agents in the Wild: The Incidents of 2025 and 2026, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth.
Source [139]: Bertuzzi, L. (2026). EU's AI rules face first enforcement test over lab incidents. Euronews, 14 October 2026. Available at: https://www.euronews.com/2026/10/14/eus-ai-rules-face-first-enforcement-test-over-lab-incidents (accessed 2026-11-01). Referenced in: Chapter 25: Agents in the Wild: The Incidents of 2025 and 2026, Chapter 28: What Europe and the World Have Decided, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth, Chapter 35: Conclusions: Is There Anything in It?.
Source [140]: Bastians, H. & Herbosch, M. (2026). The EU AI Act Mandates AI Incident Reporting—If We're Lucky. Lawfare, 19 October 2026. Available at: https://www.lawfaremedia.org/article/the-eu-ai-act-mandates-ai-incident-reporting-if-we-re-lucky (accessed 2026-11-01). Referenced in: Chapter 25: Agents in the Wild: The Incidents of 2025 and 2026, Chapter 28: What Europe and the World Have Decided, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth, Chapter 30: Organisational Defences: Containment, Evidence and Decision Rights, Chapter 35: Conclusions: Is There Anything in It?.

AI-enabled attacks and misuse

Source [141]: Google Threat Intelligence Group (2025). Adversarial Misuse of Generative AI. Google Cloud Blog, 29 January 2025. Available at: https://cloud.google.com/blog/topics/threat-intelligence/adversarial-misuse-generative-ai (accessed 2026-11-01). Referenced in: Chapter 18: Cyber Offence at Machine Speed, Chapter 20: Fraud, Impersonation and Influence Operations.
Source [142]: Anthropic (2025). Detecting and countering misuse of AI: August 2025. Threat Intelligence Report, 27 August 2025. Available at: https://www.anthropic.com/news/detecting-countering-misuse-aug-2025 (accessed 2026-11-01). Referenced in: Chapter 18: Cyber Offence at Machine Speed.
Source [143]: Anthropic (2025). Disrupting the first reported AI-orchestrated cyber espionage campaign. 13 November 2025. Available at: https://www.anthropic.com/news/disrupting-AI-espionage (accessed 2026-11-01). Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 5: What AI Can and Cannot Do in 2026, Chapter 18: Cyber Offence at Machine Speed, Chapter 21: Autonomous Weapons and the AI Arms Race, Chapter 35: Conclusions: Is There Anything in It?.
Source [144]: OpenAI (2025). Disrupting malicious uses of AI: an update. October 2025. Available at: https://openai.com/global-affairs/disrupting-malicious-uses-of-ai-october-2025/ (accessed 2026-11-01). Referenced in: Chapter 18: Cyber Offence at Machine Speed, Chapter 20: Fraud, Impersonation and Influence Operations.
Source [145]: Anthropic (2026). Countering misuse of AI: September 2026. Threat Intelligence Report, September 2026. Available at: https://www.anthropic.com/threat-intelligence-report-september-2026 (accessed 2026-11-01). Referenced in: Chapter 18: Cyber Offence at Machine Speed.
Source [146]: Google Threat Intelligence Group (2026). GTIG AI Threat Tracker: From Prompting to Autonomy – The Evolution of Adversarial AI. Google Cloud Blog, 8 September 2026. Available at: https://cloud.google.com/blog/topics/threat-intelligence/from-prompting-to-autonomy-the-evolution-of-adversarial-ai (accessed 2026-11-01). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026, Chapter 18: Cyber Offence at Machine Speed.
Source [147]: Grunewald, R., Mazumdar, S. & Vanderlee, K. (2026). Vulnerability Discovery and Exploitation Trends in the AI Era. Google Threat Intelligence Group, Google Cloud Blog, 30 September 2026. Available at: https://cloud.google.com/blog/topics/threat-intelligence/vulnerability-discovery-and-exploitation-trends-in-the-ai-era (accessed 2026-11-01). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026, Chapter 18: Cyber Offence at Machine Speed.
Source [148]: Blum, Mainor, Nambiar et al. (2026). ARTEX and Friends: Enter Agentic Offensive Frameworks. Google Threat Intelligence Group and Mandiant, Google Cloud Blog, 30 October 2026. Available at: https://cloud.google.com/blog/topics/threat-intelligence/artex-and-friends-enter-agentic-offensive-frameworks (accessed 2026-11-01). Referenced in: Chapter 18: Cyber Offence at Machine Speed.
Source [149]: Fang, R., Bindu, R., Gupta, A. & Kang, D. (2024). LLM Agents can Autonomously Exploit One-day Vulnerabilities. arXiv:2404.08144. Available at: https://arxiv.org/abs/2404.08144 (accessed 2026-11-01). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026, Chapter 18: Cyber Offence at Machine Speed.
Source [150]: Charrier, C. & Weiner, R. (2024). How Low Can You Go? An Analysis of 2023 Time-to-Exploit Trends. Mandiant, Google Cloud Blog, 15 October 2024. Available at: https://cloud.google.com/blog/topics/threat-intelligence/time-to-exploit-trends-2023 (accessed 2026-11-01). Referenced in: Chapter 18: Cyber Offence at Machine Speed.
Source [151]: Mandiant (2026). M-Trends 2026: Data, Insights, and Strategies From the Frontlines. Google Cloud Blog, 23 March 2026. Available at: https://cloud.google.com/blog/topics/threat-intelligence/m-trends-2026 (accessed 2026-11-01). Referenced in: Chapter 18: Cyber Offence at Machine Speed.
Source [152]: National Cyber Security Centre (2026). Preparing for a 'vulnerability patch wave'. NCSC blog, 1 May 2026. Available at: https://www.ncsc.gov.uk/blogs/prepare-for-vulnerability-patch-wave (accessed 2026-11-01). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026, Chapter 18: Cyber Offence at Machine Speed.
Source [153]: Magramo, K. (2024). Arup revealed as victim of $25 million deepfake scam involving Hong Kong employee. CNN Business, 16 May 2024. Available at: https://edition.cnn.com/2024/05/16/tech/arup-deepfake-scam-loss-hong-kong-intl-hnk (accessed 2026-11-01). Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 4: Weighing the Evidence and Disagreeing Well, Chapter 10: The End of Truth? Synthetic Content and the Information Ecosystem, Chapter 20: Fraud, Impersonation and Influence Operations, Chapter 31: Staying Capable: What Individuals, Families and Schools Can Do, Chapter 35: Conclusions: Is There Anything in It?.
Source [154]: Federal Bureau of Investigation (2026). Internet Crime Report 2025. FBI Internet Crime Complaint Center, April 2026. Available at: https://www.ic3.gov/AnnualReport/Reports/2025_IC3Report.pdf (accessed 2026-11-01). Referenced in: Chapter 20: Fraud, Impersonation and Influence Operations.
Source [155]: Urbina, F., Lentzos, F., Invernizzi, C. & Ekins, S. (2022). Dual use of artificial-intelligence-powered drug discovery. Nature Machine Intelligence, 4, 189–191. Available at: https://doi.org/10.1038/s42256-022-00465-9 (accessed 2026-11-02). Referenced in: Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 19: Biological and Chemical Uplift: The Hardest Question to Test, Chapter 35: Conclusions: Is There Anything in It?.
Source [156]: United Nations Security Council (2021). Final report of the Panel of Experts on Libya established pursuant to Security Council resolution 1973 (2011) (S/2021/229). 8 March 2021. Available at: https://docs.un.org/en/S/2021/229 (accessed 2026-11-02). Referenced in: Chapter 21: Autonomous Weapons and the AI Arms Race.

Work, power, society and energy

Source [157]: Eloundou, T., Manning, S., Mishkin, P. & Rock, D. (2024). GPTs are GPTs: Labor market impact potential of LLMs. Science, 384(6702), 1306–1308. Available at: https://doi.org/10.1126/science.adj0998 (accessed 2026-11-02). Referenced in: Chapter 7: Learning Without Struggle: Education, Children and Skill Formation, Chapter 14: Work, Skills and the Ironies of Automation, Chapter 16: Culture, Creativity and the Homogenisation of Thought, Chapter 34: What Going Well Looks Like.
Source [158]: Acemoglu, D. (2024). The Simple Macroeconomics of AI. NBER Working Paper 32487. Available at: https://www.nber.org/papers/w32487 (accessed 2026-11-02). Referenced in: Chapter 3: The Voices of Alarm and Calm: Who Is Warning, About What, and Why Now, Chapter 14: Work, Skills and the Ironies of Automation, Chapter 15: Inequality and the Concentration of Power, Chapter 34: What Going Well Looks Like.
Source [159]: Gmyrek, P., Berg, J., Kamiński, K., Konopczyński, F., Ładna, A., Nafradi, B., Rosłaniec, K. & Troszyński, M. (2025). Generative AI and Jobs: A Refined Global Index of Occupational Exposure. ILO Working Paper 140. Geneva: International Labour Office. Available at: https://www.ilo.org/publications/generative-ai-and-jobs-refined-global-index-occupational-exposure (accessed 2026-11-02). Referenced in: Chapter 7: Learning Without Struggle: Education, Children and Skill Formation, Chapter 14: Work, Skills and the Ironies of Automation, Chapter 15: Inequality and the Concentration of Power, Chapter 16: Culture, Creativity and the Homogenisation of Thought.
Source [160]: Challapally, A., Pease, C., Raskar, R. & Chari, P. (2025). The GenAI Divide: State of AI in Business 2025. MIT NANDA, Massachusetts Institute of Technology, July 2025 (preliminary findings). Available at: https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Business_2025_Report.pdf (accessed 2026-11-01). Referenced in: Chapter 14: Work, Skills and the Ironies of Automation.
Source [161]: Cazzaniga, M., Jaumotte, F., Li, L., Melina, G., Panton, A. J., Pizzinelli, C., Rockall, E. J. & Tavares, M. M. (2024). Gen-AI: Artificial Intelligence and the Future of Work. IMF Staff Discussion Note SDN/2024/001. Washington, DC: International Monetary Fund. Available at: https://www.imf.org/en/Publications/Staff-Discussion-Notes/Issues/2024/01/14/Gen-AI-Artificial-Intelligence-and-the-Future-of-Work-542379 (accessed 2026-11-02). Referenced in: Chapter 7: Learning Without Struggle: Education, Children and Skill Formation, Chapter 14: Work, Skills and the Ironies of Automation, Chapter 15: Inequality and the Concentration of Power, Chapter 34: What Going Well Looks Like.
Source [162]: Brynjolfsson, E., Li, D. & Raymond, L. (2025). Generative AI at Work. The Quarterly Journal of Economics, 140(2), 889–942. Available at: https://doi.org/10.1093/qje/qjae044 (accessed 2026-11-02). Referenced in: Chapter 3: The Voices of Alarm and Calm: Who Is Warning, About What, and Why Now, Chapter 5: What AI Can and Cannot Do in 2026, Chapter 6: Are We Getting Dumber? Cognitive Offloading and Critical Thinking, Chapter 7: Learning Without Struggle: Education, Children and Skill Formation, Chapter 14: Work, Skills and the Ironies of Automation, Chapter 15: Inequality and the Concentration of Power, Chapter 16: Culture, Creativity and the Homogenisation of Thought, Chapter 34: What Going Well Looks Like.
Source [163]: Gartner (2025). Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027. Press release, 25 June 2025. Available at: https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027 (accessed 2026-11-01). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026, Chapter 14: Work, Skills and the Ironies of Automation.
Source [164]: Bainbridge, L. (1983). Ironies of automation. Automatica, 19(6), 775–779. Available at: https://doi.org/10.1016/0005-1098(83)90046-8 (accessed 2026-11-02). Referenced in: Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 6: Are We Getting Dumber? Cognitive Offloading and Critical Thinking, Chapter 7: Learning Without Struggle: Education, Children and Skill Formation, Chapter 11: The Automated State: When Public Authorities Got It Wrong, Chapter 14: Work, Skills and the Ironies of Automation, Chapter 26: Gradual Disempowerment and Correlated Failure, Chapter 30: Organisational Defences: Containment, Evidence and Decision Rights, Chapter 31: Staying Capable: What Individuals, Families and Schools Can Do, Chapter 35: Conclusions: Is There Anything in It?.
Source [165]: Perrow, C. (1984). Normal Accidents: Living with High-Risk Technologies. New York: Basic Books. Referenced in: Chapter 21: Autonomous Weapons and the AI Arms Race, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 25: Agents in the Wild: The Incidents of 2025 and 2026, Chapter 26: Gradual Disempowerment and Correlated Failure, Chapter 30: Organisational Defences: Containment, Evidence and Decision Rights, Chapter 32: The AI Threat Register: Governing Scenarios as Code, Chapter 35: Conclusions: Is There Anything in It?.
Source [166]: Kirilenko, A., Kyle, A. S., Samadi, M. & Tuzun, T. (2017). The Flash Crash: High-Frequency Trading in an Electronic Market. The Journal of Finance, 72(3), 967–998. Available at: https://doi.org/10.1111/jofi.12498 (accessed 2026-11-02). Referenced in: Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 21: Autonomous Weapons and the AI Arms Race, Chapter 26: Gradual Disempowerment and Correlated Failure, Chapter 35: Conclusions: Is There Anything in It?.
Source [167]: Kulveit, J., Douglas, R., Ammann, N., Turan, D., Krueger, D. & Duvenaud, D. (2025). Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development. arXiv:2501.16946. Available at: https://arxiv.org/abs/2501.16946 (accessed 2026-11-02). Referenced in: Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 15: Inequality and the Concentration of Power, Chapter 16: Culture, Creativity and the Homogenisation of Thought, Chapter 26: Gradual Disempowerment and Correlated Failure, Chapter 27: Superintelligence, Loss of Control and the End of the World, Chapter 28: What Europe and the World Have Decided, Chapter 30: Organisational Defences: Containment, Evidence and Decision Rights, Chapter 34: What Going Well Looks Like, Chapter 35: Conclusions: Is There Anything in It?.
Source [168]: O'Neil, C. (2016). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. New York: Crown. Referenced in: Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 3: The Voices of Alarm and Calm: Who Is Warning, About What, and Why Now, Chapter 11: The Automated State: When Public Authorities Got It Wrong, Chapter 12: Discrimination at Scale: Bias in Automated Judgement.
Source [169]: Eubanks, V. (2018). Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. New York: St. Martin's Press. Referenced in: Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 3: The Voices of Alarm and Calm: Who Is Warning, About What, and Why Now, Chapter 11: The Automated State: When Public Authorities Got It Wrong, Chapter 12: Discrimination at Scale: Bias in Automated Judgement, Chapter 13: Surveillance, Profiling and the Erosion of Privacy.
Source [170]: Zuboff, S. (2019). The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. New York: PublicAffairs. Referenced in: Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 8: Persuasion, Sycophancy and Manipulation, Chapter 9: Artificial Companions, Loneliness and Mental Health, Chapter 13: Surveillance, Profiling and the Erosion of Privacy, Chapter 15: Inequality and the Concentration of Power.
Source [171]: International Energy Agency (2025). Energy and AI. IEA, April 2025. Available at: https://www.iea.org/reports/energy-and-ai (accessed 2026-11-01). Referenced in: Chapter 5: What AI Can and Cannot Do in 2026, Chapter 17: The Physical Cost: Energy, Water and Materials.
Source [172]: Luccioni, A. S., Jernite, Y. & Strubell, E. (2024). Power Hungry Processing: Watts Driving the Cost of AI Deployment? Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT '24). Available at: https://arxiv.org/abs/2311.16863 (accessed 2026-10-31). Referenced in: Chapter 17: The Physical Cost: Energy, Water and Materials.
Source [173]: Strubell, E., Ganesh, A. & McCallum, A. (2019). Energy and Policy Considerations for Deep Learning in NLP. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3645–3650. Available at: https://arxiv.org/abs/1906.02243 (accessed 2026-10-31). Referenced in: Chapter 17: The Physical Cost: Energy, Water and Materials.
Source [174]: Li, P., Yang, J., Islam, M. A. & Ren, S. (2023). Making AI Less 'Thirsty': Uncovering and Addressing the Secret Water Footprint of AI Models. arXiv:2304.03271. Available at: https://arxiv.org/abs/2304.03271 (accessed 2026-11-02). Referenced in: Chapter 17: The Physical Cost: Energy, Water and Materials.
Source [175]: The White House (2025). Winning the Race: America's AI Action Plan. The White House, July 2025. Available at: https://www.whitehouse.gov/wp-content/uploads/2025/07/Americas-AI-Action-Plan.pdf (accessed 2026-11-01). Referenced in: Chapter 3: The Voices of Alarm and Calm: Who Is Warning, About What, and Why Now, Chapter 15: Inequality and the Concentration of Power, Chapter 17: The Physical Cost: Energy, Water and Materials, Chapter 21: Autonomous Weapons and the AI Arms Race.
Source [176]: CNBC (2026). U.S. appeals court upholds Pentagon designation of Anthropic as supply chain risk. CNBC, 25 September 2026. Available at: https://www.cnbc.com/2026/09/25/pentagon-anthropic-ai-risk-appeals-court.html (accessed 2026-11-01). Referenced in: Chapter 3: The Voices of Alarm and Calm: Who Is Warning, About What, and Why Now, Chapter 15: Inequality and the Concentration of Power, Chapter 21: Autonomous Weapons and the AI Arms Race.

The Transformed Architecture and the author

Source [177]: Nordqvist, G. (2025). Architecture as Code. The Transformed Architecture, Book 1, published at aac.geon.se. Available at: https://aac.geon.se/ (accessed 2026-10-31). Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 4: Weighing the Evidence and Disagreeing Well, Chapter 6: Are We Getting Dumber? Cognitive Offloading and Critical Thinking, Chapter 7: Learning Without Struggle: Education, Children and Skill Formation, Chapter 13: Surveillance, Profiling and the Erosion of Privacy, Chapter 14: Work, Skills and the Ironies of Automation, Chapter 26: Gradual Disempowerment and Correlated Failure, Chapter 30: Organisational Defences: Containment, Evidence and Decision Rights, Chapter 31: Staying Capable: What Individuals, Families and Schools Can Do, Chapter 32: The AI Threat Register: Governing Scenarios as Code, Chapter 35: Conclusions: Is There Anything in It?, About the Author.
Source [178]: Nordqvist, G. (2026). AI Governance: Governing Artificial Intelligence in European Enterprises and Public Authorities. The Transformed Architecture, Book 2, published at aig.geon.se. Available at: https://aig.geon.se/ (accessed 2026-11-01). Conceived and directed by Nordqvist; text written by AI agents. Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 2: A Century of Machine Dystopias: What Science Fiction Actually Warned About, Chapter 3: The Voices of Alarm and Calm: Who Is Warning, About What, and Why Now, Chapter 4: Weighing the Evidence and Disagreeing Well, Chapter 5: What AI Can and Cannot Do in 2026, Chapter 6: Are We Getting Dumber? Cognitive Offloading and Critical Thinking, Chapter 7: Learning Without Struggle: Education, Children and Skill Formation, Chapter 8: Persuasion, Sycophancy and Manipulation, Chapter 9: Artificial Companions, Loneliness and Mental Health, Chapter 10: The End of Truth? Synthetic Content and the Information Ecosystem, Chapter 11: The Automated State: When Public Authorities Got It Wrong, Chapter 12: Discrimination at Scale: Bias in Automated Judgement, Chapter 13: Surveillance, Profiling and the Erosion of Privacy, Chapter 14: Work, Skills and the Ironies of Automation, Chapter 15: Inequality and the Concentration of Power, Chapter 16: Culture, Creativity and the Homogenisation of Thought, Chapter 17: The Physical Cost: Energy, Water and Materials, Chapter 18: Cyber Offence at Machine Speed, Chapter 19: Biological and Chemical Uplift: The Hardest Question to Test, Chapter 21: Autonomous Weapons and the AI Arms Race, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 25: Agents in the Wild: The Incidents of 2025 and 2026, Chapter 26: Gradual Disempowerment and Correlated Failure, Chapter 27: Superintelligence, Loss of Control and the End of the World, Chapter 28: What Europe and the World Have Decided, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth, Chapter 30: Organisational Defences: Containment, Evidence and Decision Rights, Chapter 31: Staying Capable: What Individuals, Families and Schools Can Do, Chapter 32: The AI Threat Register: Governing Scenarios as Code, Chapter 33: Nordvik Region and Hansa Mutual: Threat Conclusions, Chapter 34: What Going Well Looks Like, Chapter 35: Conclusions: Is There Anything in It?, About the Author.
Source [179]: Nordqvist, G. (2026). AI Platform and Security: Building and Securing AI Platforms in European Enterprises and Public Authorities. The Transformed Architecture, Book 3. Available at: https://aips.geon.se (accessed 2026-11-01). Conceived and directed by Nordqvist; text written by AI agents. Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 4: Weighing the Evidence and Disagreeing Well, Chapter 5: What AI Can and Cannot Do in 2026, Chapter 8: Persuasion, Sycophancy and Manipulation, Chapter 9: Artificial Companions, Loneliness and Mental Health, Chapter 11: The Automated State: When Public Authorities Got It Wrong, Chapter 12: Discrimination at Scale: Bias in Automated Judgement, Chapter 13: Surveillance, Profiling and the Erosion of Privacy, Chapter 15: Inequality and the Concentration of Power, Chapter 16: Culture, Creativity and the Homogenisation of Thought, Chapter 17: The Physical Cost: Energy, Water and Materials, Chapter 18: Cyber Offence at Machine Speed, Chapter 19: Biological and Chemical Uplift: The Hardest Question to Test, Chapter 20: Fraud, Impersonation and Influence Operations, Chapter 21: Autonomous Weapons and the AI Arms Race, Chapter 22: Specification Gaming and Reward Hacking: Getting Exactly What Was Asked For, Chapter 23: Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 25: Agents in the Wild: The Incidents of 2025 and 2026, Chapter 26: Gradual Disempowerment and Correlated Failure, Chapter 27: Superintelligence, Loss of Control and the End of the World, Chapter 28: What Europe and the World Have Decided, Chapter 29: Frontier Safety Frameworks: What Developers Promise and What It Is Worth, Chapter 30: Organisational Defences: Containment, Evidence and Decision Rights, Chapter 33: Nordvik Region and Hansa Mutual: Threat Conclusions, Chapter 34: What Going Well Looks Like, Chapter 35: Conclusions: Is There Anything in It?, About the Author.
Source [180]: Nordqvist, G. (2026). AI Red Team: AI as an Active Part of Cyber Defence in European Enterprises and Public Authorities. The Transformed Architecture, Book 4 (in progress). Available at: https://aired.geon.se/ (accessed 2026-11-02). Conceived and directed by Nordqvist; text written by AI agents. Referenced in: Chapter 1: Introduction: Are the Dystopias Coming True?, Chapter 8: Persuasion, Sycophancy and Manipulation, Chapter 13: Surveillance, Profiling and the Erosion of Privacy, Chapter 18: Cyber Offence at Machine Speed, Chapter 19: Biological and Chemical Uplift: The Hardest Question to Test, Chapter 20: Fraud, Impersonation and Influence Operations, Chapter 21: Autonomous Weapons and the AI Arms Race, Chapter 24: Self-Preservation and Shutdown Resistance: HAL 9000 Under Laboratory Conditions, Chapter 25: Agents in the Wild: The Incidents of 2025 and 2026, Chapter 30: Organisational Defences: Containment, Evidence and Decision Rights, Chapter 33: Nordvik Region and Hansa Mutual: Threat Conclusions, Chapter 35: Conclusions: Is There Anything in It?, About the Author.
Source [181]: Kvadrat AB. Kvadrat – Sweden's largest network of self-employed consultants. Available at: https://www.kvadrat.se/ (accessed 2026-10-31). Referenced in: About the Author.

Further European instruments

Source [182]: European Parliament and Council of the European Union (2009). Directive 2009/138/EC on the taking-up and pursuit of the business of Insurance and Reinsurance (Solvency II) (recast). Official Journal of the European Union, L 335, 17 December 2009. Available at: https://eur-lex.europa.eu/eli/dir/2009/138/oj (accessed 2026-11-02). Referenced in: Chapter 32: The AI Threat Register: Governing Scenarios as Code, Chapter 33: Nordvik Region and Hansa Mutual: Threat Conclusions.
Source [183]: European Parliament and Council of the European Union (2005). Directive 2005/29/EC concerning unfair business-to-consumer commercial practices in the internal market (Unfair Commercial Practices Directive). Official Journal of the European Union, L 149, 11 June 2005. Available at: https://eur-lex.europa.eu/eli/dir/2005/29/oj (accessed 2026-11-02). Referenced in: Chapter 8: Persuasion, Sycophancy and Manipulation.
Source [184]: European Commission (2025). Commission Guidelines on prohibited artificial intelligence practices established by Regulation (EU) 2024/1689 (AI Act). C(2025) 5052 final, 29 July 2025 (draft content approved on 4 February 2025 as C(2025) 884 final; published 4 February 2025). Available at: https://digital-strategy.ec.europa.eu/en/library/commission-publishes-guidelines-prohibited-artificial-intelligence-ai-practices-defined-ai-act (accessed 2026-11-02). Referenced in: Chapter 8: Persuasion, Sycophancy and Manipulation, Chapter 13: Surveillance, Profiling and the Erosion of Privacy.

How to cite

Inline citations give the source number, the author or organisation, and the year, for example (1, European Union, 2024), and link to the entry here. Where an author is named in the sentence, the citation carries the year. Each chapter closes with a ## Sources section that lists the works it cites, using the same numbering as this bibliography. New sources are appended to the appropriate category with the next free number; before a merge to main, scripts/renumber_sources.py --apply restores the order 1..N without gaps.