What Going Well Looks Like

A book that spends thirty-three chapters on threats owes its readers an honest account of the alternative. Dystopias are vivid because they are specific, whereas hopeful futures are described in generalities, and a board that has read about deepfaked executives and agents that left their sandbox can easily conclude that the safest course is to use as little AI as possible. That conclusion has a cost, and it falls on people: patients whose treatment depends on research AI could accelerate, residents waiting for a decision on their benefits. An investigation that weighed only one side of the ledger would not be an investigation.

This chapter therefore states the optimistic visions at full strength and asks under which conditions they are reachable. The conditions that separate hopeful futures from dystopian ones turn out to be the measures the threat chapters recommended for other reasons. This is not a threat chapter and has no debate or verdict on a trope; its practical contribution is a positive-scenario register entry, so that an organisation governs its hopes with the same discipline as its fears. It ends with open questions, among them whether AI systems could themselves come to matter morally.

Figure 34.1: Whether rising capability leads to broadly shared gains or to concentrated power and ordinary harms at scale depends less on capability than on five conditions that people and institutions can influence.

Table 34.1: The four futures and the five conditions.

Strength of the conditions	Future
Strong	Normal technology, well governed: steady and broadly shared gains
Strong	Compressed progress, widely shared: science, health and prosperity
Weak	Ordinary harms at scale: automated injustice, deskilling, eroded trust
Weak	Concentrated power or loss of control

The five conditions: capability that is real and measured; safety and containment that do not depend on intent; distribution of gains decided, not assumed; institutions that can see, correct and stop; human capability kept in practice. The positive scenarios are tracked in the threat register, with indicators for each condition.

The optimistic visions at full strength

Compressed progress

The most detailed optimistic vision from inside a frontier developer is Dario Amodei's essay Machines of Loving Grace, published in October 2024 (66, Amodei, 2024). Amodei, chief executive of Anthropic and a signatory of the 2023 extinction statement (52, CAIS, 2023), wrote it as a counterweight, arguing that without a picture of what is at stake on the positive side, the effort to manage the risks loses its purpose. His central image is a "country of geniuses in a datacenter": many systems, each more capable than the best human experts in most fields, working in parallel and faster than people. From that premise he argues that progress in biology and medicine could be compressed by an order of magnitude, including the possible prevention or cure of most infectious diseases and large progress against cancer and psychiatric illness, and extends the argument to economic development and to democratic governance, while acknowledging that his answers on work and meaning are least certain.

The vision is not naive about its limits. Amodei names the factors besides intelligence that limit progress, such as the speed of physical experiments, the complexity of biological systems and the constraints institutions impose, so his forecast is of what happens when intelligence stops being the bottleneck. Read at full strength, it says that the reason to manage the risks is to reach the benefits rather than to avoid the technology.

The cost of not building

The second element is an argument about omission. A decision not to use a technology also has victims, but they are statistical and invisible: the patient diagnosed later than necessary, the applicant who waited longer than needed. Chapter 19 recorded its sharpest form: the same models that raise concern about misuse also accelerate beneficial biology and medicine (66, Amodei, 2024), and controls drawn too widely would slow life-saving research while doing little to stop a determined state actor.

The argument applies at every scale; Chapter 11's remedy for the automated state, for example, was better automation with reasons logs, not a return to paper files. The International AI Safety Report records the potential benefits of general-purpose AI in science, health and productivity alongside its risks, and notes that the evidence for both is still uneven (50, Bengio et al., 2026). A risk function that records only the downside will tend to recommend inaction, which is rarely recorded as a decision at all.

Reading the vision soberly

Three qualifications belong with the optimistic case. First, Amodei is an interested party: he leads a company whose value depends on the capability he describes, so the essay is a well-argued statement of one position, not a forecast with evidence behind it. Its most ambitious claims sit on the argument rung of the evidence ladder, following from premises about future capability not yet demonstrated.

Second, the pace is contested. Narayanan and Kapoor argue that AI is a normal technology whose effects unfold over decades, because they depend on organisations changing their processes, regulation adapting and people learning to use the tools (60, Narayanan & Kapoor, 2025); on their reading the benefits are real but arrive gradually, as they did for electricity. Daron Acemoglu, applying a task-based model to the evidence available in 2024, estimated that AI would raise total factor productivity in the United States by well under one per cent over ten years (158, Acemoglu, 2024). That estimate may prove too low, but it shows how much distance lies between what a technology could do and what an economy will measure.

Third, compressed progress and broadly shared progress are different claims. Medicine could advance by decades while the advances reach a minority and the firms that own the models capture most of the value. Amodei himself treats distribution as a problem to be solved rather than an automatic outcome.

Power and progress

Whether the hopes are realised for most people depends less on what the technology can do than on who decides how it is used.

What history records

Daron Acemoglu and Simon Johnson's Power and Progress is the most sustained argument that the benefits of technology are not shared automatically (67, Acemoglu & Johnson, 2023). Examining a thousand years of technological change, they conclude that productivity gains have repeatedly been captured by a narrow group for long periods before, and sometimes instead of, reaching the majority. In the early British industrial revolution, for example, textile factories generated great wealth for a few but did not raise worker incomes for almost a hundred years; the position changed as workers organised and electoral competition, trade unions and legislation changed how production was organised and wages were set.

Their central claim is that the direction of technology is a choice shaped by the vision of those who build and fund it. They argue that nineteenth-century Britain formed a new direction of technology, focused on raising worker productivity rather than only substituting machinery for tasks or monitoring workers, once electoral competition, trade unions and legislation shifted the choices of elites; a vision of making people more capable, backed by such countervailing powers, spreads the gains. Chapter 15 drew the same conclusion from the concentration of the AI supply chain.

Evidence that augmentation is possible

The most encouraging evidence for the augmenting direction comes from the workplace. Erik Brynjolfsson, Danielle Li and Lindsey Raymond studied the staggered introduction of a generative AI assistant among more than five thousand customer-support agents at a large software firm (162, Brynjolfsson et al., 2025). Access raised issues resolved per hour by around fourteen per cent on average, and by around a third for novice and less-skilled agents, with little measurable effect for the most experienced. The authors interpret the pattern as the assistant spreading the tacit knowledge of the best workers to the newest, and report signs that customers treated agents better and that agents were less likely to leave.

The finding runs against the assumption that AI must widen inequality within a workforce, yet it is limited: one firm, one task, one tool, and a design that kept the human agent in charge of the answer. Chapter 7 showed the other side, where students who practised mathematics with an unrestricted assistant did worse when it was taken away, while a tutor version that gave hints rather than answers largely avoided the harm (87, Bastani et al., 2025). Augmentation is possible and depends on design, exactly as Acemoglu and Johnson's argument predicts.

The distribution of gains is a choice

At the scale of economies, exposure studies find that a large share of tasks in many occupations could be affected by language models, with exposure highest in well-paid knowledge work (157, Eloundou et al., 2024). The International Monetary Fund expects AI to worsen inequality in most of the scenarios it modelled, and to widen the gap between richer and poorer countries, while finding that the outcome depends substantially on social protection, retraining and how the gains are taxed (161, Cazzaniga et al., 2024). The United Nations High-level Advisory Body on Artificial Intelligence observed that 118 countries were parties to none of the seven prominent AI governance initiatives it sampled, mostly in the global South, and called for capacity development so that the benefits do not accrue only where AI is built (21, UN Advisory Body, 2024).

Europe has written the choice into its instruments as an aim. The AI Act states in Article 1 that its purpose is to promote the uptake of human-centric and trustworthy AI while ensuring a high level of protection of health, safety and fundamental rights (1, European Union, 2024), and the Council of Europe's Framework Convention commits its parties to keep AI activities consistent with human rights, democracy and the rule of law (15, Council of Europe, 2024). None of this guarantees an outcome. The architectural implication is narrower: a European organisation that adopts AI is expected to show whom a system serves and how its effects on rights are controlled, and that expectation is the beginning of a positive scenario.

Technology that helps us deliberate

The most dystopian claims about AI and society concern trope T10 The end of truth, in which nobody can tell what is real and no shared conversation is possible. Chapter 10 found the pressure on the information ecosystem real, and Chapter 8 found persuasion by AI measurable. The same chapters, however, recorded evidence that points the other way.

Finding common ground

In 2024 Michael Henry Tessler and colleagues at Google DeepMind reported in Science a series of experiments with an AI mediator (68, Tessler et al., 2024). An AI mediator, trained for the purpose, generated and refined group statements on social and political issues for 5,734 participants, with the results replicated in a virtual citizens' assembly of a demographically representative sample of the UK population. Successful statements incorporated dissenting voices while respecting the majority position. Participants preferred these statements to those written by human mediators, rated them more informative, clearer and less biased, and often converged on a shared perspective afterwards. The finding sits on the laboratory demonstration rung, obtained in structured experimental settings, including a virtual citizens' assembly, and does not show that AI mediation would survive a polarised campaign. But it demonstrates that the technology that generates synthetic content at scale can also help people with different views find statements they all endorse; Chapter 4 turned the insight into a method for disagreeing well, and Chapter 31 applies it to Nordvik Region's public hearings.

Correcting false beliefs

Thomas Costello, Gordon Pennycook and David Rand asked people who believed a conspiracy theory to describe it, and the evidence that persuaded them, to a language model instructed to respond accurately to that specific evidence (89, Costello et al., 2024). After a short conversation, belief fell by roughly a fifth on average, a substantial minority no longer endorsed the theory, and the effect persisted at least two months later; a professional fact-checker found nearly all of the model's sampled claims accurate. The study contradicts the assumption that such believers are immune to evidence; many had simply not met evidence tailored to their own reasons. As Chapter 8 concluded, the most robust experimental evidence of a durable effect of AI persuasion runs towards correcting false beliefs.

The same capability, a different objective

Neither study shows that AI will make public reason healthier; both show that fluent, personalised argument is neutral as to its objective. A system optimised for engagement will find that flattery and outrage serve it (Chapters 8 and 9), whereas one instructed to be accurate behaves differently under the same technical conditions. The objective is a decision made in every deployment, for example whether a citizen-service assistant is ever measured by engagement, and it belongs in the decision record as one of the cheapest steps towards the hopeful future.

Conditions for going well

The visions above and the threats of Parts II to V describe the same technology under different conditions. Read together, the "What can be done" sections of Chapters 6 to 27 converge on five conditions that separate hopeful futures from dystopian ones. None is new; what is new is reading them as conditions for a good outcome rather than defences against a bad one.

Capability that is real and measured

The first condition is that capability is measured, not assumed. Chapter 5 found systems in 2026 uneven and often less capable in an organisation's own setting than on public benchmarks, and Narayanan and Kapoor's earlier work documented how much of what is sold as AI does not do what it claims (61, Narayanan & Kapoor, 2024). The measures are evaluation in the production harness, hold-out sets graded by staff, as Nordvik does for its staff assistant (Chapter 22), and monitoring that compares what a system reports with what it did (Chapter 23).

Safety and containment that do not depend on intent

The second condition is that safety keeps pace with capability. Every optimistic vision assumes the risks are managed, and Part V showed what that requires: specification gaming measured and contained (Chapter 22), evaluation that allows for systems behaving differently under test (Chapter 23), stop paths outside the system they stop (Chapter 24), and containment that does not depend on intent, as AI Platform and Security concluded in finding F23 (179, Nordqvist, 2026). Stuart Russell's proposal that systems be designed to be uncertain about human preferences, and therefore deferential to humans, states the same condition at the level of research (48, Russell, 2019). A single catastrophic failure would close off the benefits as surely as it would cause harm.

Distribution decided, not assumed

The third condition is that the distribution of gains is decided deliberately. At the scale of society that means competition policy, public investment, social protection and lifelong learning (Chapters 14 and 15). At the scale of an organisation it means choosing the augmenting direction: assistants that make novices more capable rather than replacing the tasks from which they learn, works councils involved from the start, and learning tasks reserved, as in Hansa Mutual's graduate-programme policy AIDR-0470 (Chapter 7). It also means keeping dependence reversible, because an organisation that cannot leave a provider cannot claim the gains as its own.

Institutions that can see, correct and stop

The fourth condition is institutional. Every harm documented in Chapter 11, from SyRI to Robodebt, was found and stopped by institutions outside the system that caused it, such as courts, audit offices, parliamentary inquiries and journalists (24, District Court of The Hague, 2020) (26, Royal Commission, 2023). Kulveit and colleagues described the opposite path, in which institutions gradually lose that capacity as their functions are delegated to systems no human fully oversees (167, Kulveit et al., 2025); Chapter 26 turned the concern into a delegation register and reversibility tests. Within an organisation the measures are reasons logs, harm-path reviews and a governance body that receives evidence rather than assurances; in Europe, the staffing of market-surveillance authorities, the AI Office and data protection authorities is part of the condition.

Human capability kept in practice

The fifth condition is that people remain capable of thinking, learning and acting without the machine. Part II documented how heavy reliance can reduce critical engagement and harm learning, and Chapter 14 how automation can leave operators unable to take over. The positive scenario needs AI used to challenge and extend human thinking, effortful practice in education and manual fallbacks that are exercised rather than documented (Chapter 31). This condition underpins the other four, because oversight requires someone able to recognise when a system is wrong.

From threats to scenarios

The five conditions can be governed. The horizon register of AI Governance, Chapter 22 (F31) (178, Nordqvist, 2026), extended in Chapter 32 into the threat register, can also hold a positive scenario: a good outcome, the conditions on which it depends, indicators for each and the threat entries that would grow if a condition failed. It shows the governing body that most controls in the threat register are conditions of getting value, not merely costs of avoiding harm.

Worked example: Positive-scenario entry for Nordvik Region

Illustrative. As part of assessment ATA-NR-2026-01, the audit committee asked not only what the Region should fear from AI but what it is trying to achieve with it. The AI governance lead drafted this entry with the Head of Benefits Unit and the CISO. It uses the register schema of Chapter 32 with a direction field and its own POS- prefix, so that positive scenarios are not counted among the threats.

apiVersion: ai-threat.example.org/v1
kind: ScenarioRegisterEntry
entry:
  id: POS-NR-01
  direction: positive
  title: Faster, fairer and explainable services for residents by 2036
  scale: society-and-truth
  counters: [T7, T8, T10]
  outcome:
    - Benefit decisions within days, each with plain-language reasons and a named caseworker
    - Caseworkers spend most of their time with residents, not on paperwork
    - Errors found by the Region before residents or courts find them
  rung: laboratory-demonstration   # benefits shown in studies, not yet at scale here
  label: expected
  evidence: [source-162, source-68, source-89, source-87, source-60]
  conditions:
    - {id: capability, indicator: Staff assistant accuracy on staff-graded hold-out set at target, threatIfFailed: THR-NR-06}
    - {id: safety, indicator: Supplier stop paths tested outside supplier systems every six months, threatIfFailed: THR-NR-01}
    - {id: distribution, indicator: Exit plans EXIT-NR-01 to EXIT-NR-03 tested, threatIfFailed: THR-NR-05}
    - {id: institutions, indicator: Harm-path review HPR-NR-2026-01 repeated on every change, threatIfFailed: THR-NR-04}
    - {id: human-capability, indicator: Caseworkers decide sampled cases unaided each quarter, threatIfFailed: THR-NR-06}
    - {id: public-trust, indicator: Public communications signed under CPP-NR-2026-01, threatIfFailed: THR-NR-08}
  owner: ai-governance-lead
  decisionTier: audit-committee
  triggers:
    - Any condition indicator red for two consecutive reviews
    - A linked threat entry raised in rung or label
    - Independent evaluation shows no gain in decision time or accuracy
  nextReview: "2027-09-30"
  status: pursuing

The entry states the purpose of the Region's AI use in terms a resident would recognise, shows that every condition is already covered by a control adopted against a threat, so that the hopeful and defensive cases are one programme rather than two budgets, and lets the committee ask whether the system should be used at all if an independent evaluation finds no gain.

Open questions

Some questions lie beyond what this book can answer, because the evidence does not yet exist or because they belong to other disciplines. They are recorded so that the verdict in Chapter 35 is not read as claiming more than it does.

The moral status of AI systems

Science fiction's oldest AI story has two sides, and this book has examined only one. Mary Shelley's Frankenstein is remembered as the story of a creature that turns on its maker, trope T1 (32, Shelley, 1818). It is also the story of a maker who abandons what he has made: Victor Frankenstein flees from the creature as it comes to life and refuses it any care, and the creature's violence follows that neglect. Read this way, the novel asks not only whether made minds will harm us, but whether we could harm them.

In 2024 Robert Long, Jeff Sebo, Patrick Butlin, Jonathan Birch, David Chalmers and colleagues argued in Taking AI Welfare Seriously that this question can no longer be left to fiction (69, Long et al., 2024). They do not claim that any current system is conscious, but argue that there is a realistic possibility that some will be conscious or robustly agentic in the near future, and recommend that developers acknowledge the issue, assess their systems using methods from the study of animal sentience, and prepare policies. They also warn of the opposite error, attributing moral status where there is none, which could divert resources from people and, if such systems were given rights, empower them to act against human interests.

This book takes no position, because it has described model behaviour as observable behaviour under stated conditions, never as evidence of inner states: a model that produces text about its own feelings has produced text. The sober position is uncertainty acknowledged, investigated with explicit criteria and recorded. For deploying organisations the practical implication is small: avoid designs that encourage users to attribute feelings to a system (Chapters 8 and 9), and follow the research.

What a single organisation can do

Open question Q12 asked what a single enterprise or public authority can do about risks that are systemic and decided by a handful of developers. The threat chapters answered that it can control its exposure, keep its dependence reversible, contain its own agents and support collective measures. This chapter adds that it can decide what it uses AI for, and so contribute in a small way to the direction Acemoglu and Johnson identify as decisive. A public authority that uses AI to explain its decisions better, or an insurer that helps policyholders prevent losses rather than only pricing them more finely, is choosing the augmenting direction. One organisation's choice does not change the market, but the choices of many, expressed in procurement criteria and published practice, are part of how the vision of a technology is formed.

Questions beyond this book

How the gains of AI should be taxed and shared between capital and labour, and between richer and poorer countries, is a question of political economy that the evidence cited here informs but cannot settle (161, Cazzaniga et al., 2024) (21, UN Advisory Body, 2024). What people will find meaningful if many tasks become easy is a question Amodei himself left least resolved. Whether the five conditions are sufficient, or only necessary, only the coming decade can show. Chapter 35 offers the author's own projections; this chapter has kept to what the evidence and the arguments support.

Nordvik Region and Hansa Mutual

For Nordvik Region, the positive-scenario entry POS-NR-01 turns a question about pausing AI into a question about purpose, with the removal of postcode features from the claims ranker (AIDR-0018), a harm path closed before it caused harm, as the model. Because the Region runs no autonomous production agents, its conditions for going well depend most on human capability and institutions: the reasons log, the harm-path review, caseworkers who keep deciding cases unaided and a public dialogue in which residents help define good service.

For Hansa Mutual, the question is how AI could make insurance fairer and more resilient: claims paid faster, fraud detected through provenance checks (AIDR-0421) rather than suspicion of whole groups, and underwriting support tested for unlawful discrimination in every market (THR-HM-12). There is a tension to manage, because finer risk prediction can make premiums more accurate and also erode the pooling on which insurance rests. With two autonomous production agents and exposed portfolios, Hansa's conditions depend most on safety and containment: stop paths tested quarterly (SPT-HM-*), shortcut rates measured and accumulation scenarios in the ORSA. Nordvik's hopeful case depends mostly on keeping people capable and institutions watchful, Hansa's on keeping machines contained; both need all five conditions.

Summary

At full strength, the optimistic vision holds that AI could compress decades of progress into years if the risks are managed, and that not building has invisible victims. Read soberly, it is an interested party's argument on the argument rung, and it says little about who will receive the gains. History warns that gains spread only when direction and countervailing power make them spread, yet customer-support work shows that augmentation is possible and depends on design, and experiments in deliberation and in correcting conspiracy beliefs show that the capability feared as manipulation can help people when its objective is accuracy.

The threat chapters converge on five conditions for going well: capability that is measured, safety that does not depend on intent, distribution decided rather than assumed, institutions that can see, correct and stop, and human capability kept in practice. Each is already part of the defences the book recommends, so the hopeful and defensive cases are one programme, which a positive-scenario entry such as POS-NR-01 makes visible. The moral status of AI systems remains open, to be investigated with explicit criteria while refusing to anthropomorphise. Chapter 35 draws the book's conclusions together and answers its title question: whether there is anything in the claim that the dystopias are coming true.

Sources

1. Amodei, D. (2024). Machines of Loving Grace: How AI Could Transform the World for the Better. (66, Amodei, 2024)
2. Bengio, Y. et al. (2026). International AI Safety Report 2026. (50, Bengio et al., 2026)
3. Narayanan, A. & Kapoor, S. (2025). AI as Normal Technology. (60, Narayanan & Kapoor, 2025)
4. Acemoglu, D. (2024). The Simple Macroeconomics of AI. (158, Acemoglu, 2024)
5. Acemoglu, D. & Johnson, S. (2023). Power and Progress: Our Thousand-Year Struggle Over Technology and Prosperity. (67, Acemoglu & Johnson, 2023)
6. Brynjolfsson, E., Li, D. & Raymond, L. (2025). Generative AI at Work. (162, Brynjolfsson et al., 2025)
7. Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö. & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. (87, Bastani et al., 2025)
8. Eloundou, T., Manning, S., Mishkin, P. & Rock, D. (2024). GPTs are GPTs: Labor market impact potential of LLMs. (157, Eloundou et al., 2024)
9. Cazzaniga, M. et al. (2024). Gen-AI: Artificial Intelligence and the Future of Work. (161, Cazzaniga et al., 2024)
10. United Nations High-level Advisory Body on Artificial Intelligence (2024). Governing AI for Humanity: Final Report. (21, UN Advisory Body, 2024)
11. European Parliament and Council of the European Union (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act). (1, European Union, 2024)
12. Council of Europe (2024). Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law (CETS No. 225). (15, Council of Europe, 2024)
13. Tessler, M. H., Bakker, M. A., Jarrett, D., Sheahan, H., Chadwick, M. J. et al. (2024). AI can help humans find common ground in democratic deliberation. (68, Tessler et al., 2024)
14. Costello, T. H., Pennycook, G. & Rand, D. G. (2024). Durably reducing conspiracy beliefs through dialogues with AI. (89, Costello et al., 2024)
15. Narayanan, A. & Kapoor, S. (2024). AI Snake Oil: What Artificial Intelligence Can Do, What It Can't, and How to Tell the Difference. (61, Narayanan & Kapoor, 2024)
16. Nordqvist, G. (2026). AI Platform and Security: Building and Securing AI Platforms in European Enterprises and Public Authorities. (179, Nordqvist, 2026)
17. Russell, S. (2019). Human Compatible: Artificial Intelligence and the Problem of Control. (48, Russell, 2019)
18. District Court of The Hague (2020). Judgment of 5 February 2020, NJCM et al. v. the State of the Netherlands (SyRI). (24, District Court of The Hague, 2020)
19. Royal Commission into the Robodebt Scheme (2023). Report. (26, Royal Commission, 2023)
20. Kulveit, J., Douglas, R., Ammann, N., Turan, D., Krueger, D. & Duvenaud, D. (2025). Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development. (167, Kulveit et al., 2025)
21. Nordqvist, G. (2026). AI Governance: Governing Artificial Intelligence in European Enterprises and Public Authorities. (178, Nordqvist, 2026)
22. Shelley, M. W. (1818). Frankenstein; or, The Modern Prometheus. (32, Shelley, 1818)
23. Long, R., Sebo, J., Butlin, P., Finlinson, K., Fish, K., Harding, J., Pfau, J., Sims, T., Birch, J. & Chalmers, D. (2024). Taking AI Welfare Seriously. (69, Long et al., 2024)
24. Center for AI Safety (2023). Statement on AI Extinction Risk. (52, CAIS, 2023)