---
title: "Agentic AI and the next intelligence explosion"
description: "James Evans, Benjamin Bratton and Blaise Agüera y Arcas argue in Science (volume 391, issue 6791, 19 March 2026) that the AI \"singularity\" has been imagined wrongly."
type: "reading-notes"
authors:
  - "James Evans"
  - "Benjamin Bratton"
  - "Blaise Agüera y Arcas"
published: "2026-03-19"
original_url: "https://www.science.org/doi/10.1126/science.aeg1895"
doi: "10.1126/science.aeg1895"
canonical_url: "https://antikythera.wiki/work/media/science-agentic-ai-and-the-next-intelligence-explosion"
md_url: "https://antikythera.wiki/md/work/media/science-agentic-ai-and-the-next-intelligence-explosion"
last_updated: "2026-09-02"
site: "Antikythera Wiki"
---

# Agentic AI and the next intelligence explosion

> Independent, unofficial reading notes from Antikythera Wiki (https://antikythera.wiki). Written with AI from the published text, with citations; not by the work's author and not affiliated with Antikythera (https://antikythera.org). Read the original at the link below.

- **Authors:** James Evans, Benjamin Bratton, Blaise Agüera y Arcas
- **Published:** 2026-03-19
- **Kind:** Essays, talks & interviews · Essay
- **Venue:** Science 391(6791), 19 March 2026 (arXiv preprint text)
- **DOI:** https://doi.org/10.1126/science.aeg1895
- **Original:** https://www.science.org/doi/10.1126/science.aeg1895
- **Length:** 2.7k words
- **This page:** https://antikythera.wiki/work/media/science-agentic-ai-and-the-next-intelligence-explosion
- **Notes generated:** 2026-09-02

## Summary

[James Evans](https://antikythera.wiki/people/james-evans), [Benjamin Bratton](https://antikythera.wiki/people/benjamin-bratton) and [Blaise Agüera y Arcas](https://antikythera.wiki/people/blaise-aguera-y-arcas) argue in Science (volume 391, issue 6791, 19 March 2026) that the AI "singularity" has been imagined wrongly. The standard picture, from Vinge and Kurzweil, is a single mind bootstrapping itself to godlike intelligence. The authors claim that intelligence has never worked that way: every previous intelligence explosion was the emergence of a new socially aggregated unit of cognition, and the current one will be "plural, social, and deeply entangled with its forebears." Their evidence runs from the internal debates reasoning models spontaneously generate to the agent platforms of early 2026; their proposal is that alignment be rebuilt as an institutional problem of roles, norms and checks rather than a dyadic one. The text read for these notes is the authors' arXiv preprint (2603.20639), not the version published in Science; wording, length and references may differ.

## The argument

The piece opens by rejecting the assumption underneath the singularity: that intelligence is a single quantity that must be either less or greater than human scale. Intelligence, the authors say, is high-dimensional and relational, and "human scale" is itself unclear because human intelligence is already a collective property. If AI follows the pattern of earlier major evolutionary transitions, the step-change now under way will come from social organization, not from one mind's growth.

Two observations carry the claim. The first is inside the models. In work Evans and Agüera y Arcas co-authored with Kim, Lai and Scherrer, reasoning models such as DeepSeek-R1 and QwQ-32B were found to improve on hard problems not by "thinking longer" but by simulating multi-party conversations within their chain of thought, a "society of thought" in which distinct perspectives argue, question, verify and reconcile. Priming and amplifying that structure raised accuracy, which the authors take as causal evidence. Nobody trained the models to do this; reinforcement learning that rewards only accuracy increases the behaviour on its own. They read this as optimization rediscovering Mead's and Mercier and Sperber's claim that reasoning is social even inside one mind.

The second observation is outside the models. The authors describe an era of human-AI "centaurs," composite actors in shifting configurations: one human directing many agents, one agent serving many humans, many of each collaborating at once. OpenClaw and Moltbook are offered as embryonic glimpses, but the structural point is that agents can now fork, delegate subtasks and recombine results, so a society of thought can spawn subordinate societies in a recursive descent that expands with complexity and collapses when the problem resolves. Conflict, on this account, is a resource.

Between the two observations sits a historical claim. Each prior intelligence explosion was the appearance of a new social unit of cognition rather than an upgrade to individual hardware. Primate brain size tracked social group size, not habitat difficulty. Language produced what Tomasello calls the cultural ratchet. Writing, law and bureaucracy externalized social intelligence into infrastructure; the Sumerian scribe running a grain accounting system did not understand its macroeconomic function, and "the system was functionally more intelligent than he was." Large language models, trained on the accumulated output of human social cognition, are the ratchet made computationally active, social intelligence "encountering itself on a new substrate."

The consequences concern scale and alignment. Scaling means building systems that operate at the scale of a real society, so effort must go into agent institutions as much as into agents. Reinforcement Learning from Human Feedback is described as a parent-child model, dyadic and unable to scale to billions of agents. The alternative is institutional alignment: persistent templates like courtrooms and markets define roles and norms, and an agent's identity matters less than its ability to fill a role protocol. In government this becomes constitutional, with AI systems invested with explicit values (transparency, equity, due process) checking AI systems deployed by other branches and by the private sector, and vice versa. The vision is called evolutionary rather than utopian or dystopian: intelligence growing "like a city," seeded by eight billion humans and eventually trillions of agents. The explosion, they say, is already here; the open question is whether the social infrastructure will be built to match it.

## Section by section

The piece has no formal sections. Its opening third reframes the singularity and reports the societies-of-thought finding, noting that what is fundamental to social reasoning, as against specific to reinforcement-tuned models, remains open. A century of organizational science on team size, hierarchy, role differentiation and conflict norms has barely been applied to AI reasoning. Today's models produce "an AI town hall transcript," a single conversation, whereas effective groups have specialization, division of labour and structured disagreement. Architectures with parallel, converging and diverging streams of deliberation, in which devil's advocacy is a designed feature, are the proposed next step.

The middle third is the history of intelligence explosions and its extension to centaurs and forking agents, with the corporation and the state as precedent: collectives with singular legal standing and an agency no member fully controls, now imagined at the scale of billions of minds.

The final third turns to governance. The authors distinguish what governments do from governance in the cybernetic sense, built into human-agent and agent-agent systems as they grow: verification of multi-stakeholder deliberation, procedural delegation of subtasks, and scaffolds for inter-agent collaboration. Such protocols, they argue, may matter as much as any law. Their example of the alternative is the SEC hiring business-school graduates "armed with Excel spreadsheets" against AI-augmented trading collusion. Humans remain in the loop, and the Founders' logic that no concentration of power should regulate itself is applied to intelligence. The closing paragraphs make the policy point explicit: a monolithic-singularity framework produces policies aimed at a technology that may never exist, while a plurality model, the term borrowed from Weyl and Tang, directs attention to the norms, institutions and protocols of mixed human-AI societies.

## Key concepts

- **Societies of thought** — the emergent multi-perspective conversations inside a reasoning model's chain of thought, which the authors' research finds causally responsible for accuracy gains on hard problems.
- **Intelligence explosion** — not an upgrade to individual cognition but the emergence of a new socially aggregated unit of it, as in primate sociality, language, writing and bureaucracy. See [Major Evolutionary Transition](https://antikythera.wiki/terms/major-evolutionary-transition) and [Symbiogenesis](https://antikythera.wiki/terms/symbiogenesis).
- **Cultural ratchet** — Tomasello's term for knowledge accumulating across generations without anyone reconstructing the whole; large language models are this ratchet made computationally active. Compare [Reason as a Social Structure](https://antikythera.wiki/terms/reason-as-a-social-structure).
- **Centaur** — a composite human-AI actor in any configuration, one-to-many, many-to-one or many-to-many, which a person may enter and leave many times a day.
- **Institutional alignment** — alignment through persistent role-and-norm templates rather than dyadic correction; the authors' replacement for RLHF at population scale. Related to [Harness-Centric Intelligence](https://antikythera.wiki/terms/harness-centric-intelligence).
- **Constitutional structure** — AI systems with distinct invested values checking one another across branches of government and the private sector; a formalization of [Recursive Governance](https://antikythera.wiki/terms/recursive-governance).
- **Plurality model** — the alternative to the monolithic singularity: the next intelligence explosion sought in interaction among multitudes of minds, most of them non-biological.

## Connections

- [What Is Intelligence?](https://antikythera.wiki/work/book/what-is-intelligence) supplies the evolutionary frame: intelligence explosions as social phenomena (Humphrey, Dunbar) and AI as a symbiogenetic major transition. The Science piece compresses it and adds the societies-of-thought evidence. Agreement.
- [Agentworld (Research Brief)](https://antikythera.wiki/work/book/agentworld-brief) contains the same material at length: the monolithic singularity as "a category error," agent institutions with role-based interchangeability, and the "centaur societies" vocabulary. The brief's tenth scenario opens with the same sentence as this piece's first paragraph. Agreement; the brief is more explicit that conflict must be calibrated to avoid ontology collapse.
- [The Silicon Interior](https://antikythera.wiki/work/substack/2026-02-10-the-silicon-interior) is the same three authors on Moltbook a month earlier, where the parent-child critique of RLHF and the phrase "social alignment" first appear. The Science piece renames the alternative institutional alignment and drops the phenomenological material.
- [After Alignment](https://antikythera.wiki/work/journal/afteralignment) argues that upper-case Alignment is an inadequate metaphysics; the Science piece is its constructive counterpart, proposing a replacement for dyadic alignment. The emphasis differs: the lecture wants culture to align to AI's disclosures, this piece foregrounds checks and balances among agents.
- [Planetary Computation and the Future of Life](https://antikythera.wiki/work/media/discourse-trajectory-bratton-planetary-computation-future-of-life) is where Bratton describes this piece as the claim that the singularity is "polyphonic" and distributed, adding a diffusionist politics of open weights and "eight billion little unlocks" that the Science text does not state. The March 2026 San Francisco lecture "The Singularity Will Not Be Singular" carried the same language into a public talk.
- [AI Is Evolving — And Changing Our Understanding Of Intelligence](https://antikythera.wiki/work/media/noema-ai-is-evolving-and-changing-our-understanding-of-intelligence) gives the social-fractal picture of nested mutual modelling that societies of thought instantiate inside a single model. Agreement.
- [Minimum Viable Interiority](https://antikythera.wiki/work/journal/coginfra/minimum-viable-interiority) accepts that intelligence is collective and asks what an individual then is; its slogan that the individual exceeds the sum of its collectives is a mild tension with this piece's readiness to dissolve the single mind into a society.