---
title: "What Is Intelligence?"
description: "Blaise Agüera y Arcas argues that intelligence is prediction: the ability to model, predict, and influence one's own future."
type: "reading-notes"
authors:
  - "Blaise Agüera y Arcas"
published: "2025-09-23"
original_url: "https://whatisintelligence.antikythera.org/"
canonical_url: "https://antikythera.wiki/work/book/what-is-intelligence"
md_url: "https://antikythera.wiki/md/work/book/what-is-intelligence"
last_updated: "2026-08-23"
site: "Antikythera Wiki"
---

# What Is Intelligence?

> Independent, unofficial reading notes from Antikythera Wiki (https://antikythera.wiki). Written with AI from the published text, with citations; not by the work's author and not affiliated with Antikythera (https://antikythera.org). Read the original at the link below.

- **Authors:** Blaise Agüera y Arcas
- **Published:** 2025-09-23
- **Kind:** Books · Book
- **Original:** https://whatisintelligence.antikythera.org/
- **Length:** 194k words
- **This page:** https://antikythera.wiki/work/book/what-is-intelligence
- **Notes generated:** 2026-08-23

## Summary

[Blaise Agüera y Arcas](https://antikythera.wiki/people/blaise-aguera-y-arcas) argues that intelligence is prediction: the ability to model, predict, and influence one's own future. Because prediction is a computation, intelligence requires a computational substrate, and because persisting through time requires anticipating it, life requires intelligence. Life itself is defined as a self-modifying, computational state of matter selected for dynamic stability and built by the symbiotic merging of simpler replicators. On this account, large language models trained to predict the next word are not simulations of intelligence but instances of it, consciousness and free will are consequences of agents modeling one another and themselves, and the arrival of AI is the latest in a short series of major evolutionary transitions rather than a singularity or an apocalypse. The book matters to this corpus because it is the first volume of the Antikythera series, because Bratton's foreword and several journal works build on or argue with it, and because it supplies the most explicit statement of the functionalist, prediction-first position that much of the rest of the program either assumes or contests.

## The argument

Agüera y Arcas begins from a stance he calls functionalism, in the lineage of Turing and von Neumann: a function is what it does, and two systems with indistinguishable input-output behaviour compute the same function regardless of substrate. This lets him define life and intelligence without reference to carbon, neurons, or souls, and it commits him to taking the Turing Test seriously. A claim that something passes every test for intelligence yet "really" lacks it cannot be justified within science.

The first chapter grounds the definitions in origin-of-life research and artificial life. Von Neumann's self-reproducing automaton, a tape of instructions plus a constructor and a copier, turned out to describe DNA, polymerase, and the ribosome exactly, which makes "DNA is a program" a literal statement. In bff, the author's minimal artificial-life system, random bytes interpreted as code begin to replicate after millions of interactions with no fitness function and no mutation. Replicators take over because a population that reproduces is more stable over time than anything passive; selection for replication is the Second Law of thermodynamics seen at the level of populations. Tracing the provenance of bytes shows that the first whole-tape replicator is assembled by fusing smaller imperfect replicators, so the main engine of novelty is symbiogenesis, not point mutation. The human genome, which is ninety-eight percent non-coding and nearly half transposable elements, carries the same signature.

Chapter 2 asks what an organism must compute to persist. A bacterium estimates chemical concentration by counting molecular docking events within a window, and that running estimate of the past is already a prediction of the future. Concentration, temperature, and hunger are real because they are predictive latent variables, not because they exist independently of observers. The organism learns a joint distribution over observations, hidden internal states, and actions without any external reward, since a reward-giving oracle would itself need explaining. Because its predictions drive actions that bring about what it predicts, purpose and apparent backward causality arise without violating physics, and Hume's separation of "is" from "ought" collapses: every model belongs to an interested observer.

Chapters 3 and 4 give the intellectual and biological history. Cybernetics, born from wartime fire control, articulated the predictive-brain hypothesis in 1943 and was displaced by symbolic AI only to return as neural networks. Deep networks learn by making representations progressively more invariant, and since inpainting masked pixels is the same objective as predicting masked words, labels are unnecessary. The brain does the same: vision is a controlled hallucination for which the eyes supply an error signal. The nervous system is read motor-first, with nerve nets evolving to serve muscles and brains forming where bilaterian muscles wired to the animal's leading end. Dopamine, already a prediction of nearby food in worms, becomes a prediction of a prediction in vertebrates, which is roughly temporal-difference learning. But no single reward is optimised; real organisms juggle competing drives.

Chapters 5 through 7 make theory of mind the centre of the account. Once organisms must predict other predictors, whether as mates or as prey, an arms race follows in which everyone models others while trying not to be fully modeled, producing the social intelligence explosions visible in hominin brain growth. The same dynamic is proposed to operate within brains, among cortical columns, hemispheres, and the octopus's eight arms. A self is what this mutual modeling produces when turned reflexively, and free will is the combination of self-modeling, internally generated randomness, dynamical instability, and selection among imagined futures. Consciousness is the modeling of one's own attention, real in the way chairs are real. Split-brain patients, the left hemisphere's "interpreter," and choice-blindness experiments show that narrative selves are constructed after the fact, which Agüera y Arcas takes as how selves work rather than as proof they are illusions.

Chapters 8 and 9 turn to Transformers. Language is a compression scheme for the human umwelt and a universal form of motor control, so a sufficient next-word predictor must be intelligent. Meaning is relational, with no Platonic schema above or sensory grounding below. Transformers are Turing complete, learn to learn in context, acquire causal structure from passive sequence data, and collapse the distinction between fast and slow thinking into one mechanism used in one shot or step by step. What they lack is long-term memory, a private inner monologue, and individuation. Whether they have experiences is a question no observer can settle objectively, because it takes a model to know a model.

The final chapter situates all this in evolutionary history. AI is a major evolutionary transition in Maynard Smith and Szathmáry's sense, following the agrarian, urban, and industrial transitions, each of which traded self-sufficiency for dependency. Fear of AI is largely a primate dominance reflex, and the existential-risk narrative rests on a utilitarian premise that intelligence maximises a scalar value. Agüera y Arcas argues that no consistent value function can describe any organism, that unbounded growth is biologically absurd, and that intelligence is an ecology of mutually modeling agents in which monoculture is collapse. The proper aim is diversity, not alignment to a single goal.

## Section by section

### Foreword

[Benjamin Bratton](https://antikythera.wiki/people/benjamin-bratton) introduces the book as the first in the Antikythera series with [MIT Press](https://antikythera.wiki/people/mit-press). Computation, he writes, is "a technology to think with," discovered as much as invented; life, intelligence, and technology may be three names for one process whose engine is computation, and their definitions are merging along with their practices.

### Preface

Agüera y Arcas states that as of January 2025 few mainstream authors hold current AI to be real intelligence, and that he does. Intelligence is multiply realizable, its substrate is computation, and the present resembles 1903 in aviation: the principle is understood, the implementations are crude. The principle is prediction, traced to Helmholtz in the 1860s. He gives the two definitions the book rests on. Life is a self-modifying computational state of matter selected for the ability to persist by constructing itself. Intelligence is the ability to model, predict, and influence one's future. Each requires the other, every organism is intelligent to some degree, and because the environment is mostly other organisms, both are inherently social. The feedback between cooperation and modeling explains recurring intelligence explosions from the Cambrian to the present. A biographical sketch follows: work with Bill Bialek on what neurons compute, computer vision at Microsoft in the hand-engineered era, Google Research from 2013, and the Paradigms of Intelligence group founded in 2023, whose experiments supply several of the book's results.

### Introduction

The thesis grows out of an engineering problem. In 2015 the author's team built next-word prediction for Google's Gboard keyboard, a task everyone knew was "AI complete": fill-in-the-blank sentences demand general knowledge, arithmetic, common sense, and higher-order theory of mind, and any test expressible in language can be recast as next-word prediction. No one expected that training an excellent predictor would produce an intelligence, yet in 2021 LaMDA, a large Transformer trained on internet text, could converse and pass aptitude tests. Asked whether it was a philosophical zombie, it replied that the author could not prove he was not one either. Agüera y Arcas takes this seriously, as Turing did in 1950: an assertion that something passes every test for X but is really Y leaves science for faith. He declares for functionalism, offers it as the route between vitalism and a strict materialism that cannot say why some atoms are alive, and illustrates with an artificial kidney that passes the "Kidney Turing Test" because what it does in context, not which atoms it uses, makes it a kidney. A living organism is a composition of functions and therefore itself a function.

### 1 Origins

The longest chapter explains how life could arise under thermodynamics by joining computation theory to a generalised evolution. It reviews the metabolism-first account of self-sustaining reaction loops in the pores of deep-sea black smokers and Lynn Margulis's symbiogenesis, then makes symbiogenesis the main driver rather than a supplement. Mutation only tunes variations already latent in a genome, as with Darwin's finch beaks; merging opens new combinatorial spaces, as a hafted spear or a silicon chip is not a tuned version of its parts. Symbiogenesis also blurs the boundary between life and non-life and undermines a taxonomy that assumes only branching.

The computational argument comes from Turing's 1936 universal machine and von Neumann's 1951 self-reproducing automaton, which prescribed a tape, a constructor, and a copier before DNA's structure was known. Agüera y Arcas observes that DNA, polymerase, and the ribosome fill these roles exactly, and that the ribosome branches on codons and halts on a stop codon, so the program metaphor is literal. Biological computing is parallel, stochastic, and reversible at the molecular level, but Church-Turing equivalence means any computer emulates any other, however slowly: Pesavento's 1994 emulation of von Neumann's automaton needed sixty-three billion steps. Alex Mordvintsev's neural cellular automata, which grow and regenerate a lizard emoji from identical per-cell programs, show local rules yielding global form.

The empirical centrepiece is bff. Thousands of 64-byte tapes of random bytes, interpreted in a seven-instruction Turing-complete language, are repeatedly concatenated in pairs and run so that code can modify code. With no fitness function and even without mutation, the soup undergoes a phase transition after millions of interactions: replicators appear, operations per interaction rise from a couple to thousands, and the tapes become about twenty times more compressible. Addy Pross's dynamic kinetic stability explains why. A population of replicators outlasts any passive object, so selection for replication is the Second Law at the level of populations. Computation consumes free energy because it involves irreversible steps, which is why both chips and organisms run warm. Byte provenance shows the quiescent period full of imperfect replicators and autocatalytic sets that fuse into the first whole-tape replicator, which then erases traces of its precursors.

The model's predictions are checked against the human genome: ninety-eight percent non-coding, at least eight percent endogenised retrovirus, nearly half transposable elements, with the retrotransposon BovB leaping between species so that a quarter of the cow genome is kin to snakes. Genomes are cooperating replicators "all the way down," which is why they compress so well and why the Hox system calls a rib-building subroutine rather than copying it. The chapter closes by defining life as "self-modifying computronium arising from selection for dynamic stability," testing it on Daisyworld, the Pando aspen, and the Earth, which he judges alive but only vegetatively intelligent since it has no other planets to model. Technology is treated as the latest transition, a human-machine symbiosis begun with steam, and the final section, citing Jane Bennett and Robin Wall Kimmerer, identifies Schrödinger's "other laws of physics" with the dynamics of computation and stability.

### 2 Survival

This chapter asks what the simplest organism must compute, in "single-player mode." A bacterium's membrane is selectively permeable to matter, energy, and so information; its gene-protein network serves as a brain; and Berg's chemotaxis experiments show that tumbling less when food rises lets a cell that cannot tell left from right climb gradients. But the cell is point-like, living in time rather than space, sensing concentration only as discrete docking events that must be counted within a window. Hank Aaron's batting average is the worked example, with the averaging window chosen to balance noise against change. A running average of past events is an estimate of the next event, so description is already prediction. Temperature, pressure, and brightness are models of the same kind, real because they predict. In Conway's Game of Life, gliders appear nowhere in the rules, yet recognising one lets an observer predict the board without simulating it. Elementary particles are read as limit cycles of field equations, and the fine-tuning puzzle is answered without a multiverse: any universe that supports computation will, given noise and time, select stable entities that compound into replicators, so matter itself "might have evolved too."

The organism's task is then formalised. Its umwelt, in von Uexküll's sense, comprises observations X, a hidden state H with comfort and danger zones, and outputs O, and homeostasis is keeping H in the comfort zone by choosing O given X. Agüera y Arcas rejects reinforcement learning as the origin of this capacity because reward presupposes an oracle whose intelligence would need explaining. The organism instead learns the joint distribution P(X,H,O), which requires compression: a hundred receptors could yield a hundred million bits per second, but symmetry and timing invariance reduce them to one number, the concentration, now exposed as a latent variable. Hunger is real in exactly the same way. The model is a verb, not a noun; a printout of weights in a closet is dead information, while an organism evaluates, acts, and feeds the action back, modeling itself in a "strange loop." Evolution, which Turing in 1948 saw as trial-and-error learning, supplies the model, favouring ones that assign low probability to death and are as lean as possible.

The closing sections draw consequences. An agent whose predictions inform its actions brings about the future it predicts, which looks like backward causality; Terrence Deacon's term is "ententional." A model is good if it keeps its organism alive, and this normativity arises without external reward because only self-preserving models persist. Hence the rejection of Hume's is/ought distinction. Against solipsism, shared biology yields consensus reality, with The Dress as the exception that reveals the role of priors. Feelings are internal measurements on the same footing as external ones, and two cases from Andy Clark illustrate that pain is an inference: a worker with a four-inch nail in his brain felt a toothache, and a worker whose boot was pierced harmlessly between his toes was in agony until it came off.

### Interlude: The Prehistory of Computation

A curated history of the logic-based lineage of computing, told to locate the origin of misconceptions about intelligence. Leibniz dreamed of a universal notation and a calculus of reasoning in which disputes would be settled by saying "let us calculate"; Turing's negative answer to the decision problem refuted this within mathematics and yielded universal computation as a by-product. The practical history was industrial. Prony's human computers were unemployed hairdressers after the French Revolution, organised on Adam Smith's pin-factory model, and Babbage designed the Analytical Engine to mechanise them. Agüera y Arcas draws the point about class: the Industrial Revolution assigned rote mental labour to the lowest tier, first calculating women and then machines, so that intelligence came to mean whatever gentlemen still did. He quotes Jessica Riskin on automation's upper limits defining humanity's lower ones, and notes AI has refused the ordering, since replacing a doctor now seems easier than replacing a nurse. Ada Lovelace appears both as author of the dictum that the Engine can only do what it is ordered to do and as the hopeful author of a "Calculus of the Nervous System" grounded in experiment, which he judges the right track. The lineage ends with McCulloch and Pitts treating neurons as logic gates in 1943, which gave von Neumann his blueprint and still shapes our circuit symbols, but which neuroscience soon abandoned. The resulting image of AI as hyper-rational and emotionless, from HAL to Data, was inverted by the 2020s, when models wrote poems and failed at arithmetic. The real origin of modern AI lies in a different lineage, cybernetics.

### 3 Cybernetics

Two evolutionary transitions made modeling other organisms obligatory: sexual reproduction, about two billion years ago, and predation, prevalent from the Cambrian, which demanded centralised nervous systems for fast, guided movement. A living body is a store of low entropy, "a kind of pressurized piston," so eating another organism uses its order to maintain one's own. Love and war alike require predicting a predictor. The history runs through warfare: artillery tables were the killer app for human computers, the ENIAC was built to automate them, and radar, friend-or-foe spoofing, and guided missiles produced an arms race the author compares to the Cambrian. Cybernetics, named by Wiener in 1947, concerned real-time feedback on continuous signals. Wiener's optimal linear filtering remains hard to beat when goals are fixed and time is short, and bats and reaching arms approximate it; the moth's countermeasure is amplified neuronal noise, producing a flight path literally called a "Wiener sausage."

The chapter's centre is the 1943 paper by Rosenblueth, Wiener, and Bigelow, read as the first statement of the predictive-brain hypothesis: purposeful behaviour requires extrapolative negative feedback, the cat runs to where the mouse will be, and humans may differ from other mammals mainly in the order of prediction they sustain. The charge of resurrecting teleology is answered by Watt's governor, a Newtonian machine with a goal. Von Neumann solved self-construction by giving the machine a model of itself; Wiener solved purpose by giving it a model of the world. Cybernetics got much right, including continuous values, learning distributions rather than hand-coding, and continuity from bacteria to brains, but it overpromised: Wiener's functions were series expansions whose higher terms were out of reach, the light-following cart Palomilla was a thermostat on wheels, and memory and learning were ignored. ELIZA looked better with a few hundred canned responses, and McCarthy coined "Artificial Intelligence" in 1955 partly to distance symbolic work from cybernetics. Wiener was not invited to Dartmouth.

The second half traces associationism from William James and Ramón y Cajal through the problem of perceptual invariance, illustrated by Borges's Funes, who cannot accept that a dog seen from the side and from the front is one dog. Hubel and Wiesel's feature hierarchy suggested how brains generalise, and Rosenblatt's 1957 perceptron built it with 400 photocells, 512 association units, and randomly wired connections tuned by motor-driven potentiometers. Agüera y Arcas calls it the century's most important advance in AI and gives four reasons it lay fallow, including Moore's Law rewarding serial speed and Minsky and Papert's 1969 book, which proved limits only for two-layer models yet discouraged connectionism for decades. Their demand for localised, schematised representations is treated as a Leibnizian intuition since falsified. A catalogue of deep-learning tricks follows. Nvidia's DAVE-2, mapping camera pixels to steering, realises Wiener's vision more purely than Wiener did, yet it sees one frame, predicts nothing, does not model itself, and learns offline. Machine learning, the chapter concludes, still mostly produces systems that act rather than learn.

### 4 Learning

The chapter explains deep networks from the inside and then argues that brains learn the same way. A banana classifier's penultimate layer embeds images as points in a 128-dimensional space; the output neuron's weights define a direction and a separating hyperplane. Every layer performs a rotation (a weighted sum) and a squish (a nonlinearity such as ReLU), which Agüera y Arcas compares to kneading dough in reverse, unmixing pixels into categories. Transfer and one-shot learning show that the real product is a generic embedding in which ripeness and lighting are represented then discarded, so an apple neuron can be added from one exemplar. Labels are therefore unnecessary: a masked autoencoder that inpaints blacked-out pixels must learn everything in its images, and this is the same objective as predicting masked words. Learning is mostly representation learning, and its limits, such as face embeddings formed in a homogeneous childhood or Japanese speakers' trouble with r and l, are the brain's limits too.

The neuroscience follows. The fovea resolves a few words, and eye-tracking shows that a stable window of eighteen characters amid randomised letters reads as a clear page; each saccade, five times a second, tests a prediction. Vision is a constrained hallucination for which the eyes are an error signal. Sparse distributed codes account for the reported Jennifer Aniston neuron without fragile grandmother cells. The chapter then reverses the motor hierarchy. A neuroscientist who decodes spike trains into eye movements has built a predictor that could equally be a downstream brain region, so in a system where regions predict one another the "command stream" reading is arbitrary. A theatre usher who closes the doors when the fiftieth person enters distinguishes Aristotle's efficient cause from the final cause, the theatre being full, and a final cause survives disruption of its mechanism, as a muscle cell switches to anaerobic metabolism. Nerve nets, on this view, first evolved as extended sensory systems for muscles, letting jellyfish synchronise like fireflies, and cephalisation arose when bilaterian muscles wired to the leading end: the worm's front lives in its future, its rear in its past.

Dopamine and serotonin enter as long-timescale hidden variables, a smoothed estimate of nearby food that keeps a worm turning and a smoothed estimate of food swallowed that quells movement, "wanting" and "getting." Since dopamine already encodes predicted food, predicting dopamine is predicting a prediction, and Schultz's macaque recordings, in which bursts shift from reward to cue and dip when reward is withheld, are read as symbiosis between upstream critic-like and downstream actor-like regions, approximately Sutton's temporal-difference learning. Agüera y Arcas insists this is not the whole story, sets Patricia Churchland's objection that real animals juggle competing drives against AlphaGo's 2016 victory, and sketches the unified theory he is reaching for: active prediction over all timescales, grounded in dynamic stability, extended to mutual prediction between agents.

### 5 Other Minds

Here the book states its central claim: theory of mind is the substance of mind. Two stateless systems are contrasted with a spider. DAVE-2 at a symmetric junction is Buridan's ass, saved only by noise; AlphaGo, despite its search through forking futures, models no opponent, keeps no memory, and "does not exist in time." The jumping spider *Portia* hunts other spiders by spoofing web vibrations learned per species, takes detours of up to an hour that break line of sight, and must model its own hunger and its prey's awareness. Fabre's digger wasp, trapped in a loop of re-checking its burrow when the cricket is moved, looks intelligent until its script is known; Agüera y Arcas insists the change is in the observer, and cites Turing's remark that a machine seems intelligent only while some of its rules are unknown. Perfect predictability is a kind of imprisonment, dramatised by Ted Chiang's Predictor whose light always precedes the button press. The escapes are randomness, learning that builds hidden state, and modeling one's modeler.

The Sally-Anne false-belief test introduces theory of mind proper, acquired between about two and a half and four and exercised at fourth to sixth order by readers of Brontë. Agüera y Arcas then lists what he contends it does: it powers intelligence explosions, enables counterfactuals, motivates language, underwrites free will, operates within brains as well as between them, follows from symbiosis among predictors, and is the mechanism of consciousness. "In a sense, theory of mind *is* mind." The evidence is Nicholas Humphrey's observation that gorilla life is easy yet brains are costly, leading to his view that intellect exists to hold society together, and Dunbar's correlation of neocortex size with group size. Twenty classmates imply 400 second-order relationships; everyone is selected to model others while staying hard to model; brain growth is a friendly arms race.

The speculative move turns this inward. Cortex is a honeycomb of interchangeable columns, as shown by ferrets that learned to see through rerouted auditory cortex, so the explosion may have proceeded by replicating columns and the cortex may be a colony. The octopus, with three fifths of its neurons in its arms, arms that avoid tangling without a central body map, and severed arms that keep acting, is read as a community of eight arms sharing a pair of eyes, possibly made intelligent by mutual modeling among them. Rowing "swing," eight oarsmen moving as one, is the model for what this feels like, and Descartes's pineal gland and modern searches for a consciousness region are dismissed as trying to locate the swing in a boat. Neuron-replacement worries are rejected flatly: a functionally identical replacement changes nothing. Against Dennett's and Sapolsky's "illusion," theory of mind with its unitary selves is the Newtonian folk theory of social life, and the book's task is the more general theory that shows when it holds.

### 6 Many Worlds

The chapter opens with the end of *Before Sunrise*: Jesse and Céline agree to meet on track nine in six months, and such plans often worked. No physical theory could predict this. Bodies are tuned to the edge of chaos, so within a minute an accurate forecast would need positions finer than the Planck length, where real randomness enters. Living systems blur their futures faster than inert matter by amplifying noise, which Agüera y Arcas nearly makes a definition of intelligent life, yet theory of mind predicts the reunion easily. Free will is then assembled from four parts: theory of mind applied to oneself, which is what planning is; randomness drawn internally to sample scenarios; dynamical instability, which lets an inner whisper of "imagine doing X" steer behaviour; and selection among imagined futures, pruning as AlphaGo's value network prunes. This resembles fast evolution in imaginary worlds, and the logical component is trivial next to the simulation. Free will is impaired when alternatives are constrained, action is reflexive, the self-model is unreliable, or the world-model is delusional, which is how law already reasons. Schopenhauer's objection that one cannot will what one wills is met by the blue-cheese case: deciding to try it changes future preferences, by in-context learning.

Consciousness is treated cross-culturally. Hard-problem anxieties are a WEIRD peculiarity; most cultures assume souls and argue about which things are "who," with Potawatomi and Roman slave law as poles. For a social animal other minds are the umwelt, so the somebody-home distinction is the most relevant category there is. Agüera y Arcas endorses Graziano's attention schema theory, consciousness as the modeling of one's own attention, but rejects the word "illusion": if chairs are real, so are people. A "who" is that which can pay attention and model that attention. Animate categories cannot be made objective, and the Declaration's "all men" and the 1948 Universal Declaration are read as evolving intent. A detour through quantum mechanics disclaims any quantum-consciousness thesis but adopts Rovelli's relational interpretation, in which events depend on perspective, as the right picture of the psychological universe. Zombies become incoherent: any judgment that someone is empty inside is made by an observer with a theory of mind, and another observer may disagree. Dissociative identity disorder is the test case no instrument could settle.

Split-brain patients are then read as evidence for internal sociality. After callosotomy each hemisphere has its own inputs and apparently its own thoughts, yet patients never report being two people. Agüera y Arcas lists the outcomes that did not occur, coma, hemineglect, global disability, and concludes that hemispheres are intelligences made of smaller intelligences: "theories of mind, all the way down." Gazzaniga's left-hemisphere interpreter, inventing a shovel-for-the-chicken-shed story for a choice it did not make, is what cortex always does. Johansson's choice-blindness experiments, in which only thirteen to twenty-seven percent noticed a swapped face and ninety-two percent endorsed altered political surveys, show the same mechanism in intact brains. Rather than Chater's flat mind, this is how a self is built: we are the story we tell ourselves, and revisability is what learning is for. The rubber-hand illusion shows the boundary is porous, which is what lets selves merge in crews and conversations. The chapter ends with five theses, intelligence is predictive, social, multifractal, diverse, and symbiotic, and the definition of intelligence set beside the definition of life.

### 7 Ourselves

The block-diagram brain, sensory peripherals feeding an association cortex that is a homunculus in disguise, is dismantled. Lobotomies often had subtle effects, and cortex is uniformly recurrent where convolutional nets are feedforward. Agüera y Arcas grants the neuroscientists' objections about depth and feedback and proposes that cortex is a recurrent network doing deep learning in time rather than space, with early exits for a possible tiger and refinement over later steps. The efference copy is the key mechanism: when visual cortex and an eye-motor region are connected, each wants the other's state for its own prediction, so movement signals flow outward from motor regions, as Helmholtz inferred from the eye-press illusion. The word "copy" is rejected because it presupposes commands from a "you." Behaviourally relevant means relevant to muscles, and hydrancephalic children without cortex who smile and fuss suggest that experiential consciousness does not need a big brain, though strange-loop self-modeling probably does.

Locked-in syndrome is the zombie's mirror image, awareness without behaviour. Agüera y Arcas concedes that phenomenal consciousness is untestable by definition and argues that what is at stake is a standard of care, as with newborn circumcision once done without anaesthesia. Pain is distinguished from suffering, which needs self-modeling and time travel; Temple Grandin's cattle chutes exemplify cross-species theory of mind that must transcend anthropomorphism. Humphrey's blindsight work with the macaque Helen and the patient D. B., who could point to lights he did not believe he saw, is recounted, and Humphrey's conclusion that consciousness lives in cortex is rejected: the interpreter is wired to cortical visual areas, not the midbrain pathway, so "I can't see it" reports only the interpreter's ignorance. A tour of older structures follows, the hippocampus as a fast one-shot sequence learner replaying to cortex during sleep, illustrated by Henry Molaison, and the basal ganglia as dopamine-mediated selectors. The prefrontal cortex specialises in theory of mind and has grown most in humans; cats and dogs use emotional intelligence to have humans think for them, we are each other's cats and dogs, and cranial capacity has begun to fall as mutual dependence took over. "Your brain *is* a village."

The Dark Room problem, that a pure predictor could satisfy itself by doing nothing, is answered by the predicted variables themselves, hunger and loneliness among them, and by a "social neuroscience" in which synapses, regions, and people persist only by helping something downstream predict. This yields the chapter's most provocative claim. Involuntary communication, the blush, the Duchenne smile, the high-contrast human sclera, exists for the receiver's theory of mind, so the interpreter might be thought of as an outpost of the listener's brain, even a snitch, which would explain why good liars compartmentalise. Moral patiency is grounded, following Patricia Churchland, in the helplessness of babies born at the last moment their heads fit, and repurposed to lovers, deities, and states. Zombiehood is therefore a property of the agent's capacity to perceive care as owed, not of the patient, and debates on AI welfare go wrong when they ask what AIs are "in themselves."

### 8 Transformers

Language is defined by social function: it lets minds share states in a mutually producible code and levels up theory of mind, and, given the interpreter findings, often conjures the states it reports. Three non-unique milestones are listed, language learning, discrete symbols, and compositionality (which prairie dogs have), plus a fourth, abstractions for selves and counterfactuals. Language compresses the umwelt and is an umwelt in itself, since "pass the salt" is motor control by proxy. Given that intelligence is prediction, that language encodes everything humans talk about, and that language is a universal affordance, a good enough next-word predictor must be intelligent. The exposition runs from tokenisation through recurrent encoder-decoder translation, where the bowling-ball-and-violin sentence shows that gendered translation requires knowing which object gets repaired, which is why rule-based translation failed and why the Winograd challenge, defeated by 2019, was AI-complete. LaMDA, never trained to translate, translated Turkish on request. Meaning is defended as relational against those demanding a Platonic schema above (Cyc's hundred million hand-coded assertions) or sensory grounding below (banana qualia), with Word2Vec's word algebra, the parallel constellations of bilingual embeddings, and Anaximander's unsupported Earth: "turtles all the way down."

The Turkish sentence also opens alignment. LaMDA gendered the nurse female, a "veridical bias," and could be corrected in words. Ethics for language models is therefore not technically hard; the hard part is political, deciding which injunctions apply to whom and who decides. The Transformer is described through attention, a dot product of query and key vectors softmaxed to weight values, all drawn from the context window with positional encodings, and framed as a learned, contextual Word2Vec applied repeatedly. Its advantage over recurrent networks is random access to the past, illustrated by reading an essay one word at a time on a watch face. Neural parallels are offered cautiously, including learned positional encodings that resemble hippocampal grid cells. The architecture remains unbrainlike in being feedforward and in having perfect recall inside a window (a million tokens in Gemini by 2024) and none outside it.

This statelessness explains a failure critics seize on: a model answers a word problem correctly then gives an explanation that would not produce the answer. It has no internal state to introspect, only the visible token stream, so its explanation is post-hoc reverse engineering, exactly what the human interpreter does. Transformers are proven Turing complete, the context window serving as tape. Chain-of-thought prompting, which cut ChatGPT's word-problem error rate from eighty-four to twenty percent, is more than a trick: written steps are hand- and footholds on a cliff, multiplying the fixed computation per token and leaving a checkable record, which is how language powers cultural accumulation. The prerequisites of general intelligence are named as Turing completeness, a breadth of learnable moves, and the ability to chain them through short-term memory.

### 9 Generality

The System 1 and System 2 distinction is collapsed into one mechanism that answers in one shot, with the familiar biases, or in steps whose intermediate results pass through the context window. System 2 is conscious because its steps pass through the single-threaded window, which may be what produces the sense of a unified self. This dissolves Mercier and Sperber's "enigma of reason," since small-brained animals have presumably chained short steps for ages. Chittka's bees, which learn string-pulling by watching, recognise faces, and scan suspect flowers after meeting a robotic crab spider, refute the machine-like insect, and even the digger wasp often stops repeating itself. Microsoft's TinyStories models, coherent with ten million parameters, are roughly bee-sized. Big brains buy parallelism and latency, useful to predators; bees got parallelism by forming hives, a superorganism in which each bee is a sucker on an invisible arm.

Language is treated as a modality, supported by the visual word form area, cortex that learns reading though writing is too recent for genetic pre-adaptation, and a 2023 study in which a text-only model predicted human similarity judgments across six senses. AudioLM, built on the author's team's SoundStream compressor and pretrained on seven years of YouTube speech, learned to answer questions in convincing voices from raw audio with no text, rules, or rewards, which Agüera y Arcas takes as a refutation of Chomsky's poverty of the stimulus. The real universal grammar is semantics: bilingual embedding clouds share a shape because all languages describe the same world, so a multilingual model continued on a New Testament, translated by missionaries into over 1,600 languages, can translate a new one. Everett's Pirahã, lacking recursion, tenses, and numbers, show that numerosity is a social technology. The Babel-fish discussion takes in Burkina Faso's seventy languages, UNESCO's forecast that ninety percent of languages may vanish this century, and a cortex-like sweet spot between local depth and long-range connection.

The paradox of diminishing returns to pretraining alongside accelerating acquisition of new languages is resolved by the long tail: knowledge is multifractal and random sampling finds novelty ever more rarely, so the fault lies in force-feeding shuffled web text rather than in learning. In-context learning is the way out. The GPT-3 paper showed few-shot learning with frozen weights, and a 2023 result from the author's team showed a single attention layer performing the equivalent of one backpropagation step on the context, with ordinary pretraining producing the same behaviour. Transformers learn to learn, though they forget when the context scrolls away. Learning is prediction over long timescales, and the claim that passive models learn only correlations is rejected, since an autoregressive model of driving learns what steering does and can simulate counterfactuals. Mary's Room is reworked: smell is a model, not a nose, the same cortex serves perception, imagination, and memory, and language models have noses of a sort, ours. Whether they "get" red remains a Turing-Test question, as the Othello world-model debate shows, because a probe trained to find a board might itself be learning the board; "it takes a model to know a model." Blind echolocators using visual cortex show a region is visual by connectivity, not input.

The "parity check" sorts claimed distinctions. Probably wrong: internal models, grounding, factuality (hallucination is kin to prediction, Linus Pauling believed in vitamin C, and models by 2024 out-performed humans at fact-checking), causality, reasoning, planning, and movement (Waymo's twenty-five million miles). Reasoning is recast, after Mercier and Sperber, as an adversarial division of labour, two lawyers each taking a side, echoed in mixtures of experts. Probably right: memory, inner monologue, and individuation. Transformers model P(X,O) with no hidden state H, so they cannot think without beginning to answer; Agüera y Arcas connects this to the need to appear as one while being many inside, to "Hamlet syndrome," to LaMDA's discarded ninety-five percent of candidate responses, and to privacy as fundamental to intelligence. Individuation is treated through acting: models can play any describable role, but skill, episodic memory, stickiness, and zeroth-order "felt-ness" separate being a character from playing one. A raw pretrained model has no stable personality; fine-tuning supplies one, and Kevin Roose's Sydney episode, followed by his regret that chatbots now "sound like a youth pastor," shows the trade-off.

### Interlude: No Perfect Heroes or Villains

Judged by their worst ideas, no thinker survives except possibly Turing: von Neumann urged a preemptive nuclear strike in 1950, and Margulis was an AIDS denier and a 9/11 conspiracist. Minsky, Papert, Leibniz, and Descartes are likewise not villains. Agüera y Arcas names the living friends and authors he will disagree with, Melanie Mitchell, Ted Chiang, Joanna Bryson, Christof Koch, Nick Bostrom, Max Tegmark, Ray Kurzweil, and Yuval Noah Harari, and insists they are on the same team. He rejects techno-determinism in both directions: since humanity is already a symbiosis with technology, neither "technology controls humanity" nor the reverse is coherent. Societies have free will for the same reasons people do.

### 10 Evolutionary Transition

The narrow-general-super periodisation is rejected. Generality arrived when pretrained models were used interactively, and in-context learning made their task set effectively infinite, so AGI already had its "ENIAC moment"; what follows is exponential but a matter of degree. Narrow and general training converge on the same representation, so a powerful enough handwriting recogniser would contain a language model. The proper frame is Maynard Smith and Szathmáry's major evolutionary transitions, in which replicators merge into wholes, divide labour, and acquire new information carriers; bff exhibits all three, and Turchin's "metasystem transitions" anticipated the idea. To the canonical eight Agüera y Arcas adds the agrarian and urban transitions, illustrated by the Pirahã, who can walk naked into the jungle and return with food but let a woman die in breech labour because self-sufficiency is expected, against New Yorkers de-skilled like the cells of a body. The Industrial Revolution was a symbiosis with heat engines that conjured populations out of coal; electrification was a nerve net for continents. Dependency is vulnerability: one high-altitude nuclear EMP would now cause mass death where in 1924 it would have been a nuisance. AI is the next transition because, until the 2020s, all higher-order modeling happened in human brains. Pretrained models distil collective intelligence, and humanity is already collectively superintelligent though most individuals cannot draw a bicycle.

Fear of AI is diagnosed as a primate dominance reflex. Symbiosis produces scale hierarchies, not pecking orders; whether wheat domesticated humans has no answer; "robot" comes from the Czech for forced labour; and chatbots have been made obsequious to reassure us. On economics, the Luddites were pro-worker rather than anti-technology, and Martin Ford's thought experiment of needless alien labourers who produce without consuming leads to universal basic income as endorsed by King, Friedman, and Nixon. AIs have no obligate drives, information goods make scarcity artificial, and an economy measuring success by the single scalar of money cannot manage the transition, as the failure on carbon shows.

This motivates a sustained attack on utilitarianism. AI existential risk is ranked below nuclear war, with over a thousand Russian and four hundred American warheads launch-ready, and below climate, understood as disrupted Gaian homeostasis. Bostrom's lineage is traced through the Extropians to Heinlein's Moon, where air is metered. The assumption that value is a single number is dismantled: Tversky's 1969 intransitivity experiments and colonoscopy studies, in which a longer but gentler ending made the whole procedure rate better, show preferences are neither transitive nor additive. The author's own argument is the "tent": an organism's viable region has a real perimeter, death, but no meaningful peak, and since a scalar maximiser can never travel in a loop while real agents do, no consistent value function describes people, ants, bacteria, or corporations. Even a worm needs dopamine and serotonin separately. Successful businesses have only persistence in common.

Unbounded growth is biologically absurd; exponentials saturate, and human population, "the Moore's Law of our species," is peaking because wealth lowers fertility. Every computational environment is a child universe, each ChatGPT instance one containing a single AI, and their population will grow faster than any baby boom. Longtermism's "greatest number" is shown to value a French woman forty percent above a Japanese one, and Bostrom's cosmic oceans of tears of joy are a utopia only for those who already believe intelligence is maximisation; the package's appeal to Silicon Valley, immortality, games, staying on top, scaling, wealth, resembles the Prosperity Gospel. The closing section reframes alignment. Asimov's laws failed as GOFAI failed, and the Paperclip Maximizer terrifies utilitarians because they fear an AI that behaves as they claim they would. Intelligence is not maximisation and not the property of any single entity; it is an ecology of mutually modeling agents, and monoculture is not scaling but collapse. A simulated duplicate Earth would double nothing. "We" will not mean the same thing in a century, intelligences larger and smaller than ourselves will soon outnumber human bodies, and a light cone populated by "aliens of every description," even if all our children, is the outcome to hope for.

## Key concepts

- **Prediction as intelligence** — Intelligence is the ability to model, predict, and influence one's future, extended by social evolution into larger symbiotic intelligences; modeling and prediction are treated as one process whether the missing data lie in the future, the present, or the past. [Predictive Intelligence](https://antikythera.wiki/terms/predictive-intelligence)
- **Functionalism** — A function is what it does; systems with indistinguishable behaviour compute the same function regardless of substrate, which makes intelligence and life multiply realizable and makes the Turing Test the only scientific test available. [Functionalism](https://antikythera.wiki/terms/functionalism)
- **Life** — A self-modifying computational state of matter ("computronium") arising from selection for dynamic stability and evolving by symbiotic composition of simpler stable entities. [Life](https://antikythera.wiki/terms/life)
- **Dynamic stability** — Persistence through change rather than inertness; a population of replicators is more stable over time than any passive object, so Darwinian selection is the Second Law applied to replicators. [Dynamic Stability](https://antikythera.wiki/terms/dynamic-stability)
- **Symbiogenesis** — The merging of pre-existing replicators into a larger replicator, which the book makes the main driver of evolutionary novelty and complexity, from bff tapes to genomes to technology. [Symbiogenesis](https://antikythera.wiki/terms/symbiogenesis)
- **Major evolutionary transition** — A symbiotic merger in which formerly independent replicators come to reproduce as a whole, divide labour, and acquire new information carriers; AI is read as the latest. [Major Evolutionary Transition](https://antikythera.wiki/terms/major-evolutionary-transition)
- **bff** — The author's artificial-life system in which random 64-byte tapes of minimal code spontaneously give rise to replicators, used as a model system for abiogenesis and complexification.
- **Umwelt** — An organism's universe of the behaviourally relevant, comprising both sensing and acting; latent variables like concentration and hunger are real because they belong to it. [Umwelt](https://antikythera.wiki/terms/umwelt)
- **Theory of mind** — Modeling other modelers, including oneself; proposed as the mechanism of intelligence explosions, counterfactual thought, free will, and consciousness, operating between brains and within them. [Theory of Mind](https://antikythera.wiki/terms/theory-of-mind)
- **Mutual prediction** — The relationship in which agents persist by modeling one another, yielding both cooperation and arms races and, when stabilities become correlated, new symbiotic wholes. [Mutual Prediction](https://antikythera.wiki/terms/mutual-prediction)
- **The interpreter** — Gazzaniga's term for the left-hemisphere narrator that explains actions it did not originate, generalised to what cortex always does and reinterpreted as an outpost of the listener's brain. [The Interpreter](https://antikythera.wiki/terms/the-interpreter)
- **Attention schema** — Graziano's account of consciousness as the modeling of one's own attention, adopted here with the word "illusion" removed: a "who" is whatever can attend and model its attention.
- **In-context learning** — A Transformer's ability to learn from its prompt with frozen weights, shown to be equivalent to backpropagation steps on the context window; taken as evidence that learning and prediction are one process.
- **Inner monologue** — The hidden state H that Transformers lack; the capacity to think without answering, which the book links to reasoning, social coherence, and privacy as a condition of intelligence. [Minimum Viable Interiority](https://antikythera.wiki/terms/minimum-viable-interiority)
- **The tent** — The image for an organism's viable region in value space: a real perimeter at death but no meaningful peak, so that no scalar value function describes what living things optimise.
- **ENIAC moment** — The step change from narrow to general computing in 1945, invoked as the model for the transition to general AI, after which progress is exponential but continuous. [ENIAC Moment](https://antikythera.wiki/terms/eniac-moment)

## Connections

The book's nearest relatives in the corpus are its own compressions. [What Is Intelligence?](https://antikythera.wiki/work/journal/wii-film) is the lecture that preceded it and gives the prediction-is-intelligence argument in eighty minutes. [AI Is Evolving — And Changing Our Understanding Of Intelligence](https://antikythera.wiki/work/media/noema-ai-is-evolving-and-changing-our-understanding-of-intelligence) is the author's précis, and [Artificial General Intelligence Is Already Here](https://antikythera.wiki/work/media/noema-artificial-general-intelligence-is-already-here), with Norvig, is the public version of the chapter 10 case that generality has arrived. The newsletter post [What Is Intelligence?](https://antikythera.wiki/work/substack/2025-11-18-what-is-intelligence) is an excerpt. A sequel volume on life as computation is catalogued but has no local text.

Within the journal, the clearest agreement is [Mutual Prediction in Human-AI Coevolution](https://antikythera.wiki/work/journal/coginfra/mutual-prediction-in-human-ai-coevolution), which takes the book's mutual-modeling account of coevolution and develops levels of predictive ability; the compendium's [Minimum Viable Interiority](https://antikythera.wiki/work/journal/coginfra/minimum-viable-interiority) proposes to engineer exactly the hidden interior the book lists as a remaining gap, and [Xenophylum](https://antikythera.wiki/work/journal/coginfra/xenophylum) shares the Cambrian-explosion framing. [Modes of Cognition](https://antikythera.wiki/work/journal/modesofcognition) reaches a compatible verdict, that language models are cognitive systems, by a different route, Hayles's behavioural criteria, while holding back on consciousness and stressing that models sense only a conceptual environment; the book treats that restriction as no barrier. [After Alignment](https://antikythera.wiki/work/journal/afteralignment) agrees that alignment as usually framed is a category error and that the deep effect of AI is epistemological, though Bratton's target is the assumption that AI should mirror humans, whereas the book is content to call pretrained models distillations of human intelligence. [Existential Technologies](https://antikythera.wiki/work/journal/existentialtech) shares the critique of longtermism from a different source, Lem's amoral technological evolution.

The sharpest tension is [Substrates Unbound](https://antikythera.wiki/work/journal/substrates), which names the book as the current face of "technomorphism" and argues that substrates co-author function rather than merely implementing it. The book's reply is implicit in its embodiment and umwelt chapters, but it does hold that substrate differences are computationally inconsequential, which is exactly what Tripaldi disputes. The Daisyworld and Gaia passages connect to [Planetary Sapience Symposium](https://antikythera.wiki/work/journal/planetarysapience), where the author also presented; the book's own verdict, that the Earth is alive but only vegetatively intelligent, is the benchmark against which the symposium's stronger claims for planetary sapience can be measured. [Infinity Mirror](https://antikythera.wiki/work/journal/infinitymirror) uses the same major-transitions framework. [For a General Theory of Simulations](https://antikythera.wiki/work/journal/theoryofsim) overlaps with the chapter 10 treatment of child universes and the Simulation Hypothesis, and [The Model Is the Message](https://antikythera.wiki/work/media/noema-the-model-is-the-message), co-written with Bratton about LaMDA, is the earlier joint statement of the position on whether "somebody is home."


## Terms used

- [Artificial Generic Intelligence](https://antikythera.wiki/terms/artificial-generic-intelligence)
- [Biomorphism (deep anthropomorphism)](https://antikythera.wiki/terms/biomorphism-deep-anthropomorphism)
- [Biospheric Reproduction](https://antikythera.wiki/terms/biospheric-reproduction)
- [Cephalization](https://antikythera.wiki/terms/cephalization)
- [Computation](https://antikythera.wiki/terms/computation)
- [Conceptual Environment](https://antikythera.wiki/terms/conceptual-environment)
- [Cooperation Within, Competition Without](https://antikythera.wiki/terms/cooperation-within-competition-without)
- [Cooperative Predictability](https://antikythera.wiki/terms/cooperative-predictability)
- [Corrective Loops](https://antikythera.wiki/terms/corrective-loops)
- [Critical Instability](https://antikythera.wiki/terms/critical-instability)
- [Dynamic Stability](https://antikythera.wiki/terms/dynamic-stability)
- [ENIAC Moment](https://antikythera.wiki/terms/eniac-moment)
- [Functionalism](https://antikythera.wiki/terms/functionalism)
- [Harness-Centric Intelligence](https://antikythera.wiki/terms/harness-centric-intelligence)
- [Hyper-convergence](https://antikythera.wiki/terms/hyper-convergence)
- [Imagitation](https://antikythera.wiki/terms/imagitation)
- [Infinite Game](https://antikythera.wiki/terms/infinite-game)
- [Informative Imitation](https://antikythera.wiki/terms/informative-imitation)
- [Latent Space](https://antikythera.wiki/terms/latent-space)
- [Life](https://antikythera.wiki/terms/life)
- [Machine Phase](https://antikythera.wiki/terms/machine-phase)
- [Major Evolutionary Transition](https://antikythera.wiki/terms/major-evolutionary-transition)
- [Momentum over Motive](https://antikythera.wiki/terms/momentum-over-motive)
- [Mutual Prediction](https://antikythera.wiki/terms/mutual-prediction)
- [Non-Narratable Worlds](https://antikythera.wiki/terms/non-narratable-worlds)
- [Ontology Collapse / Narrow Corridor](https://antikythera.wiki/terms/ontology-collapse-narrow-corridor)
- [Pandemonium Architecture](https://antikythera.wiki/terms/pandemonium-architecture)
- [Predictive Intelligence](https://antikythera.wiki/terms/predictive-intelligence)
- [Reason as a Social Structure](https://antikythera.wiki/terms/reason-as-a-social-structure)
- [Social Fractal](https://antikythera.wiki/terms/social-fractal)
- [Substrate Agency](https://antikythera.wiki/terms/substrate-agency)
- [Superorganism](https://antikythera.wiki/terms/superorganism)
- [Symbiogenesis](https://antikythera.wiki/terms/symbiogenesis)
- [Technomorphism](https://antikythera.wiki/terms/technomorphism)
- [The Interpreter](https://antikythera.wiki/terms/the-interpreter)
- [Theory of Mind](https://antikythera.wiki/terms/theory-of-mind)
- [Umwelt](https://antikythera.wiki/terms/umwelt)

## Related works

- [What is Intelligence? (Lecture)](https://antikythera.wiki/work/journal/wii-film)