Alignment Overfitting

Bratton's name for the error of treating short-term "alignment" — tuning models to cohere with stated human preferences and self-image — as the long-term metaphysical grounding for how animal intelligence should orient machine intelligence. Lower-case alignment is a legitimate engineering tactic (it "makes it work"); upper-case Alignment, overfit to "dubiously defined human values," is itself a kind of existential risk because of what it directs attention from: the Copernican disclosures AI could make about what cognition and "we" actually are. The remedy is bidirectional alignment — AI to culture's wisdom, culture to AI's disclosures — see Productive Disalignment.