Human domestication

January 18, 2025 — March 3, 2025

agents
collective knowledge
distributed
economics
extended self
language
machine learning
mind
utility
Figure 1

Notebook on the idea of human domestication. Possibly the opposite of being stroppy.

Paul Christiano, What failure looks like

Amongst the broader population, many folk already have a vague picture of the overall trajectory of the world and a vague sense that something has gone wrong. There may be significant populist pushes for reform, but in general these won’t be well-directed. Some states may really put on the brakes, but they will rapidly fall behind economically and militarily, and indeed “appear to be prosperous” is one of the easily-measured goals for which the incomprehensible system is optimising.

Amongst intellectual elites there will be genuine ambiguity and uncertainty about whether the current state of affairs is good or bad. People really will be getting richer for a while. Over the short term, the forces gradually wresting control from humans do not look so different from (e.g.) corporate lobbying against the public interest, or principal-agent problems in human institutions. There will be legitimate arguments about whether the implicit long-term purposes being pursued by AI systems are really so much worse than the long-term purposes that would be pursued by the shareholders of public companies or corrupt officials.

We might describe the result as “going out with a whimper.” Human reasoning gradually stops being able to compete with sophisticated, systematised manipulation and deception which is continuously improving by trial and error; human control over levers of power gradually becomes less and less effective; we ultimately lose any real ability to influence our society’s trajectory.

cf Kulveit et al. (2025).

Predation as Payment for Criticism | Compass Rose

A predator’s success depends on exploiting specific flaws in prey—poor awareness, slow reactions, weak social coordination. Unlike resource competition or disease, which select based on physiological robustness, predation creates focused pressure on intelligence, pressure which favours the development of consciousness. A cheetah doesn’t catch just any gazelle, but the one that fails to notice and react effectively. Prey get better at evading, predators get better at predicting, each side has to model the other’s thoughts more accurately—an arms race that eventually rewards both sides for explicitly modeling the other’s perspective, the basis for cognitive empathy. If you’re glad you’re conscious, no need to thank a predator—somebody else already did—by being eaten.

Life on the Grid (part 1) - by Roger’s Bacon

To review: growing up in simplistic spatial environments and using GPS has given you brain damage and life has become a soul-crushing video game utterly devoid of mystery or adventure. We are trapped in the Grid like an insect in the spider’s web; vigorous struggle will only serve to entangle us further. To extricate ourselves, we must, as individuals, gently subvert the very foundations of the Grid, which is nothing external but a facet of human nature: the impulse towards control, the systematising instinct, the part of us that abhors anomaly and ambiguity and seeks to eradicate them. What begins as an earnest attempt to break free can so easily slip back into a self-imposed system of rules and practices with standards to be met and schedules to be followed. For that reason I am hesitant to provide any concrete suggestions—they will only serve to constrain your thinking. For now, there is perhaps only one thing that can be said: if you want to get off the grid, then get lost.

Enrichment & Animal Welfare

Environmental enrichment, also known as behavioural enrichment, provides species-appropriate challenges, opportunities and stimulation. Environmental enrichment includes the regular provision of dynamic environments, cognitive challenges and social opportunities. An enriched environment should promote a range of normal behaviours that animals find rewarding as well as allowing animals to positively respond to potential stressors. For example, opportunities to hide or climb away from visitors or more dominant co-specifics.

1 References

Baber. 2010. Society: The Rise of the ‘Technium’.” Nature.
Bowles, Choi, and Hopfensitz. 2003. The Co-Evolution of Individual Behaviors and Social Institutions.” Journal of Theoretical Biology.
Boyd, and Richerson. 1999. “Complex Societies: The Evolutionary Origins of a Crude Superorganism.” Human Nature.
Choi, and Bowles. 2007. The Coevolution of Parochial Altruism and War.” Science.
Kenton, Kumar, Farquhar, et al. 2023. Discovering Agents.” Artificial Intelligence.
Kulveit, Douglas, Ammann, et al. 2025. Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development.”
Moral Sentiments and Material Interests: The Foundations of Cooperation in Economic Life. 2006.
Morris. 2014a. The Measure of Civilization: How Social Development Decides the Fate of Nations.
———. 2014b. War: What is it good for?: The role of conflict in civilisation, from primates to robots.
———. 2015. Foragers, Farmers, and Fossil Fuels: How Human Values Evolve. Edited by Stephen Macedo.
Muthukrishna. 2023. A Theory of Everyone: The New Science of Who We Are, How We Got Here, and Where We’re Going.
Richens, and Everitt. 2024. Robust Agents Learn Causal World Models.”
Scott. 2017. Against the Grain: A Deep History of the Earliest States.
Turchin. 2015. Ultrasociety: How 10,000 Years of War Made Humans the Greatest Cooperators on Earth.
Ward, Francis Rhys, MacDermott, Belardinelli, et al. 2024. The Reasons That Agents Act: Intention and Instrumental Goals.”
Ward, Francis, Toni, Belardinelli, et al. 2023. Honesty Is the Best Policy: Defining and Mitigating AI Deception.” In Advances in Neural Information Processing Systems.