Human domestication

Notebook on the idea of human domestication. Possibly the opposite of being stroppy.

Paul Christiano, What failure looks like

Amongst the broader population, many folk already have a vague picture of the overall trajectory of the world and a vague sense that something has gone wrong. There may be significant populist pushes for reform, but in general these won’t be well-directed. Some states may really put on the brakes, but they will rapidly fall behind economically and militarily, and indeed “appear to be prosperous” is one of the easily-measured goals for which the incomprehensible system is optimising.

Amongst intellectual elites there will be genuine ambiguity and uncertainty about whether the current state of affairs is good or bad. People really will be getting richer for a while. Over the short term, the forces gradually wresting control from humans do not look so different from (e.g.) corporate lobbying against the public interest, or principal-agent problems in human institutions. There will be legitimate arguments about whether the implicit long-term purposes being pursued by AI systems are really so much worse than the long-term purposes that would be pursued by the shareholders of public companies or corrupt officials.

We might describe the result as “going out with a whimper.” Human reasoning gradually stops being able to compete with sophisticated, systematised manipulation and deception which is continuously improving by trial and error; human control over levers of power gradually becomes less and less effective; we ultimately lose any real ability to influence our society’s trajectory.

Predation as Payment for Criticism | Compass Rose

A predator’s success depends on exploiting specific flaws in prey—poor awareness, slow reactions, weak social coordination. Unlike resource competition or disease, which select based on physiological robustness, predation creates focused pressure on intelligence, pressure which favors the development of consciousness. A cheetah doesn’t catch just any gazelle, but the one that fails to notice and react effectively. Prey get better at evading, predators get better at predicting, each side has to model the other’s thoughts more accurately—an arms race that eventually rewards both sides for explicitly modeling the other’s perspective, the basis for cognitive empathy. If you’re glad you’re conscious, no need to thank a predator—somebody else already did—by being eaten.

Life on the Grid (part 1) - by Roger’s Bacon

To review: growing up in simplistic spatial environments and using GPS has given you brain damage and life has become a soul-crushing video game utterly devoid of mystery or adventure. We are trapped in the Grid like an insect in the spider’s web; vigorous struggle will only serve to entangle us further. To extricate ourselves, we must, as individuals, gently subvert the very foundations of the Grid, which is nothing external but a facet of human nature: the impulse towards control, the systematising instinct, the part of us that abhors anomaly and ambiguity and seeks to eradicate them. What begins as an earnest attempt to break free can so easily slip back into a self-imposed system of rules and practices with standards to be met and schedules to be followed. For that reason I am hesitant to provide any concrete suggestions—they will only serve to constrain your thinking. For now, there is perhaps only one thing that can be said: if you want to get off the grid, then get lost.