Gradient steps to an ecology of mind

1 Loss functions versus survival functions

“Entities that optimize for goals, above all,” versus “entities that replicate and persist, above all.” Two different models for adaptive entities.

As an evolutionary replicator, myself, I am inclined to fear optimizers, and wonder how our interests can actually align.

There are non-optimising paradigms for AI (Lehman and Stanley 2011; Ringstrom 2022). Also works like Arcas et al. (2024) suggest that replicating is itself a way of producing entities.

2 Consciousness

Is subjective continuity a convenient way of getting entities to invest in their own persistence?

3 Incoming

Neural Annealing: Toward a Neural Theory of Everything.

• Gradient Dissent, a blog of reasons that large backpropagation-trained networks might be worrisome. A mixed bag, combining insightful ideas and hyperbole.

  Question: If it were true that there are externalities from backprop networks (i.e. that they are a kind of methodological pollution that produces private benefits but public costs) then what kind of mechanisms should be applied to disincentivise them?

• C&C Against Predictive Optimization

• Gordon Brander, Coevolution creates living complexity

• Orthogonality Thesis - LessWrong

Professor Javen Qinfeng Shi says:

Mind is a choice maker. Choices shape the mind

• Q learning: do what a good/kind person would do (moment to moment), learn wisdom (V function) and have faith in future and self-growth. It naturally leads to optimal long term accumulative rewards (Bellman equation)
• Policy gradient: learn from past successes (to repeat or mimic) and mistakes (to avoid). Require complete episodes to reveal the end accumulative reward per episode

This is the first time I have heard of policy gradient as utilitarianism versus Q learning as virtue ethics.

4 References