Fancy decision theories

1 Causal vs Evidential decision theory

Apparently, I should read this:

• Causal decision theory
• Evidential decision theory

A reflective variant of game theory worries about decision problems with smart predictive agents. Strong AI risk people are excitable in the vicinity of these.

Although their reading list is occasionally IMO undiscerning, you might want to start with MIRI’s intro which at least exists.

Existing methods of counterfactual reasoning turn out to be unsatisfactory both in the short term (in the sense that they systematically achieve poor outcomes on some problems where good outcomes are possible) and in the long term (in the sense that self-modifying agents reasoning using bad counterfactuals would, according to those broken counterfactuals, decide that they should not fix all of their flaws).

2 Updateless decision theory

• Open-minded updatelessness
• Updatelessness doesn’t solve most problems

3 Commitment races

Commitment Races are important in international relations and also seem popular in AI safety theory, although I am not sure why, since I don’t understand how AIs can credibly commit to things.