The three pilots problem for CDT
In a comment on an old blog post, a person called "D" brought up a nice puzzle for Causal Decision Theory. Here's (my version of) the scenario.
You have just taken over as one of three pilots on a spaceship that is on its way to Betelgeuse. The spaceship's flight operations are largely automatised. The only input needed from the pilots is the destination. At present, the only available destinations are Betelgeuse and a service station on a nearby moon. (Other destinations could not be safely reached with current fuel levels.)
There's a low chance that a pilot gets hit by cosmic rays, which would cause them to issue random commands. That's why the spaceship is controlled by three pilots, operating independently. The spaceship changes its course only if at least two of its pilots enter a new destination (and they agree on the new destination).
If one of the pilots got hit by cosmic rays then it would be best to divert to the service station for medical treatment. If nobody has been hit, it would be best to continue to Betelgeuse.
All of this is common knowledge between you and the other pilots. (You also know that you have not been hit by cosmic rays.) Should you vote to divert to the service station or to continue to Betelgeuse?
CDT appears to say that you should vote 'divert'. Your vote will be effective only if the other pilots issue different commands: if one of them votes 'continue' and the other 'divert'. Since all three pilots have the same information and the same goals, they would issue different commands only if one was hit by cosmic rays. So your vote is effective only if it is best to divert. So you should vote 'divert'.
If the spaceship is operated by CDT pilots, then it looks like it will always divert to the nearest service station, even though there is almost never any need for medical treatment. EDT pilots, by contrast, would safely get the spaceship to Betelgeuse.
I like this puzzle, because it feels like a Prisoner's Dilemma, and yet everyone's incentives are aligned.
So. Does CDT really say you should vote 'divert'? There's a tiny loophole.
'Divert' dominates 'continue' if the only way in which the other pilots could issue different commands is that one of them has been hit by cosmic rays. The loophole is that the other pilots could rationally issue different commands even though neither has been hit. This is represented by the third column in the following decision matrix.
| ¬H&CC | ¬H&DD | ¬H&CD | H&CC | H&DD | H&CD | |
|---|---|---|---|---|---|---|
| C | 10 | 0 | 10 | 0 | 1 | 0 |
| D | 10 | 0 | 0 | 0 | 1 | 1 |
('¬H&CC' means nobody has been hit and the other pilots both vote 'continue'. '¬H&DD' means nobody has been hit and the other pilots both vote 'divert'. '¬H&CD' means nobody has been hit and the others disagree.)
Assuming all pilots obey CDT, the ¬H&CD state (the third column) has positive probability only if CDT permits both actions. And if the third column has positive probability then CDT may well permit both actions. This is the loophole.
Suppose you think that (on the assumption that nobody has been hit by cosmic rays), each of the other pilots is equally likely to vote for 'continue' and 'divert'. Then your credence in the '¬H&CD' state is roughly 1/2, and you should vote 'continue'.
But could you rationally think that the others are equally likely to vote for 'continue' and for 'divert'? If this were rational for you, you should think it's rational for the others. And since you know that the others are rational, you should then expect them to vote 'continue'. Contradiction.
We need a deliberation equilibrium.
Let's find the Nash equilibria in the three-player game – but taking into account that a player who is hit by cosmic rays is fixed to choose either option with equal probability. There are, I think, three equilibria.
One is where everyone who is rational (not hit by rays) votes 'divert'. In this situation, the '¬H&CD' state has probability 0 and 'divert' dominates among the states with positive probability.
The other equilibria are mixed. Where they lie depends on the probability that someone has been hit by cosmic rays. Let's say that probability is 0.01.
Then a second equilibrium lies at a point where rational players votes 'continue' with probability 0.9995 and 'divert' with probability 0.0005. At the third equilibrium, 'continue' has probability 0.0005 and 'divert' 0.9995.
In either case, the '¬H&CD' state is highly improbable, but just probable enough to balance out the impact of the 'H&CD' column, so that both pure options have equal expected utility.
How should we interpret these three equilibria?
If we say that a rational player may aim for any one of them, then you don't know if the other players (assuming they aren't hit) will (1) both vote 'divert', or (2) both choose the mixed act [0.9995 'continue', 0.0005 'divert'], or (3) both choose the mixed act [0.0005 'continue', 0.9995 'divert'], or (4) choose different combinations of these three options. Depending on how you divide your credence among all these possibilities, you should either vote 'continue' or 'divert', or choose a randomised act. But the chances that the two pure options have equal expected utility are negligible, so you actually shouldn't randomise. And if you are almost certain not to randomise, you can't really believe that the others will. Your credence in the '¬H&CD' state should be extremely low. And so you should vote 'divert'.
But suppose we say (as I think I want) that in a decision problem with multiple deliberation equilibria, rational players have to choose the best equilibrium. In the three pilots case, the best equilibrium is the one where rational players vote 'continue' with probability 0.9995. If you know that this is what rationality requires, you can infer that the other players will do it as well (provided they haven't been hit by cosmic rays). And then the equilibrium is stable. You should do the same.
If the pilots follow a "best-equilibrium" version of CDT, they can squeeze through the loophole. Everyone votes 'continue' with probability 0.9995. The spaceship is almost certain to arrive at Betelgeuse.