Preference Reflection (EDC, ch.7, part 2)
Why should you take both boxes in Newcomb's Problem? The simplest argument is that you are then guaranteed to get $1000 more than what you would get if you took one box. A more subtle argument is that there is information – about the content of the opaque box – of which you know that if you had that information, then you would rationally prefer to take both boxes. Let's have a closer look at this second argument, and at what Arif says about it in chapter 7 of Evidence, Decision, and Causality.
The argument is sometimes presented in terms of an imaginary friend. Imagine you have a friend who has inspected the content of the opaque box. No matter what the friend sees in the box, she would advise you to two-box. You should do what your better-informed friend advises you to do. In the original Newcomb scenario, you don't have such a friend. But you don't need one, for you already know what she would say.
I find this argument persuasive. The general principle behind it might be expressed as follows.
General Preference Reflection: If you prefer A to B conditional on every possible answer to some question Q, then you should unconditionally prefer A to B.
General Preference Reflection is related to dynamic consistency. Suppose you have to choose between A and B. After registering your choice, you are told the answer to Q. You are then given an opportunity to revise your earlier choice, for a small fee. If you violate General Preference Reflection then you will initially prefer the plan A-and-don't-switch over all other plans. But you can't implement that plan. If you chose A in the first stage, you will switch to B in the second.
Another motivation for General Preference Reflection turns on the fact that we can evaluate an agent's options not only relative to their actual beliefs, but also relative to merely hypothetical beliefs (but holding fixed the agent's basic desires). In particular, we can consider what the agent should do in light of all relevant facts. An act that is best in light of all the facts is "objectively best", in some sense of that term. Often, we don't know which of our options are objectively best. That's why we need to go by the credence-weighted average of the goodness of our options. If, however, we do know that a certain option is objectively best then we don't need to bother with that.
Special Preference Reflection: If you know that A is objectively better than B then you should prefer A to B.
As Spencer and Wells (2019) point out, Special Preference Reflection directly supports two-boxing in Newcomb's Problem. Here two-boxing is best in light of all the facts. Your imaginary friend who knows what's in the opaque box plausibly knows all relevant facts about your situation. Whatever else she knew about the world, she would still think two-boxing is (or would be) the better choice.
Special Preference Reflection is a special case of General Preference Reflection, where the question Q asks about all (relevant) facts.
Think of subjective and objective rightness as endpoints of a multidimensional scale. At one end, we have rightness in light of your actual information. At the other end, we have rightness in light of your information plus all other facts. In between, there is rightness in light of your information plus some other facts. Special Preference Reflection says that if your information already settles that A is the right choice from the perspective of the objective endpoint then your ignorance makes no practical difference: A is also best relative to your limited information. Similarly, General Preference Reflection says that if your information settles that A is the right choice from the perspective of some point towards the objective endpoint then the information difference between your point and that point makes no practical difference for your current choice.
EDT invalidates General Preference Reflection – as one can see in Newcomb's Problem. It is not clear whether EDT invalidates Special Preference Reflection, since it is not clear whether EDT can even make sense of objective rightness. If you know all the facts, including what you will do, then the conditional probabilities that figure in the definition of evidential expected utility might become undefined. Arif Ahmed, at least, thinks that there is no such thing as objective rightness. This is mentioned at several points in Ahmed (2014), and stated more explicitly in Ahmed and Spencer (2020).
Even if one rejects Special Preference Reflection as unintelligible, one could still accept General Preference Reflection for more mundane questions Q. Friends of EDT need to explain what is wrong with that principle.
One might have thought that General Preference Reflection (for mundane questions Q) holds even if preference is understood in terms of news value. On this reading, General Preference Reflection is closely related to the principle of "Desire Reflection" proposed in Arntzenius (2008). Informally, that principle says that if a certain proposition is desirable conditional on every answer to some question Q, then it is also unconditionally desirable.
But that principle is demonstrably false. Arif gives a nice counterexample involving Simpson's Paradox. I gave a simpler counterexample in this blog post. On reflection, it is clear that General Preference Reflection fails for "news value preferences".
When I introduced the Reflection principles, I didn't assume a "new value" reading of preference. By 'you prefer A to B', I meant that you regard A as more choiceworthy than B. So understood, the principle still looks highly plausible to me. That it fails for news value preference only shows that comparisons in terms of news value are not the same as comparisons in terms of choiceworthiness.
Can friends of EDT say something against Preference Reflection that does not presuppose an evidentialist reading of preference?
Arif argues that the principle has intuitive counterexamples. Consider the following scenario (adapted from p.197).
Armour. You have the option to buy a suit of armour ahead of tomorrow's battle. Alternatively, you can spend the money on booze. You believe that the armour would strongly increase your chance of survival, and you greatly value surviving, much more than you value booze.
You should obviously buy the armour. But now imagine that before you make your choice, a time-traveller swings by and tells you whether you will survive the battle. According to Arif, it is clear that whatever she tells you, you should not buy the armour. The reasoning goes as follows.
Suppose the time-traveller informs you that you will die. Then buying the armour is a waste of money. You know that you'll be dead by tomorrow evening, so you might as well spend the money on booze. Suppose, alternatively, that the time-traveller informs you that you will survive. Then, too, buying the armour is a waste of money. You no longer need to fear that you will die in the battle. You know you'll survive. Why waste your money on armour?
If we endorse this "intuitive" line of thought, we have a counterexample to General Preference Reflection. You unconditionally prefer to buy the armour, but you prefer to buy the booze conditional on all answers to 'will I survive?'. As a corollary, we also have a reason to distrust the argument for two-boxing with the imaginary friend. For we then have a case in which one should not do what one knows a better-informed adviser would advise us to do.
Now, I can see that some people might be persuaded by the above argument about what you should do once you hear the time-traveller's report. I myself don't find the argument at all persuasive. I suspect that intuitions about this case will split along the same lines as they split about Newcomb's Problem. If that's true, we have made no progress. We have an "intuitive counterexample" to Preference Reflection, but only for those whose intuitions align with one-boxing. We already had an example of this kind: Newcomb's Problem.
What should you do after you hear the time-traveller's report? It depends on what she tells you.
Suppose she tells you that you will die. If you decide not to buy the suit of armour – as Arif intuits you should – you can then be confident that you'll enter tomorrow's battle without a suit of armour, and that you will be killed. How will you be killed? You don't know. But much of your credence should go to scenarios in which you will die in an attack that you could have survived with a suit of armour. More formally, conditional on the assumption that you won't wear armour and that you will die, you should give high credence to the hypothesis that you will die in an attack that you would have survived with a suit of armour. Given these beliefs, can you really rest content with not buying the armour? That doesn't sound intuitive to me at all. You should think that the decision to not buy the armour is a mistake. And so you shouldn't make that decision.
CDT agrees. According to CDT, the information that you will die puts you in an unstable decision problem. If you're confident that you won't buy armour, it will seem better to buy armour. If you're confident that you will buy armour, it will seem better to not buy armour. In that second case, you are confident that you will die in battle, wearing the suit of armour. If you hadn't bought the suit of armour, you would probably still have died. But there's some chance you might have survived. And at any rate you would at least have enjoyed the booze.
What if the time-traveller tells you that you won't die? Suppose you're confident that you won't buy the armour. Then you're confident that you'll survive tomorrow's battle despite not wearing armour. Would it have been better to buy armour? It depends. If you take into account the possibility of injury, it arguably would. For then you might give high credence to surviving the battle with injuries that could have been prevented with a suit of armour. But let's stipulate that you only care about survival and booze. You don't care about injuries. Then you should arguably rest content with your decision not to buy the armour.
Suppose, alternatively, that you think you will buy the armour. Then you're confident that you will enter tomorrow's battle with armour and survive. You should then give high credence to the assumption that the armour offered you protection. So you should rest content with your decision to buy the armour.
We don't have an unstable decision problem. If you care about injury, you should definitely buy the armour. If you don't, it looks like you could rationally decide for either option. The decision problem has at least two equilibrium solutions. In fact, it has three: there's also an equilibrium state of indecision in which you're confident, but not certain, that you won't buy the armour.
What CDT recommends in such a case isn't clear. Some would say all three states are permissible. Skyrmsian deliberation dynamics would arguably pull you into the decision to buy the armour (mainly because the information that you will survive should already strongly increase your credence that you will wear armour). As I mentioned in an earlier post, I generally favour choosing the best of the available equilibria. Here this is the equilibrium where you decide not to buy armour.
We don't need to settle the question. Whatever we say, we don't have a counterexample to General Preference Reflection. In the Armour case, you prefer buying armour to not buying armour, and you don't know what you would prefer if you knew the answer to 'will I survive?'.
(Would be good to have a proof that CDT validates General Preference Reflection. I vaguely remember having worked this out once, but I can't find it any more.)