Wolfgang Schwarz

Shangri La Variations

I

There are two paths to Shangri La. One goes by the sea, the other by the mountains. You are on the mountain path and about to enter Shangri La. You can choose how your belief state will change as you enter through the gate, in response to whatever evidence you may receive. At the moment, you are (rationally) confident that you have travelled by the mountains. You know that you will not receive any surprising new evidence as you step through the gate. You want to maximize the expected accuracy of your future belief state – at least with respect to the path you took. How should you plan to change your credence in the hypothesis that you have travelled by the mountains?

Answer: You should plan to remain confident that you have travelled by the mountains. Obviously.

II

This time you have taken the path by the sea. You know that when you step through the gate to Shangri La, you will have vivid memory experiences as of having travelled by the mountains. As before, you can choose how to update your beliefs, and you want to maximize your expected future accuracy. How should you plan to change your credence in the hypothesis that you have travelled by the sea?

Answer: You should plan to remain confident that you have travelled by the sea.

Your memory experiences would normally be evidence against this hypothesis. But if you know in advance that you will have these experiences even if you came by the sea then they do not constitute evidence against the sea hypothesis. (E is evidence for H only if E is more likely given H than given not-H.)

III

As in case I, you have travelled by the mountains. You can choose an update disposition, but not all possible dispositions are available. When you step through the gate to Shangri La, your belief state will become insensitive to your previous beliefs about which path you took. As a result, you can't choose an update disposition that would make your new beliefs about your journey depend on your current beliefs. With respect to your journey, the chosen update disposition must be a function of your new evidence alone. And you know that you would receive the same new evidence no matter which path you came from. How should you plan to change your credence in the hypothesis that you have travelled by the mountains?

Answer: You should plan to remain confident that you have travelled by the mountains.

It is easy to see that the update disposition that maps your (irrelevant) new evidence to high credence in the mountain hypothesis maximizes expected future accuracy, relative to your present credences.

This disposition would, of course, lead to highly inaccurate beliefs in a counterfactual situation in which you came by the sea. But you know that you are not in that situation. (It doesn't matter if you could easily have been in the situation – say, because you have chosen your path by the flip of a coin.)

Before we continue, let's have a look at what happens if you adopt the recommended disposition, step through the gate, and remain confident that you have travelled by the mountains. Do you still know that you have travelled by the mountains? I'd say yes. The belief is true and has been formed on impeccable grounds. Do you have any evidence that you have travelled by the mountains? You may have your belief state as evidence, but suppose the state isn't introspectible. Then, arguably, you don't have any evidence that you have travelled by the mountains. You have knowledge without evidence.

IV

You are setting out to travel to Shangri La. You are going to flip a coin to decide whether to take the path by the mountains or the path by the sea. Before you toss the coin, you can choose how your belief state will change once you enter Shangri La. As in case III, you cannot make your new belief state sensitive to the previous belief state. You know that the new evidence you will receive upon arrival will be the same no matter which path you took. How confident should you plan to become in the hypothesis that you have travelled by the mountains, if you want to maximize expected accuracy?

Answer: You should plan to give equal credence to having come by the mountains and having come by the sea.

Notice that you will regret this choice as soon as you've chosen a path, and you know that you will. At that point, we're in case III and you will favour a disposition that retains your belief about the chosen path. That's OK. Decisions that are rational in light of some information are often irrational in light of further information.

Case IV can be used to support my claim about case III, that it involves knowledge without evidence. For suppose you adopt the disposition recommended in case IV. You arrive at Shangri La and become unsure about your journey. Is your uncertainty proportioned to your evidence? Plausibly yes. It would be strange to think that after stepping through the gate you have strong evidence that you have, say, travelled by the mountains. But if you can't introspect your beliefs then your relevant evidence in this case appears to be exactly the same as it is in case III. It follows that in case III you also don't have any evidence about your journey.

V

We are building a robot that will travel to Shangri La. The robot's path will be chosen by the flip of a coin: heads means mountains, tails sea. The robot has sensors that tell it where it is. It also has a memory in which it can store previously received information. We have to decide how the memory should be updated once the robot arrives at Shangri La. The problem is that passing through the gate will wipe out any existing memory about the path the robot has taken. Given this constraint, what is the optimal update process, if we want the robot's new memory state to be as accurate as possible? How confident should the robot become in the hypothesis that it has travelled by the mountains?

Answer: The optimal update process depends on the outcome of the coin toss. If the coin lands heads then the optimal update process makes the robot certain that it has travelled by the mountains. If the coin lands tails then the optimal update process makes the robot certain that it has travelled by the sea.

To be sure, we don't know the outcome of the coin toss. We can't make our robot design depend on the outcome of the coin toss. In light of our limited information, it would be best to let the robot become unsure about which path it has taken. But why is our limited information as robot designers relevant to what is in fact the optimal design?

VI

We are still designing the robot. But let's ask a different question. In terms of accuracy, the optimal robot design would make the robot certain of all truths. This doesn't fit our intuitive concept of epistemic rationality. Intuitively, whether an agent's beliefs are rational is determined solely by how the agent has responded to their evidence, without taking into account any other contingent facts about the world. To evaluate whether a given robot design would be rational, we should therefore evaluate it from the perspective of a rational ur-prior that is unperturbed by contingent information. Here, then, is the new question: given the constraints of the scenario, how should a robot be designed so that its credence at any point maximizes ur-prior expected accuracy? In particular, what should this robot believe about its journey once it arrives at Shangri La?

Answer: The robot should become unsure about which path it has taken.

Let Cr₀ be a rational ur-prior. We assume that accuracy measures are proper. Relative to Cr₀, Cr₀ itself then maximizes expected accuracy. So our robot should start out with Cr₀. In normal cases, it should then update its beliefs by conditionalising on any new evidence it receives: under the usual idealising assumptions, successive conditionalisation from Cr₀ maximizes Cr₀-expected accuracy.

But we can't design a robot that is guaranteed to conditionalise when it steps through the gate to Shangri La. We can't make the robot's belief state sensitive to its previous belief state because the relevant parts of that state have been erased. Let's also assume that any update disposition must be "local" in the sense that it can't directly operate on earlier evidence, unmediated by belief. This means that the robot is guaranteed to arrive in the same belief state no matter which path it came from. In our state of ur-prior ignorance, both cases are equally probable. The update process that maximizes ur-prior expected accuracy therefore makes the robot unsure of which path it has taken.

We might say that an epistemically ideal agent would retain their knowledge about which path they came from, but an epistemically rational agent would lose this information.

If that's true then something interesting is happening in case III. There we saw that once you know which path you are on, you should plan to update your beliefs in a way that retains this information. You should plan to update your beliefs in an irrational manner!

The explanation of this oddity is that you have more information than the ur-prior function that gets to decide what is rational. (Compare: if you have more information than me, and you could decide how my credences should evolve, then the update that maximizes expected accuracy, relative to your credence function, is an "irrational" update that makes me confident of your information.)