Self-locating priors, primary intensions, and cosmological measures
If a certain hypothesis entails that N percent of all observers in the universe have a certain property, how likely is it that we have that property – conditional on the hypothesis, and assuming we have no other relevant information?
Answer: It depends on what else the hypothesis says. If, for example, the hypothesis says that 90 percent of all observers have three eyes, and also that we ourselves have two eyes, then the probability that we have three eyes conditional on the hypothesis is zero.
This effect is easy to miss because many hypotheses that appear to be just about the universe as a whole secretly contain special information about us. Consider the following passage from Carroll (2010), cited in Arntzenius and Dorr (2017):
Imagine we have two theories of the universe that are identical in every way, except that one predicts that an Earth-like planet orbiting the star Tau Ceti is home to a race of 10 trillion intelligent lizard beings, while the other theory predicts there are no intelligent beings of any kind in the Tau Ceti system. Most of us would say that we don’t currently have enough information to decide between these two theories. But if we are truly typical observers in the universe, the first theory strongly predicts that we are more likely to be lizards on the planet orbiting Tau Ceti, not humans here on Earth, just because there are so many more lizards than humans. But that prediction is not right, so we have apparently ruled out the existence of that many observers without collecting any data at all about what's actually going on in the Tau Ceti system.
I share Carroll's intuition. The fact that we aren't lizards is no reason to reject a theory according to which there are many lizards at Tau Ceti.
Carroll, Arntzenius, and Dorr assume that this intuition clashes with a principle of "anthropic reasoning" according to which we should regard ourselves as randomly sampled from the observers in the world. Less metaphorically, the principle says that conditional on the hypothesis that N percent of all observers in the world have property P, the prior probability that we have P is N percent.
In Carroll's example, P is the property of being a lizard. The hypothesis that there are 10 trillion intelligent lizards at Tau Ceti entails (let's assume) that a large proportion of observers in the world are lizards. The anthropic principle therefore seems to suggest that conditional on this hypothesis there is a high prior probability that we are lizards. Conditional on the alternative hypothesis (that there aren't any lizards at Tau Ceti) the probability is much lower. By Bayes' Theorem, it would follow that the fact that we are not lizards strongly supports the hypothesis that there aren't any lizards at Tau Ceti.
But the hypothesis that there are 10 trillion intelligent lizards at Tau Ceti doesn't just say that the proportion of lizards in the universe is high. It says more. In particular, it says where most of the lizards can be found – at Tau Ceti. And 'Tau Ceti', like all names, isn't a bare tag that simply picks out a certain star. It picks out the star under a certain mode of presentation, relating it to our own position in the universe.
Imagine we are looking into the night sky. I point at a distant star and say, "let's call this star 'Psi'". While I'm still pointing (awkwardly), we wonder whether there are trillions of intelligent lizards in the Psi system. This hypothesis is epistemically equivalent to the hypothesis that there are trillions of intelligent lizards on a planet orbiting the star that I'm pointing at. Ignoring the far-fetched possibility that we ourselves are in the system of the star at which I'm pointing, the hypothesis essentially says that there are lots of lizards somewhere else. And so the anthropic principle doesn't apply. Conditional on the assumption that most observers are lizards and that most of these lizards inhabit a planet far away from ours, it is not especially likely that we are lizards.