Validity judgments
Philosophers (and linguists) often appeal to judgments about the validity of general principles or arguments. For example, they judge that if C entails D, then 'if A then C' entails 'if A then D'; that 'it is not the case that it will be that P' is equivalent to 'it will be the case that not P'; that the principles of S5 are valid for metaphysical modality; that 'there could have been some person x such that actually x sits and actually x doesn't sit' is an unsatisfiable contradiction; and so on. In my view, such judgments are almost worthless: they carry very little evidential weight.
The problem is that judgments about validity are universal. If P is a valid principle, there must be no coherently conceivable scenario in which any instance of P is false. But have you surveyed all instances of P and all conceivable scenarios? If not, how do you know the principle is valid?
According to Frege's Axiom V, the set of Fs equals the set of Gs iff all Fs are Gs and all Gs are Fs. Offhand, that sounds plausible — because we tend to think only of harmless instances. As soon as we plug in Russell's "is a set that does not contain itself" for both F and G (and pause to think for a moment), the principle no longer sounds plausible at all. But one counterexample is enough to invalidate the principle.
Suppose Alice the time traveler is visiting her younger self back in 2010. She finds her younger self asleep. In this scenario, it is plausible that there is a person x and a time t such that at t, x is asleep and at t, x is not asleep. So the negation of that statement is not generally valid, even though it may easily have seemed so if you had simply stared at it and asked yourself whether it is valid.
One could give many more examples. If you think excluded middle and bivalence are generally valid, make sure to consider instances of ambiguity and vagueness. If you think the T-schema is valid, don't forget the Liar. When contemplating principles about time, make sure to consider the possibility of branching, multi-dimensional, and circular time. (And remember that there is no objective simultaneity even in the actual world.)
Philosophers who rely on judgments about validity often come up with elaborate theories to validate the relevant principles even in cases where they seems to fail. People have even come up with paraconsistent set theories that validate Axiom V. Others have developed theories of vagueness that respect classical logic, and theories of modality that respect S5. Almost all recent discussions of counterpart theory are concerned with trying to preserve the standard logic of 'actually'. Nobody has ever explained why those principles should be valid. In fact, if we consider modal analogues to time-travel cases, I think the "standard logic of 'actually'" is clearly invalid. So that whole literature is wasted effort.
In general, we should recognize validity judgments for what they are: risky conjectures about an open-ended domain. We have no direct access to validity facts.
There may be exceptions. For example: 'if set A and set B have the same members, then A and B are identical.' Or: 'If x is divisible by 4, then x is not prime'. Here it seems OK to rely on our validity judgment. Why is that? Loosely speaking, I guess the reason is that the relevant principles partly define what we mean by 'set' and 'prime'. If someone claims that there might be prime numbers divisible by 4, she is either confused or misunderstands what it means to be prime.
Similarly, I think it is defensible to stipulatively use 'knowledge' in such a way that knowledge entails truth, or 'possible' in such a way that 'possibly p' is equivalent to 'not necessarily not p'. But even here one must be careful, for there are often surprising constraints on what one can stipulate. Again, Frege's Axiom V is a case in point. Similarly, if you think that knowledge entails truth, and that the objects of knowledge are more fine-grained than sets of possible worlds, you're dangerously close to clashing with elementary arithmetic and classical logic.