It's possibly that Chapman is arguing against a straw-man of Yudkowskian rationality, but it's also the case that many people really do seem to believe in somet

It's possibly that Chapman is arguing against a straw-man of Yudkowskian rationality, but it's also the case that many people really do seem to believe in something like that straw-man, and so it's necessary to argue against it anyway. It's true that in any attempt to orient oneself to the world, probabilities will be involved. Some of these are modelled implicitly and autonomously by your perceptual system, some are experienced as intuitions, and some are modelled explicitly as part of a problem-solving task, and some are considered reflectively. In the problem-solving scenario, we are taking a certain model of the situation as a given, and assigning probabilities based on that. Sometimes that's good: our model is accurate and gives us things that are relatively easy to assign probabilities to. In some situations, like casino gambling, the map/territory distinction is almost nil. We still have some problems, though: not all maps are accurate, and some accurate maps don't make it easy to assign probabilities. We can still do it - we can always assign some probability - but we might not be getting very good results any more. Changing our map requires, in Chapman's terms, changing our "stance": our idea about what the hell is going on in the situation, including our ideas about who we are in relation to it. Roughly speaking, stances are cached sets of techniques, assumptions and perspectives which make certain actions and perceptions available to us. They are associated with schools of practice, and have evolved for suitability for certain situations. We may have reason to believe that a given stance is correct for a situation, but our reason for thinking so is not that the stance gives us access to an explicit model of probabilities. A lot of this is actually pretty obvious, and we do it all the time. The feeling of "oh, this is one of those situations, I know what to do now" is familiar to most of us. To the extent that a stance gives us a probabilistic model to work with, it is by updating our perceptual and intuitive senses of what to do and what kind of result to expect. "Meta-rationality", in Chapman's terms, is the skill of maintaining a constant awareness of one's stance, the possibility of changing it, and some well-trained intuition about when to do so. Introspection on these stance-associated intuitions can be very valuable, but this is a reflective practice, requiring a much greater intensity of attention than merely acting on them (and this is generally not compatible with acting on them in real time, c.f "choking" in sports). It is more "computationally" (and hence metabolically) expensive, and is generally something one would do only after gaining familiarity with the practice. One cannot use a priori probabilistic reasoning to imagine what one would do in a situation if one were, say, skilled in the art of kung fu - one can only understand the implications of being good at kung fu by actually becoming good at it (see Agnes Callard's book Aspiration[1] on this point). After which point, it may or may not be good for your kung fu practice to examine it with the tools of probabilistic rationality. I imagine Chapman would argue that, for many stances and their associated practices, it isn't really worth it, but you'd have to ask him. Of course, we can certainly describe the entire process using probabilities: my decision to treat this as a kung fu problem rather than a diplomatic problem ultimately rests on some model I have of the likely outcomes. But this is merely descriptive: I am not "shutting up and calculating" in order to make the decision, merely rationalising it after the fact. Perhaps this does not seem particularly insightful to you, and I would agree that perhaps it should not - you may well have figured this out for yourself, and experienced rationalists may be familiar with all of it. Most of these ideas are within the grasp of thoughtful teenagers. And yet, at the same time, there really are people who act as if they believe that knowledge of probability theory will enable them to make, in near-real-time, optimal decisions even in unfamiliar domains. And most of us, if we're honest, forget just how much of a difference a change in stance can make to our perception of a situation. Many of us are naive realists who have trouble accepting that our very ideas of what is "real" in a situation is mentally constructed and dependent on our sense of what is relevant. It's not coincidental that Chapman's writings are often a re-working of certain Buddhist treatments of how our view constructs our reality. [1]: https://academic.oup.com/book/4936 https://academic.oup.com/book/4936