4 ms·

> Less Wrong relative to a presumed-ordered natural world The "natural world" doesn't have beliefs, the world itself cannot be right or wrong, it just is. So t

> Less Wrong relative to a presumed-ordered natural world The "natural world" doesn't have beliefs, the world itself cannot be right or wrong, it just is. So the question again is, Less Wrong relative to whom? > To the end of shaping this world to be more in line with one's preferences. And what if one is unsure of one's preferences?

FeepingCreature 5y ago
> The "natural world" doesn't have beliefs, the world itself cannot be right or wrong, it just is. So the question again is, Less Wrong relative to whom? What? Okay, let me try really small steps. We assume the natural world is ordered. We assume that the brain holds beliefs, which are compressive/predictive patterns, about the observations of the natural world. Inasmuch as these are predictive, and inasmuch as we can take multiple actions, then a belief can be qualitatively better or worse in selecting actions that make the world be more good by one's judgment. Beliefs can fail to be appropriate to select actions on (at least) two metrics: they can be "right or wrong", or they can be "more or less useful." An example of a wrong belief: if you think it is raining, and so you bring an umbrella, but it is not raining (the belief is wrong), then you have failed to improve your position: you are dry anyway, but now you are also carrying an umbrella, which is annoying. An example of a useless belief: you can believe that your shirt has more fibers than the average shirt of its kind. This may be true, but even if it is true, there is no decision that you would make where the belief is instrumental in deciding which of the possible actions leads to a world that better matches your preference. (Unless you have very specific preferences about clothing.) The goal of becoming Less Wrong is for your brain to contain fewer beliefs that are wrong, and more beliefs that are right; and also (but less importantly) fewer beliefs that are useless, and more beliefs that are useful. The term "Less Wrong" does not refer to a comparison to some other existing human, but rather to the counterfactual outcome of failing to reflect on your thoughts - Less Wrong is a collection of techniques whose goal is, when applied, to make your brain have less wrong beliefs and less useless beliefs, in order to improve your ability to shape the world to your preferences. (Whether they actually achieve this is a hot topic of debate - but that is the standard which they set.) > And what if one is unsure of one's preferences? That depends on your preferences for your preferences. :) In any case, setting your preferences is not the purpose of rationality. (Though as your preferences are part of the ordered natural world, rationality also aims to improve your ability to shape your preferences, this does not tend to be a central consideration.)
- Nav_Panel 5y ago
  So, to be perfectly clear, the main reason for studying rationality as you explain it, is the belief that many individuals are not skilled at determining whether their beliefs about the observable world are right or wrong (as in your umbrella example), and that this is something they would benefit from working on?
  - FeepingCreature 5y ago
    Well, most people can generally recognize when a belief is wrong if they are directly confronted with the fact. I think it's a bit more meta - such as creating modes of thought that systematically (again, holding in mind the assumption that nature is regular) produce right and useful thoughts, and dismantling modes of thought that systematically produce wrong or useless thoughts. But in essence, yes. (Though of course, again, the ultimate goal is to be better able to achieve preferred outcomes in the world, and if thinking only the word "blue" on repeat produced outcomes perfectly in line with your preferences, then rationality would espouse blue blue blue blue. But at that point, most of the techniques gathered under the label would be useless, because yadda yadda ordered world etc.)
    - Nav_Panel 5y ago
      Thanks for sticking with me as I clarified, I see the crux of our disagreement now. Basically, I am personally of the belief that any "wrong" or "useless" beliefs that one holds are always actually useful, as all belief is instrumental. Ignorance is maintained because it serves a purpose. I am also of the belief that the only way to determine why one holds a wrong belief is to understand the "emotional" or desire-based reason that underlies or supports it. Hence why I spent a lot of time above discussing emotions. We have seen in recent times the repeated failure of individuals to adjust their beliefs even in the face of overwhelming evidence. Why is this? My stance is that their "wrong" beliefs serve deep emotional needs for them. Most people have stronger desires than the desire to arrive at truth in their evaluation of the world. The difference in emphasis here is shifting from "the brain as a prediction-making device" to "the human body including the brain is a system that acts to relieve tensions placed upon it", and that prediction-making is one biological strategy for doing so. Of course, better predictions can and will relieve more tension than worse predictions, but my stance is that being able to conceptualize the causes and nature of tension directly will lead to better outcomes, as it permits stronger meta-cognition. I see the term "preference" doing a lot of work in rationalist thought, and I generally taboo it. The term comes from economics and fails IMO to adequately describe the nature of desire. But if I were to use the term in this instance, I would claim that disentangling one's preferences from internal conflict is the first step one should take prior to tackling the problem of rationality (because we need to know what we even want to get done before we figure out how to do it). This involves interrogating our beliefs about what our preferences are and why, which is something that the rationalist focus on the "world" seems uniquely unequipped for dealing with. I hope this all makes sense!

FeepingCreature 5y ago

> The "natural world" doesn't have beliefs, the world itself cannot be right or wrong, it just is. So the question again is, Less Wrong relative to whom? What? Okay, let me try really small steps. We assume the natural world is ordered. We assume that the brain holds beliefs, which are compressive/predictive patterns, about the observations of the natural world. Inasmuch as these are predictive, and inasmuch as we can take multiple actions, then a belief can be qualitatively better or worse in selecting actions that make the world be more good by one's judgment. Beliefs can fail to be appropriate to select actions on (at least) two metrics: they can be "right or wrong", or they can be "more or less useful." An example of a wrong belief: if you think it is raining, and so you bring an umbrella, but it is not raining (the belief is wrong), then you have failed to improve your position: you are dry anyway, but now you are also carrying an umbrella, which is annoying. An example of a useless belief: you can believe that your shirt has more fibers than the average shirt of its kind. This may be true, but even if it is true, there is no decision that you would make where the belief is instrumental in deciding which of the possible actions leads to a world that better matches your preference. (Unless you have very specific preferences about clothing.) The goal of becoming Less Wrong is for your brain to contain fewer beliefs that are wrong, and more beliefs that are right; and also (but less importantly) fewer beliefs that are useless, and more beliefs that are useful. The term "Less Wrong" does not refer to a comparison to some other existing human, but rather to the counterfactual outcome of failing to reflect on your thoughts - Less Wrong is a collection of techniques whose goal is, when applied, to make your brain have less wrong beliefs and less useless beliefs, in order to improve your ability to shape the world to your preferences. (Whether they actually achieve this is a hot topic of debate - but that is the standard which they set.) > And what if one is unsure of one's preferences? That depends on your preferences for your preferences. :) In any case, setting your preferences is not the purpose of rationality. (Though as your preferences are part of the ordered natural world, rationality also aims to improve your ability to shape your preferences, this does not tend to be a central consideration.)

Nav_Panel 5y ago
So, to be perfectly clear, the main reason for studying rationality as you explain it, is the belief that many individuals are not skilled at determining whether their beliefs about the observable world are right or wrong (as in your umbrella example), and that this is something they would benefit from working on?
- FeepingCreature 5y ago
  Well, most people can generally recognize when a belief is wrong if they are directly confronted with the fact. I think it's a bit more meta - such as creating modes of thought that systematically (again, holding in mind the assumption that nature is regular) produce right and useful thoughts, and dismantling modes of thought that systematically produce wrong or useless thoughts. But in essence, yes. (Though of course, again, the ultimate goal is to be better able to achieve preferred outcomes in the world, and if thinking only the word "blue" on repeat produced outcomes perfectly in line with your preferences, then rationality would espouse blue blue blue blue. But at that point, most of the techniques gathered under the label would be useless, because yadda yadda ordered world etc.)
  - Nav_Panel 5y ago
    Thanks for sticking with me as I clarified, I see the crux of our disagreement now. Basically, I am personally of the belief that any "wrong" or "useless" beliefs that one holds are always actually useful, as all belief is instrumental. Ignorance is maintained because it serves a purpose. I am also of the belief that the only way to determine why one holds a wrong belief is to understand the "emotional" or desire-based reason that underlies or supports it. Hence why I spent a lot of time above discussing emotions. We have seen in recent times the repeated failure of individuals to adjust their beliefs even in the face of overwhelming evidence. Why is this? My stance is that their "wrong" beliefs serve deep emotional needs for them. Most people have stronger desires than the desire to arrive at truth in their evaluation of the world. The difference in emphasis here is shifting from "the brain as a prediction-making device" to "the human body including the brain is a system that acts to relieve tensions placed upon it", and that prediction-making is one biological strategy for doing so. Of course, better predictions can and will relieve more tension than worse predictions, but my stance is that being able to conceptualize the causes and nature of tension directly will lead to better outcomes, as it permits stronger meta-cognition. I see the term "preference" doing a lot of work in rationalist thought, and I generally taboo it. The term comes from economics and fails IMO to adequately describe the nature of desire. But if I were to use the term in this instance, I would claim that disentangling one's preferences from internal conflict is the first step one should take prior to tackling the problem of rationality (because we need to know what we even want to get done before we figure out how to do it). This involves interrogating our beliefs about what our preferences are and why, which is something that the rationalist focus on the "world" seems uniquely unequipped for dealing with. I hope this all makes sense!