4 ms·

It’s nice to think that everyone is on equal footing, but even ignoring differences in education and genetics, the universal existence of motivated reasoning, e

It’s nice to think that everyone is on equal footing, but even ignoring differences in education and genetics, the universal existence of motivated reasoning, emotional reasoning, confirmation bias, and other cognitive distortions tends to refute your assertion that everyone is equally good at determining whether or not any given idea is bullshit. All humans—myself included—will go to great lengths to reject reality when it feels like it is a threat to a core value. This is especially the case when people have tied their identity too tightly to a given subject (i.e. hyper-partisans.) The average person, sufficiently prejudiced toward believing a given falsehood, is not going to be able to determine that it is false because their brain will start to play tricks on them. I have written about this elsewhere[0], and The Story of Us[1][2] goes into this process in much more detail. [0] https://news.ycombinator.com/item?id=25359003 https://news.ycombinator.com/item?id=25359003 [1] https://waitbutwhy.com/2019/12/political-disney-world.html https://waitbutwhy.com/2019/12/political-disney-world.html [2] https://waitbutwhy.com/2019/08/story-of-us.html https://waitbutwhy.com/2019/08/story-of-us.html (table of contents)

chordalkeyboard 6y ago
> It’s nice to think that everyone is on equal footing, but even ignoring differences in education and genetics, the universal existence of motivated reasoning, emotional reasoning, confirmation bias, and other cognitive distortions tends to refute your assertion that everyone is equally good at determining whether or not any given idea is bullshit. The universal existence of motivated reasoning, emotional reasoning, confirmation bias, and other cognitive distortions is precisely why no person or group can be expected to perform better at evaluating evidence and determining which ideas are too dangerous to allow other people to be exposed to. > All humans—myself included—will go to great lengths to reject reality when it feels like it is a threat to a core value. This is especially the case when people have tied their identity too tightly to a given subject (i.e. hyper-partisans.) The average person, sufficiently prejudiced toward believing a given falsehood, is not going to be able to determine that it is false because their brain will start to play tricks on them. This is precisely why no person can be expected to perform this role as a gatekeeper of truth. Thanks for your links, I'm a big fan of waitbutwhy.com. With regard to [1], there's no reliable test to see where a person is on the psych spectrum, even heavy doses of introspection can lead to limited and imperfect insight, and its likely the case that the same person will move up and down on the psych spectrum depending on a variety of factors. Because cognition is costly, if someone reaches a conclusion while they are in tribal mode, its going to be difficult to reevaluate their position later when they are in scientist mode. This is why its so important to have access to a variety of opinions, thinkers, and perspectives. Even the best of us are vulnerable to motivated reasoning and other cognitive biases.
- csnover 6y ago
  > The universal existence of motivated reasoning, emotional reasoning, confirmation bias, and other cognitive distortions is precisely why no person or group can be expected to perform better at evaluating evidence and determining which ideas are too dangerous to allow other people to be exposed to. This isn’t what I was hoping folks would take away from my comment. Have you ever been too invested in some problem, or too upset by something, to see the truth of the situation? And when you ask someone else, who is not emotionally invested, they easily point out the way forward? That is the kind of phenomenon I am talking about. Most humans have the capability to objectively assess ideas in general, but when someone has a strong emotional attachment to a specific idea—such as partisans who are predisposed to believe voter fraud misinformation—they are not going to be capable of evaluating that specific idea as well as someone who isn’t predisposed to believe it. I am not claiming that there is any single entity that is capable of being a gatekeeper of all truths. I am saying that there are people who are going to be less capable than others to evaluate the truthfulness of a specific idea. In this case, YouTube moderators are almost certainly going to be more capable of objectively evaluating whether or not a video is proof of widescale voter fraud than a poster who is strongly motivated to lie (intentionally or not), or an audience member who is strongly motivated to agree with that lie. And in that way, there are some people who are more capable of evaluating the objective truth than others. > Because cognition is costly, if someone reaches a conclusion while they are in tribal mode, its going to be difficult to reevaluate their position later when they are in scientist mode. This is why its so important to have access to a variety of opinions, thinkers, and perspectives. Even the best of us are vulnerable to motivated reasoning and other cognitive biases. Yes. Exactly. I’m confused how we are arriving at such different conclusions from the same basic understanding. I hope that this added explanation helps.
  - chordalkeyboard 6y ago
    > Have you ever been too invested in some problem, or too upset by something, to see the truth of the situation? And when you ask someone else, who is not emotionally invested, they easily point out the way forward? That is the kind of phenomenon I am talking about. Yes I have. Unfortunately, I am not aware of a reliable test to identify whether a person is capable of rational thought on a given subject. This means that when we witness a dispute between people about an issue, we are unable to reliably and provably evaluate the disputants meta-level reasoning. This is complicated by the fact that evaluating someone's reasoning on the meta level is nearly always complicated by object-level concerns, including our own cognitive biases. > Most humans have the capability to objectively assess ideas in general, but when someone has a strong emotional attachment to a specific idea—such as partisans who are predisposed to believe voter fraud misinformation—they are not going to be capable of evaluating that specific idea as well as someone who isn’t predisposed to believe it. Thats why its so harmful to put people in a position of being an information gatekeeper. You have no way of ensuring that your information is being filtered by a person who is emotionally detached. In fact, due to the potential for influencing other people, those positions are much more likely to be occupied by partisans who will then use it to advance their own perspective, often without even realizing it. > In this case, YouTube moderators are almost certainly going to be more capable of objectively evaluating whether or not a video is proof of widescale voter fraud than a poster who is strongly motivated to lie (intentionally or not) This is exactly the problem. There is no reason to suppose that a youtube moderator is non-partisan, a high rung thinker, or emotionally detached from the subject they are evaluating. > And in that way, there are some people who are more capable of evaluating the objective truth than others. Yes but there is not a reliable, objective way to identify them or to check their reasoning process for errors. > Yes. Exactly. I’m confused how we are arriving at such different conclusions from the same basic understanding. I hope that this added explanation helps. I feel exactly the same way. I'm at a loss to explain this except that either we have some different premises that have not been revealed in this discussion so far, or perhaps Aumann's agreement theorem [0] is not applicable to this issue for some reason. [0] https://en.wikipedia.org/wiki/Aumann%27s_agreement_theorem https://en.wikipedia.org/wiki/Aumann%27s_agreement_theorem

chordalkeyboard 6y ago

> It’s nice to think that everyone is on equal footing, but even ignoring differences in education and genetics, the universal existence of motivated reasoning, emotional reasoning, confirmation bias, and other cognitive distortions tends to refute your assertion that everyone is equally good at determining whether or not any given idea is bullshit. The universal existence of motivated reasoning, emotional reasoning, confirmation bias, and other cognitive distortions is precisely why no person or group can be expected to perform better at evaluating evidence and determining which ideas are too dangerous to allow other people to be exposed to. > All humans—myself included—will go to great lengths to reject reality when it feels like it is a threat to a core value. This is especially the case when people have tied their identity too tightly to a given subject (i.e. hyper-partisans.) The average person, sufficiently prejudiced toward believing a given falsehood, is not going to be able to determine that it is false because their brain will start to play tricks on them. This is precisely why no person can be expected to perform this role as a gatekeeper of truth. Thanks for your links, I'm a big fan of waitbutwhy.com. With regard to [1], there's no reliable test to see where a person is on the psych spectrum, even heavy doses of introspection can lead to limited and imperfect insight, and its likely the case that the same person will move up and down on the psych spectrum depending on a variety of factors. Because cognition is costly, if someone reaches a conclusion while they are in tribal mode, its going to be difficult to reevaluate their position later when they are in scientist mode. This is why its so important to have access to a variety of opinions, thinkers, and perspectives. Even the best of us are vulnerable to motivated reasoning and other cognitive biases.

csnover 6y ago
> The universal existence of motivated reasoning, emotional reasoning, confirmation bias, and other cognitive distortions is precisely why no person or group can be expected to perform better at evaluating evidence and determining which ideas are too dangerous to allow other people to be exposed to. This isn’t what I was hoping folks would take away from my comment. Have you ever been too invested in some problem, or too upset by something, to see the truth of the situation? And when you ask someone else, who is not emotionally invested, they easily point out the way forward? That is the kind of phenomenon I am talking about. Most humans have the capability to objectively assess ideas in general, but when someone has a strong emotional attachment to a specific idea—such as partisans who are predisposed to believe voter fraud misinformation—they are not going to be capable of evaluating that specific idea as well as someone who isn’t predisposed to believe it. I am not claiming that there is any single entity that is capable of being a gatekeeper of all truths. I am saying that there are people who are going to be less capable than others to evaluate the truthfulness of a specific idea. In this case, YouTube moderators are almost certainly going to be more capable of objectively evaluating whether or not a video is proof of widescale voter fraud than a poster who is strongly motivated to lie (intentionally or not), or an audience member who is strongly motivated to agree with that lie. And in that way, there are some people who are more capable of evaluating the objective truth than others. > Because cognition is costly, if someone reaches a conclusion while they are in tribal mode, its going to be difficult to reevaluate their position later when they are in scientist mode. This is why its so important to have access to a variety of opinions, thinkers, and perspectives. Even the best of us are vulnerable to motivated reasoning and other cognitive biases. Yes. Exactly. I’m confused how we are arriving at such different conclusions from the same basic understanding. I hope that this added explanation helps.
- chordalkeyboard 6y ago
  > Have you ever been too invested in some problem, or too upset by something, to see the truth of the situation? And when you ask someone else, who is not emotionally invested, they easily point out the way forward? That is the kind of phenomenon I am talking about. Yes I have. Unfortunately, I am not aware of a reliable test to identify whether a person is capable of rational thought on a given subject. This means that when we witness a dispute between people about an issue, we are unable to reliably and provably evaluate the disputants meta-level reasoning. This is complicated by the fact that evaluating someone's reasoning on the meta level is nearly always complicated by object-level concerns, including our own cognitive biases. > Most humans have the capability to objectively assess ideas in general, but when someone has a strong emotional attachment to a specific idea—such as partisans who are predisposed to believe voter fraud misinformation—they are not going to be capable of evaluating that specific idea as well as someone who isn’t predisposed to believe it. Thats why its so harmful to put people in a position of being an information gatekeeper. You have no way of ensuring that your information is being filtered by a person who is emotionally detached. In fact, due to the potential for influencing other people, those positions are much more likely to be occupied by partisans who will then use it to advance their own perspective, often without even realizing it. > In this case, YouTube moderators are almost certainly going to be more capable of objectively evaluating whether or not a video is proof of widescale voter fraud than a poster who is strongly motivated to lie (intentionally or not) This is exactly the problem. There is no reason to suppose that a youtube moderator is non-partisan, a high rung thinker, or emotionally detached from the subject they are evaluating. > And in that way, there are some people who are more capable of evaluating the objective truth than others. Yes but there is not a reliable, objective way to identify them or to check their reasoning process for errors. > Yes. Exactly. I’m confused how we are arriving at such different conclusions from the same basic understanding. I hope that this added explanation helps. I feel exactly the same way. I'm at a loss to explain this except that either we have some different premises that have not been revealed in this discussion so far, or perhaps Aumann's agreement theorem [0] is not applicable to this issue for some reason. [0] https://en.wikipedia.org/wiki/Aumann%27s_agreement_theorem https://en.wikipedia.org/wiki/Aumann%27s_agreement_theorem