4 ms·

Oh, it gets worse. One of our biases is towards conservation of mental energy. If we can, we choose the easiest path. Which means that we actively prefer exper

Oh, it gets worse. One of our biases is towards conservation of mental energy. If we can, we choose the easiest path. Which means that we actively prefer experts who are smart, well-informed, and certain of their beliefs. Because we can then be comfortable handing our thinking over to them. Why bother questioning the expert when we'd surely agree if we just knew enough? And then confirmation bias keeps us from ever questioning their results. The result, as https://www.amazon.com/Expert-Political-Judgment-Good-Know/dp/0691128715 https://www.amazon.com/Expert-Political-Judgment-Good-Know/d... documents, is that our most popular and highest paid pundits have those characteristics. But those experts are ALSO the ones who are worst at making predictions about the world. As I pointed out in https://www.lesswrong.com/posts/TMgfapzbt5qp4Hszf/doubt-certainty https://www.lesswrong.com/posts/TMgfapzbt5qp4Hszf/doubt-cert..., we should really be concluding "likely has cognitive bias" instead of "the expert says it, so case closed". The documented result is that we wind up trusting the experts that we should trust least!

jjoonathan 3y ago
"Likely cognitive bias" is not a conclusion, it's an invitation to do more research. The catch is that the amount of research you can do is extremely limited compared to what would be needed to verify the facts you hear, so you can't actually escape the need to trust and manage your trust relationships. An incredibly common mistake is to reject a decent but imperfect source in favor of a bad source, and for this reason I consider "trust less" actively harmful. Trust A over B when C (subject to real time limits on computing C) = ok Trust Less = harmful
- RationalDino 3y ago
  It is both an invitation and a conclusion. As you note, we need to decide who to trust, and we can't do sufficient research. So we need heuristics. The heuristics that I recommend are these: 1. Certainty, especially if there is debate among experts, indicates likely cognitive bias. 2. The ability to discuss how multiple perspectives would lead different people to different conclusions, suggests good reasoning. 3. A documented history of quantitative reasoning, along with data showing historical accuracy, is the best evidence of a trustworthy source. I base those heuristics on the research in https://www.amazon.com/Expert-Political-Judgment-Good-Know/dp/0691128715 https://www.amazon.com/Expert-Political-Judgment-Good-Know/d.... Applying this heuristic to "sounds certain" results in a conclusion, that is an invitation to further research that may or may not happen.
  - mistermann 3y ago
    Did you (or to what degree) take epistemic soundness and logic into consideration during your analysis, or are you only coming at it from Bayesian perspective? To what degree did you take the degree of rationality you've achieved into consideration (Rationalists aspire to rationality, though they often forget that) during your analysis?

jjoonathan 3y ago

"Likely cognitive bias" is not a conclusion, it's an invitation to do more research. The catch is that the amount of research you can do is extremely limited compared to what would be needed to verify the facts you hear, so you can't actually escape the need to trust and manage your trust relationships. An incredibly common mistake is to reject a decent but imperfect source in favor of a bad source, and for this reason I consider "trust less" actively harmful. Trust A over B when C (subject to real time limits on computing C) = ok Trust Less = harmful

RationalDino 3y ago
It is both an invitation and a conclusion. As you note, we need to decide who to trust, and we can't do sufficient research. So we need heuristics. The heuristics that I recommend are these: 1. Certainty, especially if there is debate among experts, indicates likely cognitive bias. 2. The ability to discuss how multiple perspectives would lead different people to different conclusions, suggests good reasoning. 3. A documented history of quantitative reasoning, along with data showing historical accuracy, is the best evidence of a trustworthy source. I base those heuristics on the research in https://www.amazon.com/Expert-Political-Judgment-Good-Know/dp/0691128715 https://www.amazon.com/Expert-Political-Judgment-Good-Know/d.... Applying this heuristic to "sounds certain" results in a conclusion, that is an invitation to further research that may or may not happen.
- mistermann 3y ago
  Did you (or to what degree) take epistemic soundness and logic into consideration during your analysis, or are you only coming at it from Bayesian perspective? To what degree did you take the degree of rationality you've achieved into consideration (Rationalists aspire to rationality, though they often forget that) during your analysis?