3 ms·

You're alluding to, without naming it, the "Dunning-Kruger" effect: you need to be knowledgeable enough to determine the complexity of a field. Beginners will o

You're alluding to, without naming it, the "Dunning-Kruger" effect: you need to be knowledgeable enough to determine the complexity of a field. Beginners will often overestimate their ability. https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect

kkylin 5y ago
Reminds me of two things that readers of this post may also find interesting: https://terrytao.wordpress.com/career-advice/dont-prematurely-obsess-on-a-single-big-problem-or-big-theory/ https://terrytao.wordpress.com/career-advice/dont-prematurel... https://terrytao.wordpress.com/career-advice/be-sceptical-of-your-own-work/ https://terrytao.wordpress.com/career-advice/be-sceptical-of... Tao is specifically speaking to mathematicians, but some of the lessons carry to other fields (and perhaps other areas of life).
dahart 5y ago
No no no!! Oh man, I’m really sad my comment invoked DK, I don’t think this is the same thing at all, and I have a strong dislike of this paper because it’s both misrepresented and almost universally misunderstood. (And the authors are complicit in encouraging this misunderstanding, because the paper made them famous.) The paper did not show confidence being inversely proportional to knowledge or skill as many people believe. It shows a positive correlation between confidence and skill. But the paper itself is very hyperbolic and draws invalid conclusions that are unsupported by their own data IMO. You should read the actual paper, it’s enlightening to look at what they actually tested, and compare that to how they wrote about it. They didn’t test high skill tasks. And they didn’t test incompetent people. They tested only Cornell undergrads who were volunteering for extra credit. They didn’t control for this completely skewed population sample in any way. They also didn’t test people’s estimation of themselves either. Not at all. They asked people to rank themselves in a group of others, without knowledge of the others’ skills. This simply means people were guessing, not that they were overestimating themselves!! This is a great post explaining what’s most likely happening with DK: regression to the mean. https://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-kruger-effect-is-and-isnt/ https://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-...
- mrjangles 5y ago
  Ironically, the best evidence for the DK effect comes from people confidently referring to it as though it was a real thing.
  - dahart 5y ago
    If it’s not a real effect, then it can’t be used that way because it’s not true. And if it is a real effect it also can’t be used that way -- again, the data in the DK paper shows a positive correlation between confidence and skill, which means if people seem sure about it then chances are higher that they’re right. People can’t be blamed for getting the wrong idea from the DK paper. The authors spun the narrative very carefully to imply or seem like high confidence is suspect, while not actually claiming it in the paper, and their data shows that shows people who are more confident are also more skilled on average. The idea that certainty and confidence is some kind of tell that reveals incompetence is just something people really want to believe. It feels good, but it’s specious and not actually true. Everyone knows someone who’s been arrogant and wrong, and it feels so good to think that science somehow backs the idea that arrogance shows someone is faking their expertise. Unfortunately, the only science that Dunning & Kruger did shows the opposite, that higher confidence is associated with higher skill, regardless of the fact that “skill” and “confidence” weren’t measured in any of the ways people would naturally assume when they read those words. I don’t blame @MayeulC for believing what Dunning & Kruger claimed, especially when there’s a Wikipedia article backing it up and making it seem like an accepted and solid idea. I feel bad they got downvoted because of my comment, maybe throw an upvote their way if you feel like it.
    - MayeulC 5y ago
      Oh, I don't care much for Internet points, but I like being pointed out wrong. I might have been a bit too direct referencing DK, and I think I understand your concerns. I haven't read the original paper, and I'm willing to accept your criticism of it at face value for now. However, what I named (and what I believe is widely understood as) the DK effect is quite similar to what you pointed out in your original post. Even if that's not what the original authors intended, I believe that giving names to such psychological phenomena/cognitive biases is valuable to contextualize a discussion: "we're going to discuss the Dunning-Kruger effect" is shorter than a full introduction. Those who don't know what DK is can look it up (or read the full introduction). Moreover, I have myself experienced what you describe multiple times, so I am willing to give additional credit to the idea (that I was describing as DK), even if quantifying cognitive biases is probably a lost cause. > certainty and confidence [...] reveals incompetence That is certainly not a conclusion that I am willing to make, nor something that I have ever seen expressed anywhere. The only thing I associate DK with is underestimating the deepness and breadth of a topic when you know it only superficially. And finding examples of this is quite easy. > The authors spun the narrative very carefully to imply or seem like high confidence is suspect Again, I was not aware of this, nor I am under the impression that this is a conclusion that people jump to (it really seems like a stretch). At least in my circles. I will document myself a bit more, and decide if I should keep calling this DK or not, thanks for bringing this to my attention.

kkylin 5y ago

Reminds me of two things that readers of this post may also find interesting: https://terrytao.wordpress.com/career-advice/dont-prematurely-obsess-on-a-single-big-problem-or-big-theory/ https://terrytao.wordpress.com/career-advice/dont-prematurel... https://terrytao.wordpress.com/career-advice/be-sceptical-of-your-own-work/ https://terrytao.wordpress.com/career-advice/be-sceptical-of... Tao is specifically speaking to mathematicians, but some of the lessons carry to other fields (and perhaps other areas of life).

dahart 5y ago

No no no!! Oh man, I’m really sad my comment invoked DK, I don’t think this is the same thing at all, and I have a strong dislike of this paper because it’s both misrepresented and almost universally misunderstood. (And the authors are complicit in encouraging this misunderstanding, because the paper made them famous.) The paper did not show confidence being inversely proportional to knowledge or skill as many people believe. It shows a positive correlation between confidence and skill. But the paper itself is very hyperbolic and draws invalid conclusions that are unsupported by their own data IMO. You should read the actual paper, it’s enlightening to look at what they actually tested, and compare that to how they wrote about it. They didn’t test high skill tasks. And they didn’t test incompetent people. They tested only Cornell undergrads who were volunteering for extra credit. They didn’t control for this completely skewed population sample in any way. They also didn’t test people’s estimation of themselves either. Not at all. They asked people to rank themselves in a group of others, without knowledge of the others’ skills. This simply means people were guessing, not that they were overestimating themselves!! This is a great post explaining what’s most likely happening with DK: regression to the mean. https://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-kruger-effect-is-and-isnt/ https://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-...

mrjangles 5y ago
Ironically, the best evidence for the DK effect comes from people confidently referring to it as though it was a real thing.
- dahart 5y ago
  If it’s not a real effect, then it can’t be used that way because it’s not true. And if it is a real effect it also can’t be used that way -- again, the data in the DK paper shows a positive correlation between confidence and skill, which means if people seem sure about it then chances are higher that they’re right. People can’t be blamed for getting the wrong idea from the DK paper. The authors spun the narrative very carefully to imply or seem like high confidence is suspect, while not actually claiming it in the paper, and their data shows that shows people who are more confident are also more skilled on average. The idea that certainty and confidence is some kind of tell that reveals incompetence is just something people really want to believe. It feels good, but it’s specious and not actually true. Everyone knows someone who’s been arrogant and wrong, and it feels so good to think that science somehow backs the idea that arrogance shows someone is faking their expertise. Unfortunately, the only science that Dunning & Kruger did shows the opposite, that higher confidence is associated with higher skill, regardless of the fact that “skill” and “confidence” weren’t measured in any of the ways people would naturally assume when they read those words. I don’t blame @MayeulC for believing what Dunning & Kruger claimed, especially when there’s a Wikipedia article backing it up and making it seem like an accepted and solid idea. I feel bad they got downvoted because of my comment, maybe throw an upvote their way if you feel like it.
  - MayeulC 5y ago
    Oh, I don't care much for Internet points, but I like being pointed out wrong. I might have been a bit too direct referencing DK, and I think I understand your concerns. I haven't read the original paper, and I'm willing to accept your criticism of it at face value for now. However, what I named (and what I believe is widely understood as) the DK effect is quite similar to what you pointed out in your original post. Even if that's not what the original authors intended, I believe that giving names to such psychological phenomena/cognitive biases is valuable to contextualize a discussion: "we're going to discuss the Dunning-Kruger effect" is shorter than a full introduction. Those who don't know what DK is can look it up (or read the full introduction). Moreover, I have myself experienced what you describe multiple times, so I am willing to give additional credit to the idea (that I was describing as DK), even if quantifying cognitive biases is probably a lost cause. > certainty and confidence [...] reveals incompetence That is certainly not a conclusion that I am willing to make, nor something that I have ever seen expressed anywhere. The only thing I associate DK with is underestimating the deepness and breadth of a topic when you know it only superficially. And finding examples of this is quite easy. > The authors spun the narrative very carefully to imply or seem like high confidence is suspect Again, I was not aware of this, nor I am under the impression that this is a conclusion that people jump to (it really seems like a stretch). At least in my circles. I will document myself a bit more, and decide if I should keep calling this DK or not, thanks for bringing this to my attention.