3 ms·
I find the article frustrating because of the tone. It's also wrong. They misunderstood what lines mean. This article is absolutely dripping with condescension
by titzer 5y ago
I find the article frustrating because of the tone. It's also wrong. They misunderstood what lines mean.
This article is absolutely dripping with condescension throughout and is really pushing a "gotcha" that doesn't exist. It then argues basic statistics, generates a DK-looking graph from random data, and then claims the phenomena doesn't exist. When in fact, as other people have commented, when people are bad at estimating their own ability (i.e. random), the DK effect still exists; it falls out of statistics.
Sigh, the author misunderstood the very definition of the DK effect:
> "The Dunning–Kruger effect is the cognitive bias whereby people with low ability at a task overestimate their ability. Some researchers also include in their definition the opposite effect for high performers: their tendency to underestimate their skills."
In all the examples, this holds, even if the assessment ability is totally random. Even if every quartile gives themself an average score, like the random data generated here. The author seems to think that it should be even more lopsided or something to demonstrate the effect. (I mean, honestly, what are they expecting, a line above 50th percentile? A line with negative slope? What?)
If there were no DK effect, the two lines would be the same.
Instead, if we go back and look at the original data, we see indeed, the two lines are not the same, the average for the bottom quantile is over 50%, there is some small increase in perceived ability associated with actual ability (and not the opposite).
The sin here isn't some autocorrelation gotcha, but rather, DK should have put error bars on the graph. If it was totally random, the error bars would be all over the place.
- bena 5y agoThe fact that you can generate a Dunning-Kruger looking graph using nothing but noise does indicate that the graph isn't proof of anything. He also points out that the problem is that there's nothing below zero and nothing above 100. You can't have people who estimate beyond that. He uses another study and it turns out, the less knowledgeable you are about a skill, the worse you are at estimating your ability at all. In both directions. If the lines were the absolute difference between perceived ability and actual ability, for no effect, the lines still shouldn't be the same. They should converge towards those who are knowledgeable. If anything, the difference line should be nearly a horizontal line. Because there should be greater variance in estimations at the lower end.
- titzer 5y agoIt would seem like using violin plots in these type of graphs would help a lot. If low-skilled people are bad at estimating, their variance (and distribution) will be a lot wider.