3 ms·

I'm not sure what you mean with "the wider the line will get". But here is the issue: The least competent person cannot underestimate their relative competency

I'm not sure what you mean with "the wider the line will get". But here is the issue: The least competent person cannot underestimate their relative competency. Any not exactly accurate estimate they do is an overestimate. Correspondingly, the most competent person cannot overestimate their relative competency. This leads to the perception of bias where there is none, except a trivial tautological one.

danbruc 5y ago
I made you a picture [1]. I randomly generated 100 test scores between 0 and 1, then different self assessments. Top left, self assessment matches actual score, top middle, self assessment varies uniformly by ±0.1 around the test score, top right, self assessment varies uniformly by ±0.2 around the test score. None of those have a Dunning-Kruger effect. If you aggregate data points, there will be - as you mentioned - an edge effect because the self assessment will get clipped. In the bottom row I added a Dunning-Kruger effect, at a test score of 0.7 the self assessment is perfect, below and above that the self assessment is off by 0.5 times the distance of the test score from 0.7. Otherwise the bottom charts are the same, no random variation on the left, ±0.1 in the middle and ±0.2 on the right. You can see that the edge effect is less important as the data points are steered away from the corners. I will admit that the original Dunning-Kruger chart could or could not show a real effect, really depends on how they aggregated the data and how noisy self assessments are. But if you have a raw data set like the one I generated, you could easily determine if there is an effect. If one could find such a data set, I would like to have a look. [1] https://imgur.com/g4frW6p https://imgur.com/g4frW6p
- geysersam 5y ago
  I see what you mean. It should be possible to determine if there is an effect using the raw data from the experiment.

danbruc 5y ago

I made you a picture [1]. I randomly generated 100 test scores between 0 and 1, then different self assessments. Top left, self assessment matches actual score, top middle, self assessment varies uniformly by ±0.1 around the test score, top right, self assessment varies uniformly by ±0.2 around the test score. None of those have a Dunning-Kruger effect. If you aggregate data points, there will be - as you mentioned - an edge effect because the self assessment will get clipped. In the bottom row I added a Dunning-Kruger effect, at a test score of 0.7 the self assessment is perfect, below and above that the self assessment is off by 0.5 times the distance of the test score from 0.7. Otherwise the bottom charts are the same, no random variation on the left, ±0.1 in the middle and ±0.2 on the right. You can see that the edge effect is less important as the data points are steered away from the corners. I will admit that the original Dunning-Kruger chart could or could not show a real effect, really depends on how they aggregated the data and how noisy self assessments are. But if you have a raw data set like the one I generated, you could easily determine if there is an effect. If one could find such a data set, I would like to have a look. [1] https://imgur.com/g4frW6p https://imgur.com/g4frW6p

geysersam 5y ago
I see what you mean. It should be possible to determine if there is an effect using the raw data from the experiment.