3 ms·

> The D-K hypothesis is that self assessment and actual performance are less correlated for weaker than higher performing individuals. I’m not sure that’s an a

by dahart 3y ago

> The D-K hypothesis is that self assessment and actual performance are less correlated for weaker than higher performing individuals. I’m not sure that’s an accurate summary. The correlation of the perceived ability is effectively the slope of the line, and the slope is more or less constant. The paper suggests that the bias of the bottom quartile is higher than the bias of the upper quartile, not that the correlation is any different. But it’s strange that the DK paper makes an example of the lower performers, since the bias of the scores appears to be constant; it appears the high performers have pretty much the same bias as the low performers — it’s a straightish line that goes through 65% in the middle rather than the expected straight line that goes through 50% in the middle. If the ‘high performers’ had a different bias, then the line wouldn’t be so straight.

JamesBarney 3y ago
Yeah my understanding is 1. the slope of self-perceived ability is lower than actual ability 2. The y intercept is dependent on difficulty of test Therefore with an easier test the better testies are more accurate, and with a very difficult test the worse testies are more accurate because of where the lines intersect. Meaning DK is artifact of test difficulty. This also means if the test was difficult enough you could create a bizarro-DK effect where the better testies were less accurate.
- dahart 3y ago
  For 1, the data is based on guessing, so it’s zero surprise that self-perceived ability doesn’t correlate perfectly with actual ability. It would be extremely surprising and unbelievable if the slopes were the same, right? For 2, the DK paper shows one thing, but the replication attempts have show this effect doesn’t even exist for very complex tasks, like being an engineer or lawyer. The DK effect doesn’t generalize, and doesn’t even measure exactly what it claims to measure, which is why we don’t need to speculate about the bizarro-DK reversal effect - we already have evidence that it doesn’t happen, and we already have a big enough problem with people mistakenly believing that DK showed an inverse correlation between confidence and competence, when they did no such thing.

by dahart 3y ago

JamesBarney 3y ago

Yeah my understanding is 1. the slope of self-perceived ability is lower than actual ability 2. The y intercept is dependent on difficulty of test Therefore with an easier test the better testies are more accurate, and with a very difficult test the worse testies are more accurate because of where the lines intersect. Meaning DK is artifact of test difficulty. This also means if the test was difficult enough you could create a bizarro-DK effect where the better testies were less accurate.

dahart 3y ago
For 1, the data is based on guessing, so it’s zero surprise that self-perceived ability doesn’t correlate perfectly with actual ability. It would be extremely surprising and unbelievable if the slopes were the same, right? For 2, the DK paper shows one thing, but the replication attempts have show this effect doesn’t even exist for very complex tasks, like being an engineer or lawyer. The DK effect doesn’t generalize, and doesn’t even measure exactly what it claims to measure, which is why we don’t need to speculate about the bizarro-DK reversal effect - we already have evidence that it doesn’t happen, and we already have a big enough problem with people mistakenly believing that DK showed an inverse correlation between confidence and competence, when they did no such thing.