4 ms·
> Exactly, that was my thought. How would it be possible to get anything other than the D-K effect, even if it wasn't just averaging to the mean? Quite easily
by anonymouskimmer 3y ago
> Exactly, that was my thought. How would it be possible to get anything other than the D-K effect, even if it wasn't just averaging to the mean?
Quite easily with the method they demonstrate in the study in figure 11. In that study test participants are not rating themselves in terms of population percentages, but in terms of the percentage correct they got on the test. In such a case the test could be designed to have a huge ceiling that even the most knowledgeable participants would have trouble reaching. And could have such a low floor that even the least knowledgeable participants would still get some answers correct (unless they weren't even trying, which would allow throwing out their data points).
With 20 questions you could have four gimmes and four impossible questions, bounding the worst participants to about 20% and the best to about 80%.
- SamBam 3y agoRight. To clarify, I meant: with the original study design, how could they not have gotten the result they did? (And that's rhetorical.)
- anonymouskimmer 3y agoIt would have been noteworthy in the original design if more than one group of participants were, on average, within their quartiles on the guessing. I also find it noteworthy that the average guess of the lowest quartile is lower than the average guess of the second lowest quartile, and on up the quartiles. On one hand this shows some awareness of relative ability along a massively smooshed logarithmic scale. On the other hand I wonder if this laddering follows as the averages are split into quintiles and deciles.