Agreed, the positive correlation makes the results harder to judge, and specifically it means that one cannot reliably use confidence to predict skill, other th

Agreed, the positive correlation makes the results harder to judge, and specifically it means that one cannot reliably use confidence to predict skill, other than higher confidence is generally associated with higher skill. It's unfortunate how many people cite DK to erroneously justify linking confidence with lack of skill. Based on the paper, the only thing we can say about skill by looking at confidence is that more confident people are statistically likely to be more skilled. You are right, they did mention self-evaluation of score. "Lest one think these results reflect erroneous peer assessment rather then erroneous self-assessment, participants in the bottom quartile also overestimated the number of test items they had gotten right by nearly 50%." But the results and conclusions of all the studies are focusing on the rankings and not the scores. It's difficult to even understand what the absolute range of scores is, and of course we don't see the actual questions they asked, so we don't know what the scores mean. Figures 2 and 3 say to me that zero people in the study knew how well they ranked or what their score was, the survey data is basically flat with noise. This could be explained by questions that are hard to understand, or by participants guessing at the answers. They didn't propose those explanations though, they wax on and on about how people who underperform overestimate their abilities. How do we know the participants even tried to answer correctly? They were all undergrads getting extra credit (which suggests they needed extra credit and were within the range of grades where it might help to get extra credit... consider what that means relative to skill: it's a narrow range of students not getting an F nor getting an A.) And they were getting extra credit just for being there, not for answering correctly. What if they were lazy? What if they were told their answers didn't matter? Why should we assume they didn't answer randomly? Even if they did try, the participants were all Ivy League undergrads who have been told they're among the elite and made it through a highly selective process of admission. Of course they rate & rank themselves highly. 100% of the participants were in a single college psych class, there were no uneducated people, no morons, and nobody who actually had meaningful "deficits in metacognitive skill" relative to the general population. Why is the set of people used in this study not considered an invalidating confounding factor for the results?