15 ms·

The study has an interesting approach to avoiding self-reporting level of confidence. > We propose and use in the paper, an indirect measure of confidence in k

The study has an interesting approach to avoiding self-reporting level of confidence. > We propose and use in the paper, an indirect measure of confidence in knowledge, defined as the ratio of incorrect to ‘don’t know’ answers in any knowledge questionnaire, as long as it had the format true/false/don’t know (or similar). The rationale is that an incorrect answer corresponds to an overestimation of one’s knowledge (more details in the main text). On face value it seems like a good way to avoid the pitfalls of self-report surveys, perhaps also useful in affective modeling.

svnt 3y ago
How do they/you remove the confound of becoming more conservative vs less confident, or do they assume that overconfidence and being conservative are opposites? I might be motivated to be more conservative because I have a reputation to protect, but not actually be less confident. This may have an effect just as much as because I am more experienced. I may also have just learned not to express my overconfidence interpersonally, while still exhibiting it in practice. Expressing appropriate levels of confidence is a skill, not just the absence of overconfidence. I can’t access the paper currently.
- subw00f 3y ago
  > learned not to express my overconfidence interpersonally, while still exhibiting it in practice What does that mean? If you're overconfident in the forest and nobody hears it, it doesn't really matter, right?
  - svnt 3y ago
    If you still act on your confidence, but learn to give the right answers understating it in conversation, it could be that you gave those same right answers on the test reactively and then their interpretation is probably flawed. In this case, additional knowledge wouldn’t actually represent a diminishment of your confidence in a functional sense, but a learned compensation to social feedback.
- cutemonster 3y ago
  Isn't participating in such research usually fairly anonymous? And I'm guessing the researchers didn't tell the participants how they were going to interpret "I don't know" vs wrong answers? > Expressing appropriate levels of confidence is a skill, That's interesting, confidence can be gamed, just like other traits in personality tests
- lusus_naturae 3y ago
  Per my understanding of their method, the participants were just asked to do the survey without mentioning what was being measured. It's like you said, someone can manage their expression of confidence to a degree (or not, maybe), but privately they may not do that, i.e. when just doing some surveys for a study. One thing I remember from running my own studies like this: usually the participant is subjected to a series of such tests (called a battery, sometimes) in succession so they're likely to get fatigued. I think that's a tougher effect that can possibly confound studies. Though I am not sure mental fatigue matters so much for this study. Edit: I'd share the article but have no idea where to host it.
mistermann 3y ago
One issue though is that in realtime cognition under non-laboratory conditions, "I don't know" often isn't available, unlike when it is explicitly given as an answer. The number of science fans I've met who sincerely proclaim they possess knowledge of the unknowable is scary.

svnt 3y ago

How do they/you remove the confound of becoming more conservative vs less confident, or do they assume that overconfidence and being conservative are opposites? I might be motivated to be more conservative because I have a reputation to protect, but not actually be less confident. This may have an effect just as much as because I am more experienced. I may also have just learned not to express my overconfidence interpersonally, while still exhibiting it in practice. Expressing appropriate levels of confidence is a skill, not just the absence of overconfidence. I can’t access the paper currently.

subw00f 3y ago
> learned not to express my overconfidence interpersonally, while still exhibiting it in practice What does that mean? If you're overconfident in the forest and nobody hears it, it doesn't really matter, right?
- svnt 3y ago
  If you still act on your confidence, but learn to give the right answers understating it in conversation, it could be that you gave those same right answers on the test reactively and then their interpretation is probably flawed. In this case, additional knowledge wouldn’t actually represent a diminishment of your confidence in a functional sense, but a learned compensation to social feedback.
cutemonster 3y ago
Isn't participating in such research usually fairly anonymous? And I'm guessing the researchers didn't tell the participants how they were going to interpret "I don't know" vs wrong answers? > Expressing appropriate levels of confidence is a skill, That's interesting, confidence can be gamed, just like other traits in personality tests
lusus_naturae 3y ago
Per my understanding of their method, the participants were just asked to do the survey without mentioning what was being measured. It's like you said, someone can manage their expression of confidence to a degree (or not, maybe), but privately they may not do that, i.e. when just doing some surveys for a study. One thing I remember from running my own studies like this: usually the participant is subjected to a series of such tests (called a battery, sometimes) in succession so they're likely to get fatigued. I think that's a tougher effect that can possibly confound studies. Though I am not sure mental fatigue matters so much for this study. Edit: I'd share the article but have no idea where to host it.

mistermann 3y ago

One issue though is that in realtime cognition under non-laboratory conditions, "I don't know" often isn't available, unlike when it is explicitly given as an answer. The number of science fans I've met who sincerely proclaim they possess knowledge of the unknowable is scary.