4 ms·

> The tasks aren't arbitrary. They're meant to be a proxy for some universal concept of competence. I’ve seen absolutely nothing suggesting this. It’s explicit

by samhw 4y ago

> The tasks aren't arbitrary. They're meant to be a proxy for some universal concept of competence. I’ve seen absolutely nothing suggesting this. It’s explicitly about task competency; no particular task is specified nor needs to be specified. > That's why DK is a well known effect, it claims to hold true for anything even though they can't test every possible task. Yes, they claim it holds true for everything because it’s how human beings introspectively experience being poor at a task. It’s really not necessary to have some Platonic ideal of Task Competency … which is then specifically restricted to logical tasks for reasons known only to you. > Logical reasoning was chosen because: It’s objective. I think there’s a kernel of truth in this, albeit assuming by ‘objective’ you instead mean (as people often do) something like “people almost always agree in their evaluations of this quality”. You need that for a good experiment. I’m still not sure how it relates at all to your point here. Personally I would find it easier to just say “I was wrong, it’s not explicitly about logic, I just associated it with that because it’s commonly adduced in silly arguments about logic/intelligence on the internet” - but ah well, it’s an interesting theory so I’m happy to discuss it.

mike_hearn 4y ago
The reason no particular task needs to be specified to invoke DK is exactly because they argue that their initial selection of experimental tasks is so general, that the effect must apply to everything. It feels like you and danbruc are inverting causality here. You start from the assumption that DK is a real effect and then say, because it's real and general, it doesn't matter what tasks they used to prove it. But that's backwards. We have to start from the null hypothesis of no effect existing, and then they have to present evidence that it does in fact exist. And because they claim it's both large and very general, they need to present evidence to support both these ideas. That's why they explicitly argued that their tasks reflect general attributes like wisdom and intelligence: they wanted to be famous for discovering a general effect, not one that only applies in very specific situations. But their tasks aren't great. The worst are ridiculous, the best are unverifiable. Thus the evidence that DK is a real and general effect must either be taken as insufficient, or you could widen the argument to include studies by other psychologists that pursue the same finding via different means.
- danbruc 4y ago
  And because they claim it's both large and very general, they need to present evidence to support both these ideas. To me the claims in the paper do not really seem that strong, almost to the point that I am not sure if they claim anything at all. If you read through the conclusions, they mostly report the findings of their experiments. The closest thing to any claims about generality I can find is that they discuss in which scenarios their findings will not apply. You could maybe read into this that they claim that in all other scenarios their findings apply, but that is not what they actually do. But I guess the better way to discuss this is that you just quote the claims from the paper that you consider too strong and unjustified instead of me trying to anticipate what you are referring to or me going over each claim in the paper.

by samhw 4y ago

mike_hearn 4y ago

The reason no particular task needs to be specified to invoke DK is exactly because they argue that their initial selection of experimental tasks is so general, that the effect must apply to everything. It feels like you and danbruc are inverting causality here. You start from the assumption that DK is a real effect and then say, because it's real and general, it doesn't matter what tasks they used to prove it. But that's backwards. We have to start from the null hypothesis of no effect existing, and then they have to present evidence that it does in fact exist. And because they claim it's both large and very general, they need to present evidence to support both these ideas. That's why they explicitly argued that their tasks reflect general attributes like wisdom and intelligence: they wanted to be famous for discovering a general effect, not one that only applies in very specific situations. But their tasks aren't great. The worst are ridiculous, the best are unverifiable. Thus the evidence that DK is a real and general effect must either be taken as insufficient, or you could widen the argument to include studies by other psychologists that pursue the same finding via different means.

danbruc 4y ago
And because they claim it's both large and very general, they need to present evidence to support both these ideas. To me the claims in the paper do not really seem that strong, almost to the point that I am not sure if they claim anything at all. If you read through the conclusions, they mostly report the findings of their experiments. The closest thing to any claims about generality I can find is that they discuss in which scenarios their findings will not apply. You could maybe read into this that they claim that in all other scenarios their findings apply, but that is not what they actually do. But I guess the better way to discuss this is that you just quote the claims from the paper that you consider too strong and unjustified instead of me trying to anticipate what you are referring to or me going over each claim in the paper.