3 ms·

They are controversial within popular culture but the idea that "Ability to solve these puzzles correlates with ability in a diverse range of cognitive abilitie

They are controversial within popular culture but the idea that "Ability to solve these puzzles correlates with ability in a diverse range of cognitive abilities" is not controversial within the relevant scientific community. Here's a nice short book which cites a lot of literature: https://www.amazon.com/Intelligence-That-Matters-Stuart-Ritchie https://www.amazon.com/Intelligence-That-Matters-Stuart-Ritc... There's even a vox explainer on it, hard to believe they would publish this today: https://www.vox.com/2016/5/24/11723182/iq-test-intelligence https://www.vox.com/2016/5/24/11723182/iq-test-intelligence

andybak 4y ago
OK. A few off the cuff objections: 1. It seems fairly widely reported that you can practice at IQ tests and get better. Are there any studies confirming or denying this? I presume you'd agree that if true it rather kills the validity of IQ tests dead. 2. I have ADHD and I'm acutely aware of how environmental factors affect my performance in many tasks. How do you square this with the idea that they measure an objective innate quality? 3. How do you account for the Flynn Effect? https://en.wikipedia.org/wiki/Flynn_effect https://en.wikipedia.org/wiki/Flynn_effect - are people getting smarter? 4. What's your thoughts on the argument that it's impossible to create genuinely culture free tests? This was something I was taught at college and it made sense to me then. Has it been debunked? Conclusively? 5. Steven Jay Gould wrote an entire book critiquing IQ tests : https://en.wikipedia.org/wiki/The_Mismeasure_of_Man https://en.wikipedia.org/wiki/The_Mismeasure_of_Man - do any of the points he raises stand up to scrutiny?
- brian_cloutier 4y ago
  Correct me if I'm wrong, but (1) and (2) and (4) all seem to come from a place of believing that if a test is not 100% accurate then it is useless. Hopefully when I phrase it that way you can see why none of these objections "kill the validity of IQ tests dead". (3) is very interesting and there is not yet any agreement as to what is causing it. Flynn seems to have thought our culture is slowly becoming better at training us to think abstractly. There's some evidence for this: one Australian study found that over 20 years the students in a set of schools became better at a vocabulary test but did not improve at all in processing speed. This is not what you would expect to see if the improvements were due to better nutrition or less environmental lead. For (5), haven't read it. He's stunningly erudite and a careful author though, it's worth taking him seriously. Do you have some examples of points he raised?
- solveit 4y ago
  1. No, it only kills the validity of IQ tests for which the examinee has practiced, and only if the amount of practice is unknown and the amount of improvement is large and unpredictable. Nevertheless IQ tests correlate with performance on a wide range of cognitive tasks. 2. Environmental factors affect everything. You try to standardise them as much as possible for an operational definition of the thing you're measuring. You will never succeed completely. Nevertheless IQ tests time correlate with performance on a wide range of cognitive tasks. 3. Not sure anybody really understands what's going on with the Flynn effect. Nevertheless IQ tests correlate with performance on a wide range of cognitive tasks. 4. I'm sure it is in fact impossible on some level to create genuinely culture free tests. If we're going to be anal about this, this is a consequence of the various no-free-lunch theorems in statistical inference. For any culture/person/algorithm there are problems where they'll do better relative to others, and problems where they'll do worse. Nevertheless IQ tests correlate with performance on a wide range of cognitive tasks. 5. I'm sure they do, but I haven't read the book and the summary in the Wikipedia article is too vague to tell. None of these objections actually engage with the parent comment's assertion that "Ability to solve these puzzles correlates with ability in a diverse range of cognitive abilities".

andybak 4y ago

OK. A few off the cuff objections: 1. It seems fairly widely reported that you can practice at IQ tests and get better. Are there any studies confirming or denying this? I presume you'd agree that if true it rather kills the validity of IQ tests dead. 2. I have ADHD and I'm acutely aware of how environmental factors affect my performance in many tasks. How do you square this with the idea that they measure an objective innate quality? 3. How do you account for the Flynn Effect? https://en.wikipedia.org/wiki/Flynn_effect https://en.wikipedia.org/wiki/Flynn_effect - are people getting smarter? 4. What's your thoughts on the argument that it's impossible to create genuinely culture free tests? This was something I was taught at college and it made sense to me then. Has it been debunked? Conclusively? 5. Steven Jay Gould wrote an entire book critiquing IQ tests : https://en.wikipedia.org/wiki/The_Mismeasure_of_Man https://en.wikipedia.org/wiki/The_Mismeasure_of_Man - do any of the points he raises stand up to scrutiny?

brian_cloutier 4y ago
Correct me if I'm wrong, but (1) and (2) and (4) all seem to come from a place of believing that if a test is not 100% accurate then it is useless. Hopefully when I phrase it that way you can see why none of these objections "kill the validity of IQ tests dead". (3) is very interesting and there is not yet any agreement as to what is causing it. Flynn seems to have thought our culture is slowly becoming better at training us to think abstractly. There's some evidence for this: one Australian study found that over 20 years the students in a set of schools became better at a vocabulary test but did not improve at all in processing speed. This is not what you would expect to see if the improvements were due to better nutrition or less environmental lead. For (5), haven't read it. He's stunningly erudite and a careful author though, it's worth taking him seriously. Do you have some examples of points he raised?
solveit 4y ago
1. No, it only kills the validity of IQ tests for which the examinee has practiced, and only if the amount of practice is unknown and the amount of improvement is large and unpredictable. Nevertheless IQ tests correlate with performance on a wide range of cognitive tasks. 2. Environmental factors affect everything. You try to standardise them as much as possible for an operational definition of the thing you're measuring. You will never succeed completely. Nevertheless IQ tests time correlate with performance on a wide range of cognitive tasks. 3. Not sure anybody really understands what's going on with the Flynn effect. Nevertheless IQ tests correlate with performance on a wide range of cognitive tasks. 4. I'm sure it is in fact impossible on some level to create genuinely culture free tests. If we're going to be anal about this, this is a consequence of the various no-free-lunch theorems in statistical inference. For any culture/person/algorithm there are problems where they'll do better relative to others, and problems where they'll do worse. Nevertheless IQ tests correlate with performance on a wide range of cognitive tasks. 5. I'm sure they do, but I haven't read the book and the summary in the Wikipedia article is too vague to tell. None of these objections actually engage with the parent comment's assertion that "Ability to solve these puzzles correlates with ability in a diverse range of cognitive abilities".