5 ms·

It's almost impossible to compare IQ tests across countries, because cognition test results are really only valid for people from the cultural context the test

It's almost impossible to compare IQ tests across countries, because cognition test results are really only valid for people from the cultural context the test was written for.

pjerem 2y ago
I suppose you can’t compare the absolute results but you can see if there is a similar trend.
logicchains 2y ago
That's why Raven's progressive matrices were invented. And in spite of it being developed by westerners, East Asian countries tend to score highest on it.
resonious 2y ago
If true, this seems quite damning for IQ as a metric. If the kilogram varied across countries, we would be in big trouble.
- DoreenMichele 2y ago
  It's true, fwiw.
- seanmcdirmid 2y ago
  IQ metrics are continually damned. There is still no real consensus that they are useful at all. But the social sciences is filled with a lot of soft metrics like IQ because studying complex human behavior with hard science is very hard, and probably impossible.
  - pi-e-sigma 2y ago
    [dead]
  - reitanqild 2y ago
    They are also constantly defended. Good, calibrated IQ tests measure something, and that something has a positive correlation with work efficiency in at least certain types of work. That said, there are many types of IQ tests and some of them are in my opinion rather weird, for example anything where your score depends on your knowledge of English language or English history as I hear has traditionally been the case in US. Around here it seems they try to make the tests as little dependent on language, history and even math as possible and the ones I have taken has largely been about: - pattern matching (of these 6 squares with various patterns, which one doesn't fit in?) - sequences (based on these 3 squares with patterns, which is the next one?) - and that kind of stuff These skills are things that are very real, very measurable and often linked to work throughput. Yes, I rarely visualize boxes in my head and rotate them and I rarely sift through through black and white squares with alternating patterns at work, but it seems the ability to do so is often linked to the ability to mentally single step code or spot patterns in bug reports. It doesn't mean that you are good person if your score is high (or low) and it doesn't mean that you will be a good employee and not waste your time on HN.
    - seanmcdirmid 2y ago
      The tests that work in measuring human behavior are fairly simply and very repeatable. Think Fitts's or Hick's Law, which involve hit testing and searching through an unsorted list. I can imagine pattern matching being fairly repeatable as well, but I'm not sure how it really relates to real problem solving. Then again, medical doctors are basically pattern matchers (and why they are fairly excited about AI assistance in...matching patterns).
- chefandy 2y ago
  The approach's limitations are pretty well-known, though the results have always been considered estimates rather than absolute values. The "Reliability and validity" section of the wikipedia page for Intelligence_quotient is actually a pretty solid and well-written overview if you're looking for one.
- viraptor 2y ago
  One of them is https://sci-hub.se/https://pubmed.ncbi.nlm.nih.gov/16123251/ https://sci-hub.se/https://pubmed.ncbi.nlm.nih.gov/16123251/ but there's a few more. Anyway, it seems obvious it's culture dependent. Given a person with usual western education and a person who's never dealt with shapes and abstract straight lines in their life - would you expect them to score similarly if they're both very intelligent people? These tests heavily rely on geometric puzzles which already make an assumption about education. The tricky question is how much does that affect differences you'd see between random people tested for comparison.
- deleted 2y ago
  [deleted]
- abdullahkhalids 2y ago
  The mere assumption that something as multidimensional as intelligence can be quantified along a single axis is already highly suspect. How can you compare the mental abilities of Einstein and Bach and Shakespeare and Magnus Carlesen and Zidane?
Terr_ 2y ago
Except parent poster isn't talking about doing that, they're talking about computing the relative change over time separately and individually for each country within its own singular unique cultural-context et-cetera. Then those ratios are what you compare across countries, which is significantly less-problematic.
invalidusernam3 2y ago
It's not about comparing IQ between countries, it's about pre covid IQ scores with post covid IQ scores within the same country
- danielheath 2y ago
  If two countries both report a 5-point shift, how do you know the shift is actually the same size if the points are counted differently? Perhaps the actual change was bigger or smaller but the approach to measurement highlighted the change more or less strongly. There's no statistical validity to be had.
concordDance 2y ago
> cognition test results are really only valid for people from the cultural context the test was written for. They're perfectly good for looking at changes over time within a population, you just have to be careful doing between-population comparisons (whether a test is "valid" at that point depends what specific question you want to answer).

It's almost impossible to compare IQ tests across countries, because cognition test results are really only valid for people from the cultural context the test was written for.

pjerem 2y ago

I suppose you can’t compare the absolute results but you can see if there is a similar trend.

logicchains 2y ago

That's why Raven's progressive matrices were invented. And in spite of it being developed by westerners, East Asian countries tend to score highest on it.

resonious 2y ago

If true, this seems quite damning for IQ as a metric. If the kilogram varied across countries, we would be in big trouble.

DoreenMichele 2y ago
It's true, fwiw.
seanmcdirmid 2y ago
IQ metrics are continually damned. There is still no real consensus that they are useful at all. But the social sciences is filled with a lot of soft metrics like IQ because studying complex human behavior with hard science is very hard, and probably impossible.
- pi-e-sigma 2y ago
  [dead]
- reitanqild 2y ago
  They are also constantly defended. Good, calibrated IQ tests measure something, and that something has a positive correlation with work efficiency in at least certain types of work. That said, there are many types of IQ tests and some of them are in my opinion rather weird, for example anything where your score depends on your knowledge of English language or English history as I hear has traditionally been the case in US. Around here it seems they try to make the tests as little dependent on language, history and even math as possible and the ones I have taken has largely been about: - pattern matching (of these 6 squares with various patterns, which one doesn't fit in?) - sequences (based on these 3 squares with patterns, which is the next one?) - and that kind of stuff These skills are things that are very real, very measurable and often linked to work throughput. Yes, I rarely visualize boxes in my head and rotate them and I rarely sift through through black and white squares with alternating patterns at work, but it seems the ability to do so is often linked to the ability to mentally single step code or spot patterns in bug reports. It doesn't mean that you are good person if your score is high (or low) and it doesn't mean that you will be a good employee and not waste your time on HN.
  - seanmcdirmid 2y ago
    The tests that work in measuring human behavior are fairly simply and very repeatable. Think Fitts's or Hick's Law, which involve hit testing and searching through an unsorted list. I can imagine pattern matching being fairly repeatable as well, but I'm not sure how it really relates to real problem solving. Then again, medical doctors are basically pattern matchers (and why they are fairly excited about AI assistance in...matching patterns).
chefandy 2y ago
The approach's limitations are pretty well-known, though the results have always been considered estimates rather than absolute values. The "Reliability and validity" section of the wikipedia page for Intelligence_quotient is actually a pretty solid and well-written overview if you're looking for one.
viraptor 2y ago
One of them is https://sci-hub.se/https://pubmed.ncbi.nlm.nih.gov/16123251/ https://sci-hub.se/https://pubmed.ncbi.nlm.nih.gov/16123251/ but there's a few more. Anyway, it seems obvious it's culture dependent. Given a person with usual western education and a person who's never dealt with shapes and abstract straight lines in their life - would you expect them to score similarly if they're both very intelligent people? These tests heavily rely on geometric puzzles which already make an assumption about education. The tricky question is how much does that affect differences you'd see between random people tested for comparison.
deleted 2y ago
[deleted]
abdullahkhalids 2y ago
The mere assumption that something as multidimensional as intelligence can be quantified along a single axis is already highly suspect. How can you compare the mental abilities of Einstein and Bach and Shakespeare and Magnus Carlesen and Zidane?

Terr_ 2y ago

Except parent poster isn't talking about doing that, they're talking about computing the relative change over time separately and individually for each country within its own singular unique cultural-context et-cetera. Then those ratios are what you compare across countries, which is significantly less-problematic.

invalidusernam3 2y ago

It's not about comparing IQ between countries, it's about pre covid IQ scores with post covid IQ scores within the same country

danielheath 2y ago
If two countries both report a 5-point shift, how do you know the shift is actually the same size if the points are counted differently? Perhaps the actual change was bigger or smaller but the approach to measurement highlighted the change more or less strongly. There's no statistical validity to be had.

concordDance 2y ago

> cognition test results are really only valid for people from the cultural context the test was written for. They're perfectly good for looking at changes over time within a population, you just have to be careful doing between-population comparisons (whether a test is "valid" at that point depends what specific question you want to answer).