41 ms·

>Why would psychologists believe something so clearly wrong? I would imagine from their research? Why are you so self-assured that you are right? Often the i

by avs733 5y ago

>Why would psychologists believe something so clearly wrong? I would imagine from their research? Why are you so self-assured that you are right? Often the issue with 'IQ' tests is that the way they are written introduces biases. Any definition of intelligence is at some level socially constructed, which brings social biases into play. Designing accurate, fair, and reliable tests of any educational measure is difficult. Doing a generalized one is even more so. really obvious example: I write a question that references a baseball game, some people have the tacit knowledge that there are nine fielders and one batter...some don't. Am I measuring education? information gained? 'intelligence'? Is someone from India who makes an analogy to cricket more intelligent than someone who doesn't? IQ tests have generally been shown to mostly be predictive of...scores on IQ tests. Fundamentally, when dealing with people you are dealing with latent constructs. They, by definition, cannot be measured directly and we presume they cannot be measured exactly. The issue in this paper is they aren't even really defined - they measured cognition not 'intelligence'.

TheOtherHobbes 5y ago
LSATs and verbal reasoning tests are very obviously culturally informed. Even so - questions about baseball games are very much not something you'll see in an IQ test. But it's not a simple issue. Tests like Raven's Progressive Matrices looks like they should be culture-free but in fact they're based on assumptions, contexts, and operations that are taught implicitly in basic math classes in the West. If you don't have that education, you're going to find it hard to understand the test. Other culture have different ideas about what intelligence looks like and how it's expressed. If you threw equivalent non-Western challenges at a supposedly smart Western kid it's not obvious how well they'd do. https://www.apa.org/monitor/feb03/intelligence https://www.apa.org/monitor/feb03/intelligence
- avs733 5y ago
  totally agree with this post, I used a simple example where many more complicated ones exist. The problem is how do you disentangle Differential Item/Test Function being from cultural purposes as compared to 'we expect this group to perform worse'. The larger point is that culture, intelligence, and measures of culture developed in a particular culture are difficult to disambiguate.
- sudosysgen 5y ago
  Indeed. After taking a progressive matrices I found that half of the second half was just XOR on bitmaps.
xyzzyz 5y ago
You speak as if these issues have not been studied to death a generation ago already. Jensen’s “Bias in mental testing”, a nearly 800 pages volume full of references, covers it rather extensively, and it came out 40 years ago. What makes me self assured here is that best counter arguments to the body of scientific research I read on this topic that I hear is people musing about some hypothetical IQ tests that might possibly ask about baseball. I mean, consider Raven’s progressive matrices. It’s not the best IQ test (for reasons other than knowledge bias), but it correlated very well with WAIS and Stanford-Binet, two most popular and arguably the best IQ test. It asks for no knowledge of baseball, regattas, or anything else, in fact, it used no words at all. How comes that it nevertheless gives the same results as other, allegedly biased IQ tests? What exactly makes it biased, and why in exactly the same direction as other tests?
- sudosysgen 5y ago
  It doesn't give the same results as other IQ tests, it's correlation is nowhere near as strong as you let on, the average expected difference is 2/3rds of a standard deviation. Beyond that, yes, RPM is culturally biased. If you have experience with basic geometrical manipulations and bitmap manipulation you will find it much easier. For example if you have CS training you will notice that half or more of the questions in many RPM style tests are basically XORing two images together, with a small twist some times. You'll also find it difficult to argue that that people who culturally do less manipulation and work on paper or on a screen will have a more difficult time. Beyond that, there is clearly a body of research that finds IQ tests to be culturally loaded, and solid research that shows that even further studies in the same society where the IQ test was developped has a significant and measurable impact.
Auracle 5y ago
Tell me the biases involved on tests like this: https://www.mensa.org/public/mensa-iq-challenge https://www.mensa.org/public/mensa-iq-challenge IQ tests are incredibly predictive of a whole host of outcomes.
- avs733 5y ago
  well for starters: the test instructions are in English - which doesn't seem relevant to the test, it focuses on abstraction as a modality of measuring cognitive capability, test taking, and computer user interface knowledge. I work with high performing undergraduate students (think top 10 US public university). In one of our classes, they work on global public health design projects. Last year, the project was centered in the country of Chad. One thing the students learned was that if someone isn't part of a culture that regularly uses paper/2D representations of things then using those modalities to communicate prototypes doesn't work. Does that mean my students are smarter than the Chadian people? Why? Because they come from a culture where interacting with abstractions and representations of information is normed? The people of Chad certainly can. If they are smarter shouldn't I be able to drop my students near Lake Chad and let them learn how to navigate without a map and compass? its just cognition so they should be able to figure it out right? They could memorize the map in advance? It's not that 'some things have bias' its that all things are socially constructed at some level...
  - Auracle 5y ago
    Right, but that was just an example of the types of questions on most IQ tests. The one I was given at a psychiatrist's office had no words or even letters for the multiple choice answers. As far as dropping your students near Lake Chad, that's not general intelligence. Babies born in Chad learn how to navigate their area, just as your students can. The question would be who would be able to learn how to do it faster, and how many of each group could learn more complex things.
    - avs733 5y ago
      So what is general intelligence? And why does it always seem like definitions of general intelligence just happen to align with the current hegemonic culture?

by avs733 5y ago

TheOtherHobbes 5y ago

LSATs and verbal reasoning tests are very obviously culturally informed. Even so - questions about baseball games are very much not something you'll see in an IQ test. But it's not a simple issue. Tests like Raven's Progressive Matrices looks like they should be culture-free but in fact they're based on assumptions, contexts, and operations that are taught implicitly in basic math classes in the West. If you don't have that education, you're going to find it hard to understand the test. Other culture have different ideas about what intelligence looks like and how it's expressed. If you threw equivalent non-Western challenges at a supposedly smart Western kid it's not obvious how well they'd do. https://www.apa.org/monitor/feb03/intelligence https://www.apa.org/monitor/feb03/intelligence

avs733 5y ago
totally agree with this post, I used a simple example where many more complicated ones exist. The problem is how do you disentangle Differential Item/Test Function being from cultural purposes as compared to 'we expect this group to perform worse'. The larger point is that culture, intelligence, and measures of culture developed in a particular culture are difficult to disambiguate.
sudosysgen 5y ago
Indeed. After taking a progressive matrices I found that half of the second half was just XOR on bitmaps.

xyzzyz 5y ago

You speak as if these issues have not been studied to death a generation ago already. Jensen’s “Bias in mental testing”, a nearly 800 pages volume full of references, covers it rather extensively, and it came out 40 years ago. What makes me self assured here is that best counter arguments to the body of scientific research I read on this topic that I hear is people musing about some hypothetical IQ tests that might possibly ask about baseball. I mean, consider Raven’s progressive matrices. It’s not the best IQ test (for reasons other than knowledge bias), but it correlated very well with WAIS and Stanford-Binet, two most popular and arguably the best IQ test. It asks for no knowledge of baseball, regattas, or anything else, in fact, it used no words at all. How comes that it nevertheless gives the same results as other, allegedly biased IQ tests? What exactly makes it biased, and why in exactly the same direction as other tests?

sudosysgen 5y ago
It doesn't give the same results as other IQ tests, it's correlation is nowhere near as strong as you let on, the average expected difference is 2/3rds of a standard deviation. Beyond that, yes, RPM is culturally biased. If you have experience with basic geometrical manipulations and bitmap manipulation you will find it much easier. For example if you have CS training you will notice that half or more of the questions in many RPM style tests are basically XORing two images together, with a small twist some times. You'll also find it difficult to argue that that people who culturally do less manipulation and work on paper or on a screen will have a more difficult time. Beyond that, there is clearly a body of research that finds IQ tests to be culturally loaded, and solid research that shows that even further studies in the same society where the IQ test was developped has a significant and measurable impact.

Auracle 5y ago

Tell me the biases involved on tests like this: https://www.mensa.org/public/mensa-iq-challenge https://www.mensa.org/public/mensa-iq-challenge IQ tests are incredibly predictive of a whole host of outcomes.

avs733 5y ago
well for starters: the test instructions are in English - which doesn't seem relevant to the test, it focuses on abstraction as a modality of measuring cognitive capability, test taking, and computer user interface knowledge. I work with high performing undergraduate students (think top 10 US public university). In one of our classes, they work on global public health design projects. Last year, the project was centered in the country of Chad. One thing the students learned was that if someone isn't part of a culture that regularly uses paper/2D representations of things then using those modalities to communicate prototypes doesn't work. Does that mean my students are smarter than the Chadian people? Why? Because they come from a culture where interacting with abstractions and representations of information is normed? The people of Chad certainly can. If they are smarter shouldn't I be able to drop my students near Lake Chad and let them learn how to navigate without a map and compass? its just cognition so they should be able to figure it out right? They could memorize the map in advance? It's not that 'some things have bias' its that all things are socially constructed at some level...
- Auracle 5y ago
  Right, but that was just an example of the types of questions on most IQ tests. The one I was given at a psychiatrist's office had no words or even letters for the multiple choice answers. As far as dropping your students near Lake Chad, that's not general intelligence. Babies born in Chad learn how to navigate their area, just as your students can. The question would be who would be able to learn how to do it faster, and how many of each group could learn more complex things.
  - avs733 5y ago
    So what is general intelligence? And why does it always seem like definitions of general intelligence just happen to align with the current hegemonic culture?