3 ms·

I think that one of the huge problems is that there has never really been to my knowledge a systematic look at the epistemological limits of IQ tests. However

I think that one of the huge problems is that there has never really been to my knowledge a systematic look at the epistemological limits of IQ tests. However just looking at it quickly I can spot two huge epistemological issues which make IQ tests more or less worthless for objective study, and more or less cultural artifacts. The first, which I have mentioned in a few posts is that at some point you end up comparing apples and melons (the pun for those of you who didn't get it is that 'melon' is Classical Greek for 'apple'). These things may be things where we can make connections but any cultural weight we place on one relative to another is just that, a cultural weight. For this reason intelligence, even if IQ tests measure what we expect them to (and I am not sure of that, see below), the results cannot be objectively reduced to a single number. You would really need to look at intelligence as a multi-dimensional vector with the understanding that we might be missing some aspects anyway. The second problem though that I can spot immediately is that some skills they want to test cannot be objectively tested. The big one that comes to mind is pattern recognition. If you haven't read it, I think Heisenberg's "Physics and Philosophy" is a must read here because his discussions of the relationship between data and theory are directly relevant to this. When we recognize a pattern what we actually do is we project assumptions onto the data and find a pattern which fits both the assumptions and the data. This is the process Heisenberg articulates regarding the formation of scientific theories. The pattern recognized is both a product of our assumptions and the data and therefore patterns are at once recognized and created. Of course Heisenberg was talking about his forte (theoretical physics) but the same applies here because this is a human process. The simple fact is that several competing sets of patterns can be made to fit the data. For example, take the following set of items: Hammer, log, ax, saw Which one doesn't belong? One pattern might be to that "hammer, ax, and saw" are the pattern and log is the outsider. Another pattern might be "I can build something with the hammer, the log, and the saw, much more than I can with any other combination so the ax doesn't belong." Therefore such a question has at least two legitimate answers because two different set of assumptions can be projected onto the set with very different results. Now typically we are going to say one of those (the first, "log" doesn't fit) is correct and the other is wrong, but that doesn't actually test the ability to recognize patterns so much as it tests the ability of the test taker to figure out the answer that the author wants. This makes pattern recognition fundamentally culturally determined and if you can't get away from this you can't quantify this through standardized testing. But let's take this further. We typically refer to a genius as someone who finds useful patterns that nobody else does. The IQ test then, if we accept Heisenberg's epistemology, should actively select against genius in these areas-- either the patterns must be trivial, or they must be sufficiently complex you cannot be sure whether a "wrong" answer in fact does not have a recognizable pattern to it. So you end up having to throw out a good chunk of the test here as being very closely culturally determined (and basically an exercise of mind-reading the author). The first problem could be solved by expressing intelligence as a multidimensional vector, but the second is much more problematic. How do you know that the test answer is right and the test taker is wrong when it comes to two competing patterns seen in the question?

jacques_chester 13y ago
IQ tests are pretty sophisticated and, as the theory of intelligence or more correctly intelligences has evolved, the tests have evolved alongside. Each test is really a suite of dozens of subtests, each of which tries to tease out some aspect of mental function. Some of it is pattern matching. Some of it is memorise-and-recall. Some of it is tests of "general knowledge". And on and on. Think of what you consider to be a distinct kind of mental performance and somebody, somewhere, has built a test for it. I think it's important to distinguish between three things in discussions of IQ: 1. Are some people "more" intelligent than others? Taking the fuzziest, least precise view of what "more intelligent" or "less intelligent" means, I think yes. It would be very strange that the architecture of the brain and environmental factors would lead to perfectly equal outcomes in all subjects. And it doesn't seem to fit what we can casually observe around us. 2. What is intelligence, and what are the components of intelligence? A very large body of research discusses, explores, argues and delves into these simple questions. In general a single "IQ" figure is merely a statistical index of the subtests. The sub-category indices are much more revealing. 3. Can the tests actually accurately sample these components? Also hotly contested. The closest we can come is to compare the test scores statistically to other outcomes, or to other tests. IQ tests which correlate highly with life achievement (factoring out stuff like socioeconomic background, sex, race and mental health) are preferred over those that don't. You can argue that it's circular, and you'd be right. Welcome to psychometrics, where there is no outside world available for objective instrumentation.
- einhverfr 13y ago
  > Taking the fuzziest, least precise view of what "more intelligent" or "less intelligent" means, I think yes. No argument there. > In general a single "IQ" figure is merely a statistical index of the subtests. Which is why I point out that any weighing between subtests are relatively problematic in generating that number. > The sub-category indices are much more revealing. 100% agreed there. > The closest we can come is to compare the test scores statistically to other outcomes, or to other tests. But the problem is, the ability to accurately sample these components may not be the same for each component. For example, if you want to measure coordination of multiple pieces of information short term memory, the problems are very different than if looking at pattern recognition. So I would caution against a single answer. However if we don't understand which subtests are particularly problematic and what the epistemological problems look for each subtest, then many of the really interesting questions about these tests (for example why do Asians score higher than whites) are forever out of reach. > Welcome to psychometrics, where there is no outside world available for objective instrumentation. Yep.

jacques_chester 13y ago

IQ tests are pretty sophisticated and, as the theory of intelligence or more correctly intelligences has evolved, the tests have evolved alongside. Each test is really a suite of dozens of subtests, each of which tries to tease out some aspect of mental function. Some of it is pattern matching. Some of it is memorise-and-recall. Some of it is tests of "general knowledge". And on and on. Think of what you consider to be a distinct kind of mental performance and somebody, somewhere, has built a test for it. I think it's important to distinguish between three things in discussions of IQ: 1. Are some people "more" intelligent than others? Taking the fuzziest, least precise view of what "more intelligent" or "less intelligent" means, I think yes. It would be very strange that the architecture of the brain and environmental factors would lead to perfectly equal outcomes in all subjects. And it doesn't seem to fit what we can casually observe around us. 2. What is intelligence, and what are the components of intelligence? A very large body of research discusses, explores, argues and delves into these simple questions. In general a single "IQ" figure is merely a statistical index of the subtests. The sub-category indices are much more revealing. 3. Can the tests actually accurately sample these components? Also hotly contested. The closest we can come is to compare the test scores statistically to other outcomes, or to other tests. IQ tests which correlate highly with life achievement (factoring out stuff like socioeconomic background, sex, race and mental health) are preferred over those that don't. You can argue that it's circular, and you'd be right. Welcome to psychometrics, where there is no outside world available for objective instrumentation.

einhverfr 13y ago
> Taking the fuzziest, least precise view of what "more intelligent" or "less intelligent" means, I think yes. No argument there. > In general a single "IQ" figure is merely a statistical index of the subtests. Which is why I point out that any weighing between subtests are relatively problematic in generating that number. > The sub-category indices are much more revealing. 100% agreed there. > The closest we can come is to compare the test scores statistically to other outcomes, or to other tests. But the problem is, the ability to accurately sample these components may not be the same for each component. For example, if you want to measure coordination of multiple pieces of information short term memory, the problems are very different than if looking at pattern recognition. So I would caution against a single answer. However if we don't understand which subtests are particularly problematic and what the epistemological problems look for each subtest, then many of the really interesting questions about these tests (for example why do Asians score higher than whites) are forever out of reach. > Welcome to psychometrics, where there is no outside world available for objective instrumentation. Yep.