4 ms·

Your example isn't remotely close to being analogous to the argument that was made. If you stick a thermometer in hot water, it is measuring the water's tempera

by phaus 13y ago

Your example isn't remotely close to being analogous to the argument that was made. If you stick a thermometer in hot water, it is measuring the water's temperature accurately. If an individual studies for a specific type of IQ test, that particular IQ test fails to accurately measure his general intelligence. The IQ test is generally accepted because there aren't any practical applications where a near-perfect measurement of a person's intelligence is going to be a matter of life and death, the same cannot be said for the measurement of temperature. Basically, it doesn't matter that an IQ test isn't perfect. It is accepted because it does a merely adequate job in most cases, and even if it happened to be manipulated by someone who studied for a specific version of the test, no significant damage will be done to anything or anyone. Just to be clear, I don't have a grudge against IQ tests, in fact, I scored in the 140s on a test administered by a psychologist. I'm just saying that they are highly susceptible to manipulation (Don't worry, I didn't even know I was taking it until I arrived at his office). The entire field of psychology is in its infancy, it is one of the least developed of all sciences. We are making quite a bit of progress, but there's still a long way to go. There are a lot of really strange ideas that are accepted by experts that are going to be proven wrong as soon as our understanding of the mind matures sufficiently.

gwern 13y ago
> Your example isn't remotely close to being analogous to the argument that was made. If you stick a thermometer in hot water, it is measuring the water's temperature accurately. If an individual studies for a specific type of IQ test, that particular IQ test fails to accurately measure his general intelligence. Yes, it is. IQ tests are designed to reliably measure intelligence under certain reasonable, but not adversarial or universal, conditions. Just like a reading off a thermometer is a reliable way of measuring body temperatures under certain reasonable, non-adversarial, non-universal conditions. Memorizing the answers or training by taking the test repeatedly is akin to a kid pretending to be sick and dunking the thermometer in his hot chocolate to get out of school. It's still reporting something, but not what you think it is. > The IQ test is generally accepted because there aren't any practical applications where a near-perfect measurement of a person's intelligence is going to be a matter of life and death, Indeed. Your standard IQ test like a RAPM is not used in adversarial contexts (sadly, 'publish or perish' increasingly means that research is an adversarial context as well), and failure to understand this seems to be leading to a lot of confusion in these comments. If you want to handle even adversarial contexts, you need a procedure way more complex & costly than a 10 minute pen and paper RAPM - you need something like a proctored SAT or GRE. > The entire field of psychology is in its infancy, it is one of the least developed of all sciences. We are making quite a bit of progress, but there's still a long way to go. Intelligence and measuring it via IQ tests is some of the oldest and most conceptually & statistically deep and commonly-used parts of psychology, going back a century at this point, which is something few parts of psychology can claim. I wouldn't hold my breath waiting for psychology to abandon it because it's in 'its infancy' and 'is one of the least developed of all sciences'. At this point, it's roughly like expecting the spacing effect to go away.

by phaus 13y ago

gwern 13y ago

> Your example isn't remotely close to being analogous to the argument that was made. If you stick a thermometer in hot water, it is measuring the water's temperature accurately. If an individual studies for a specific type of IQ test, that particular IQ test fails to accurately measure his general intelligence. Yes, it is. IQ tests are designed to reliably measure intelligence under certain reasonable, but not adversarial or universal, conditions. Just like a reading off a thermometer is a reliable way of measuring body temperatures under certain reasonable, non-adversarial, non-universal conditions. Memorizing the answers or training by taking the test repeatedly is akin to a kid pretending to be sick and dunking the thermometer in his hot chocolate to get out of school. It's still reporting something, but not what you think it is. > The IQ test is generally accepted because there aren't any practical applications where a near-perfect measurement of a person's intelligence is going to be a matter of life and death, Indeed. Your standard IQ test like a RAPM is not used in adversarial contexts (sadly, 'publish or perish' increasingly means that research is an adversarial context as well), and failure to understand this seems to be leading to a lot of confusion in these comments. If you want to handle even adversarial contexts, you need a procedure way more complex & costly than a 10 minute pen and paper RAPM - you need something like a proctored SAT or GRE. > The entire field of psychology is in its infancy, it is one of the least developed of all sciences. We are making quite a bit of progress, but there's still a long way to go. Intelligence and measuring it via IQ tests is some of the oldest and most conceptually & statistically deep and commonly-used parts of psychology, going back a century at this point, which is something few parts of psychology can claim. I wouldn't hold my breath waiting for psychology to abandon it because it's in 'its infancy' and 'is one of the least developed of all sciences'. At this point, it's roughly like expecting the spacing effect to go away.