8 ms·
This may not apply to the author of the linked blog post, but it's long been a pet peeve of mine how casually people dismiss intelligence testing after seeing o
by goodside 17y ago
This may not apply to the author of the linked blog post, but it's long been a pet peeve of mine how casually people dismiss intelligence testing after seeing one or two examples of poorly designed questions. The rarely considered reality is that IQ tests vary widely in quality.
Some reasons are obvious: The test might be intentionally designed to measure something other than IQ (success in college, performance on a job, money spent on College Board® prep courses). The test might be designed to test within a particular IQ range, whether high or low. The test might be designed only to be taken by people fluent in a particular language, or who received an education from a particular school. And so on.
But the biggest and least obvious reason why bad IQ tests are everywhere is that good IQ tests are not only difficult to make, but expensive to make. Good tests are not made by geniuses in armchairs thinking up riddles. Test design is an empirical science, and the final judges of any question's value are the statistical relationships that show up in the answers when the test is given.
Empirical evidence sterilizes tests from the biases of their authors, and does so much more effectively than waiting for complaints from morally outraged observers. Questions that penalize takers of different cultural backgrounds are bad questions not because they're socially repugnant, but because they're noisy--you can demonstrate that people who otherwise score highly get the question wrong more often than they should. Questions like those addressed in the blog post, where some really smart people are attracted to a "wrong" answer, will have negative inter-question correlations among the brightest subjects, which is a giant blinking neon sign in any analysis.
This sanitizing effect is so strong that the tests which show the most desirable psychometric properties, such as Raven's Progressive Matrices, often evoke the opposite complaint: they're so minimal and homogenous in form that it's hard to accept they're a better measure of IQ than scattershot tests of imperfect analogies and odd-one-out puzzles. If you're curious, RPM goes a little something like this: http://iqtest.dk/main.swf http://iqtest.dk/main.swf
IQ testing is a complicated science, and discrediting it because there's one or two (or fifty) dumb questions on the SATs is like giving up on journalism after reading an issue of USA Today.
- hooande 17y agoStandardized tests of any kind are fundamentally flawed. You can't measure a person's intelligence using a test. You can only measure very specific things, and even then you can't expect to do a good job of it. Does a high SAT score predict success in college? People with great test scores drop out / fail out / join the peace corps by the thousands every year. Testing can a good indicator, but it's not an ideal solution. I'm not against testing. There has to be some way to decide who gets into what college, and the SAT is a published standard. But there's no point in defending a flawed system. Let's just accept that IQ tests are a bit ridiculous and move on.
- jjs 17y ago> You can't measure a person's intelligence using a test. Well, you could, but first you must define it! ;)
- camccann 17y agoIt's been said that IQ tests measure IQ, where IQ is defined as "that quantity which is measured by IQ tests". The interesting part is that, even defined in that rather circular manner, IQ can be consistently measured and compared between individuals and has strong, reliable correlations with performance on other tasks commonly seen as intelligence-based, as well as correlating with both academic and professional success. Yes, these are correlations, not guarantees, but don't underestimate the value of that, and it's not actually clear that there's any better way. The unknown quantity that such tests are trying to measure is usually referred to as g, or the general intelligence factor. Look up the term "g-loaded" for more information.
- asciilifeform 17y ago> You can't measure a person's intelligence using a test. Do you actually believe this dogma? This Usenet post should be read by all "IQ deniers": http://tiny.cc/F7gxA http://tiny.cc/F7gxA "according to IQ doubters, knowing how smart I am should tell you nothing about my IQ. It is as likely to be 50 as 150." The same poster once proposed a beautiful test of the depth of an IQ denier's beliefs: You are to be operated on, and have been given a choice between two surgeons. You know nothing about them, save for the IQ of each. Will you choose the surgeon with the lower IQ? What if the difference between them is an entire standard deviation? Two standard deviations? If you truly believe that IQ tests measure nothing - that the result of any such test is essentially a random number - you should happily choose the surgeon with the lower score. If you do not believe it, we are merely quibbling over the accuracy of existing tests. Moreover, the only way to seriously dismiss the existing IQ tests as hopelessly inaccurate is to borrow tricks from the Creationists' playbook wholesale (as Steven Jay Gould, the patron saint of IQ denial, often did) and close your eyes (and more importantly, those of your audience) to reams of high-quality data. More references, for those actually interested: http://www.psych.utoronto.ca/users/reingold/courses/intelligence/cache/1198gottfred.html http://www.psych.utoronto.ca/users/reingold/courses/intellig... And if you want to know the core driving force behind the IQ denialists: http://www.gnxp.com/blog/2007/10/james-watson-tells-inconvenient-truth_296.php http://www.gnxp.com/blog/2007/10/james-watson-tells-inconven... "Watson was thinking like a scientist. Which is exactly why he was punished. The moral laws of our society dictate that we are not allowed to think scientifically about some issues. Especially not in public."