3 ms·

The real solution is to design a test that you can't study for effectively. However, I suspect the study-ability for the SAT enhances it's predictive ability.

The real solution is to design a test that you can't study for effectively. However, I suspect the study-ability for the SAT enhances it's predictive ability. Someone who does whatever it takes to learn some words for vocab testing will do whatever it takes to learn about sociology. The truth is that standardized tests aren't less predictive for poor students. Whatever causes the score gap also causes an achievement gap once they arrive at school.

Bartweiss 10y ago
> The real solution is to design a test that you can't study for effectively. There's a very strong pressure against that. The analogy section of the SAT was a proxy IQ test, though more study-able than Raven's Matrices. People objected to the parts that could be studied (infamously, "runner : track :: oarsman : regatta"), but the solution was to drop the entire section and preserve only the more learnable portions. Which is frustrating, because as far as I know it undermined your last comment - SAT scores are actually more responsive to wealth than they used to be, because factors like study and the quality of the local math department are more important.
noxToken 10y ago
How do you design a test for which you can't study? Even if everything is laid out in the test with complete hypothetical knowledge (e.g. redefine addition for all real numbers from x + y = z to something else like x + y + (x/2) = z), you can still study for it by honing in on basic arithmetic and algebra. This isn't meant as snark. I really don't understand.
- rhino369 10y ago
  The point of the tests are to test aptitude not knowledge. So if someone can spend 10 hours learning basic algebra and massively raise their score the test is failing at being an aptitude test, it's now a knowledge test. If the college wants to know how much alegrba knowledge a person has they can just look at their math grades. They want to know aptitude. Some standardized tests have fairly narrow topics or have patterns in their problems that are learnable. Beyond just testing knowledge, these only specifically reward studying for the test. The SAT used to have a set of "SAT words" that you could memorize for the verbal section. That somewhat helps your vocab, but only to the words used on teh SAT. The ACT used to have a trick where the short answer of the grammar section was the right answer like 70% of the time. The LSAT has logic game problems that can be categorized and then solved with specific techniques. If you learn the techniques you'll never get more than a few questions wrong. This is purely rewarding studying for the test. If a new type of test is used the techniques fail until someone creates new techniques. If you just learn the five most common equations for SAT algebra your score jumps up but your math ability isn't much better.

Bartweiss 10y ago

> The real solution is to design a test that you can't study for effectively. There's a very strong pressure against that. The analogy section of the SAT was a proxy IQ test, though more study-able than Raven's Matrices. People objected to the parts that could be studied (infamously, "runner : track :: oarsman : regatta"), but the solution was to drop the entire section and preserve only the more learnable portions. Which is frustrating, because as far as I know it undermined your last comment - SAT scores are actually more responsive to wealth than they used to be, because factors like study and the quality of the local math department are more important.

noxToken 10y ago

How do you design a test for which you can't study? Even if everything is laid out in the test with complete hypothetical knowledge (e.g. redefine addition for all real numbers from x + y = z to something else like x + y + (x/2) = z), you can still study for it by honing in on basic arithmetic and algebra. This isn't meant as snark. I really don't understand.

rhino369 10y ago
The point of the tests are to test aptitude not knowledge. So if someone can spend 10 hours learning basic algebra and massively raise their score the test is failing at being an aptitude test, it's now a knowledge test. If the college wants to know how much alegrba knowledge a person has they can just look at their math grades. They want to know aptitude. Some standardized tests have fairly narrow topics or have patterns in their problems that are learnable. Beyond just testing knowledge, these only specifically reward studying for the test. The SAT used to have a set of "SAT words" that you could memorize for the verbal section. That somewhat helps your vocab, but only to the words used on teh SAT. The ACT used to have a trick where the short answer of the grammar section was the right answer like 70% of the time. The LSAT has logic game problems that can be categorized and then solved with specific techniques. If you learn the techniques you'll never get more than a few questions wrong. This is purely rewarding studying for the test. If a new type of test is used the techniques fail until someone creates new techniques. If you just learn the five most common equations for SAT algebra your score jumps up but your math ability isn't much better.