3 ms·

I guess it depends on what you want out of the test; if you want something meritocratic, what we find is that testing ends up reinforcing existing social strata

I guess it depends on what you want out of the test; if you want something meritocratic, what we find is that testing ends up reinforcing existing social strata more often than not, because wealth often correlates to factors like quality of education, which in turn correlate to test scores. A stupid rich person will always be stupid, but a smart rich person will probably have access to better schools, private tutoring, and test preparation resources. So there's the challenge that the anti-testing/testing reform crowd wants to solve: how do you call testing meritocratic when the equality of opportunity isn't there? And are we ok with that?

sokoloff 5y ago
If we’re trying to measure something (say strength), should we measure that thing as directly as we can, or should we add a bunch of adjustments to make things more “fair”? Do we want to know how strong someone is, or how surprising it is that they’re as strong as they are given their starting point? If someone has a home gym, a personal trainer, and time to workout 2x/day, 6 days a week, how much should we penalize their results? Zero? 25%? 50%? I think that being strong as demonstrated on tests of strength is a valid (and the most valid I’m aware of) measure of “merit” in the context of strength. I think the same is true in other fields as well, including scholastics.
paisawalla 5y ago
> if you want something meritocratic, what we find is that testing ends up reinforcing existing social strata more often than not Compared to what? It's not a criticism to simply state that standardized testing fails to be perfectly meritocratic.
harryh 5y ago
"less than 14% of the variance in SAT scores can be explained by SES in two massive and representative samples" https://freddiedeboer.substack.com/p/please-think-critically-about-college https://freddiedeboer.substack.com/p/please-think-critically...
- commandlinefan 5y ago
  What does SES stand for? Your link doesn’t appear to define it and I can’t find it defined anywhere else.
  - harryh 5y ago
    socioeconomic status https://en.wikipedia.org/wiki/Socioeconomic_status https://en.wikipedia.org/wiki/Socioeconomic_status

sokoloff 5y ago

If we’re trying to measure something (say strength), should we measure that thing as directly as we can, or should we add a bunch of adjustments to make things more “fair”? Do we want to know how strong someone is, or how surprising it is that they’re as strong as they are given their starting point? If someone has a home gym, a personal trainer, and time to workout 2x/day, 6 days a week, how much should we penalize their results? Zero? 25%? 50%? I think that being strong as demonstrated on tests of strength is a valid (and the most valid I’m aware of) measure of “merit” in the context of strength. I think the same is true in other fields as well, including scholastics.

paisawalla 5y ago

> if you want something meritocratic, what we find is that testing ends up reinforcing existing social strata more often than not Compared to what? It's not a criticism to simply state that standardized testing fails to be perfectly meritocratic.

harryh 5y ago

"less than 14% of the variance in SAT scores can be explained by SES in two massive and representative samples" https://freddiedeboer.substack.com/p/please-think-critically-about-college https://freddiedeboer.substack.com/p/please-think-critically...

commandlinefan 5y ago
What does SES stand for? Your link doesn’t appear to define it and I can’t find it defined anywhere else.
- harryh 5y ago
  socioeconomic status https://en.wikipedia.org/wiki/Socioeconomic_status https://en.wikipedia.org/wiki/Socioeconomic_status