4 ms·

What's disturbing to me — and this cuts both ways — is the subtle shifting of what's meant by "meritocracy". Here, on HN and on Scott's post, there's an implic

What's disturbing to me — and this cuts both ways — is the subtle shifting of what's meant by "meritocracy". Here, on HN and on Scott's post, there's an implicit equating of "meritocracy" with "test score." Do people here really think test score and ability — to say the least of merit per se — are equivalent? Go find a scatterplot of some data correlated 0.80 (e.g., https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3576830/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3576830/). That's a very generous estimate of how correlated that test score will be with actual ability, whatever that means. The correlation probably is lower, like 0.50-0.70. Look at the y axis. Now cut it and take the upper fraction of your choice. See how wide a range of the x axis is represented? That's the problem. The reason why test scores are a problem is because when you select on an imperfect indicator, at some level you are randomly selecting with regard to actual ability, which is even further removed from merit. Throw in any kind of bias of any sort associated with anything, and the problem gets worse. The truth is, it was already a lottery before. It was just a lottery that everyone pretended wasn't a lottery, and that's the problem people have with it. The problem with meritocracy isn't the idea of selecting people based on merit, it's the idea that test scores or other such quantitative metrics are equivalent to merit. The analogy with sports is fundamentally flawed because, among other things, the metrics are the thing of interest in sports. If you are interested in who can swim 50m the fastest, then your speed on the 50m swim is the metric of interest. In school, though, presumably you're not interested in test score, you're interested in someone's ability to solve problems, or create things, or be productive, or whatever. And we're not talking about sport, we're talking about gateways into careers in general. If we are selecting into schools to maximize ability to achieve high test scores on standardized tests per se, then at least be frank and honest about that, and accept that what you are arguing is that the school is an institution whose purpose is to achieve high standardized test scores, and nothing more. But if you want to argue that the school should produce people who are solving problems, innovating, and so forth, then you need to be honest about selection based on test scores, because the scores aren't what they intend to measure. Scott laments the loss of the ability of some kid to work their way into the school based on some test score. But shouldn't he be screaming about a system that doesn't provide resources to kids where they're at? That assumes the kid's ability is equivalent to their score on some entrance exam? What about those problems? I agree that throwing out test score and relying on GPA per se is a problem; relying on GPA alone is just as much of a problem as relying on test score alone. You could, for example, have a lottery for everyone whose score is above some generous threshold, or for everyone whose GPA or score is above some threshold. But staying with the system as it is seems at worst like turning a blind eye to systemic injustice, and at best like missing the forest for the trees.

rayiner 6y ago
You’re kinda smushing together two different things: reliability and merit. The SAT (which is similar to the TJ test) is highly reliable—meaning that it’s measuring something real, and not just producing a “random” result. https://joss.tcnj.edu/wp-content/uploads/sites/176/2012/04/2007-Stickler-SAT-Critical-Review.pdf https://joss.tcnj.edu/wp-content/uploads/sites/176/2012/04/2... > How reliable is the SAT? The College Board website reports that a student’s true score is within 30 points of his or her measured score (SEM = 30; College Board, FAQ, 2005). A test with a standard deviation of 100 and a SEM of 30 has an internal consistency coefficient of .91 (α = .91). In a study specifically investigating SAT I score change upon repeated testing, 1,120,563 students in the 1997 college- bound cohort who took the SAT I one to five times in their junior and senior years gained an average of 7 to 13 points on the verbal section and 8 to 16 points on the mathematics section (Nathan & Camara, 1998). Thus, a student who retested at the higher end of both ranges still would not breach the standard error of measurement, indicating high test-retest reliability. Additionally, in a study of the effects of commercial coaching on SAT scores, Powers and Rock (1999) found that the average increase in student scores after intensive coaching was only 30 points on the verbal section and 50 points on the mathematics section, demonstrating that SAT scores remain relatively stable even with intervening practice. With an SEM of 30, there is less than a 1% chance that a 100 point difference, between a 600 and a 500 on a section is due to random chance. The difference between a median score and and a TJ level score is 250. Validity, whether the test measures something we care about is a separate issue.
scarmig 6y ago
Metrics aren't the thing of interest in sports. Randomly looking at the justifications for some of the top high school athletics programs in California: "Honor, Glory and Love... the school is dedicated to the development of the whole person: spiritual, intellectual, physical, emotional and aesthetic" "students learn lifelong lessons in teamwork, discipline and leadership" "We encourage the development of team collaboration, communication, and characteristics that foster empathy, tolerance, personal growth and positive citizenship." I mean, it's all bullshit. Or at least, people aren't buying expensive houses in good school districts just so their kids can learn the value of teamwork. But if you take athletics' programs words at face value, winning and metrics aren't the primary objective. Specialized high schools are intended to build impressive application packets for college, which in turn is meant to get access to elite jobs and social opportunities. Indeed, that's one of the major reasons parents even try to get their kids into sports! One of the easiest ways into an elite university is to become an above average athlete in an obscure sport (this is the go-to approach for well-off but academically mediocre white people, actually). And so the admissions process for these high schools is geared toward facilitating that objective. TJ provided a pathway for academically gifted first generation Asian immigrants to achieve social mobility. And with this new process, many won't have access to TJ, to mostly be replaced with mediocre white kids (and a couple more token underrepresented minorities).

rayiner 6y ago

You’re kinda smushing together two different things: reliability and merit. The SAT (which is similar to the TJ test) is highly reliable—meaning that it’s measuring something real, and not just producing a “random” result. https://joss.tcnj.edu/wp-content/uploads/sites/176/2012/04/2007-Stickler-SAT-Critical-Review.pdf https://joss.tcnj.edu/wp-content/uploads/sites/176/2012/04/2... > How reliable is the SAT? The College Board website reports that a student’s true score is within 30 points of his or her measured score (SEM = 30; College Board, FAQ, 2005). A test with a standard deviation of 100 and a SEM of 30 has an internal consistency coefficient of .91 (α = .91). In a study specifically investigating SAT I score change upon repeated testing, 1,120,563 students in the 1997 college- bound cohort who took the SAT I one to five times in their junior and senior years gained an average of 7 to 13 points on the verbal section and 8 to 16 points on the mathematics section (Nathan & Camara, 1998). Thus, a student who retested at the higher end of both ranges still would not breach the standard error of measurement, indicating high test-retest reliability. Additionally, in a study of the effects of commercial coaching on SAT scores, Powers and Rock (1999) found that the average increase in student scores after intensive coaching was only 30 points on the verbal section and 50 points on the mathematics section, demonstrating that SAT scores remain relatively stable even with intervening practice. With an SEM of 30, there is less than a 1% chance that a 100 point difference, between a 600 and a 500 on a section is due to random chance. The difference between a median score and and a TJ level score is 250. Validity, whether the test measures something we care about is a separate issue.

scarmig 6y ago

Metrics aren't the thing of interest in sports. Randomly looking at the justifications for some of the top high school athletics programs in California: "Honor, Glory and Love... the school is dedicated to the development of the whole person: spiritual, intellectual, physical, emotional and aesthetic" "students learn lifelong lessons in teamwork, discipline and leadership" "We encourage the development of team collaboration, communication, and characteristics that foster empathy, tolerance, personal growth and positive citizenship." I mean, it's all bullshit. Or at least, people aren't buying expensive houses in good school districts just so their kids can learn the value of teamwork. But if you take athletics' programs words at face value, winning and metrics aren't the primary objective. Specialized high schools are intended to build impressive application packets for college, which in turn is meant to get access to elite jobs and social opportunities. Indeed, that's one of the major reasons parents even try to get their kids into sports! One of the easiest ways into an elite university is to become an above average athlete in an obscure sport (this is the go-to approach for well-off but academically mediocre white people, actually). And so the admissions process for these high schools is geared toward facilitating that objective. TJ provided a pathway for academically gifted first generation Asian immigrants to achieve social mobility. And with this new process, many won't have access to TJ, to mostly be replaced with mediocre white kids (and a couple more token underrepresented minorities).