3 ms·

Posted this elsewhere in the thread but I'll say something similar again: I love that here on HN, where from working in the industry I can sensibly assume the p

Posted this elsewhere in the thread but I'll say something similar again: I love that here on HN, where from working in the industry I can sensibly assume the posters are mostly white and Asian, no one questions whether the metrics are of any quality. Is SAT really a good measure of your talents as a human being? Are high school grades? Two things that vary based on your education before the moment of the test and the grades. The grades themselves being subjective evaluations from thousands of people with different standards. If the quotas were swapped and whites and Asians were getting the "bonus", how quickly would the arguments shift to the idea that the metrics aren't any good in the first place? You probably took the SAT, do you honestly think its' a great talent indicator or a mediocre one? How many courses are out there that promise to improve your score and how much money do they cost? Ask an SAT tutor and they'll tell you it's easy to take someone from a decent score to a good one. Is that a great metric to decide people's futures. Who has access to SAT prep courses? Wealthy people (and people who know the system well and budget to game it). Grades? You want to talk about grades? If the culture is overall discriminatory, how can we trust subjective evaluations from that culture? Not to mention if you approach the admissions officers at Ivies, particularly at Harvard, Princeton & Yale, they say that they easily could fill 3 freshman classes or more with the applicants they receive. That even most of the rejected applicants could succeed academically at their schools. So yes, I think it's okay to select for diversity. If the system is gameable why should we take its metrics seriously? And if we can't take the metrics seriously we shouldn't be upset about ignoring the metrics and making subjective choices.

Torgo 12y ago
I worked in standardized testing for ten years for SAT's competitor and am very familiar with the SAT as well, and phenomenal effort was made detect and correct for racial, gender and cultural bias. I cranked out crates of statistical analysis using SAS analyzing every demographic breakdown looking for statistical anomalies in answers so they could be excluded from scoring (and mainly preventing pretest items from ever being scored items in the first place.) If blacks (or whites, or women, or Eskimos) performed statistically significantly worse on a question, then it was thrown out of scoring. > Ask an SAT tutor and they'll tell you it's easy to take someone from a decent score to a good one. Yeah, they have a financial interest in promoting their snakeoil. They teach you how to not be a fuckup before you take the test (get good sleep, don't go drinking) and a few general facts about the test to minimize wasting time, they don't teach you how to "beat" the SAT/ACT, that hasn't been a thing since the 80's.
- ritchiea 12y ago
  I didn't say cultural bias was an issue, you're twisting my words. And the people I know who vouch for the effectiveness of test prep classes don't even teach them anymore, they taught them mostly as a summer jobs or jobs out of college before they found more of a career path job. They have no reason to vouch for the effectiveness of the classes, they are speaking about personal experience. And you honestly don't think the classes help at all? Seeing sample questions, taking sample tests, and learning strategies to handle questions you don't know. You might as well say structured practice doesn't help. And sure you could say "why don't the kids from less affluent families practice just as much?" but if your parent doesn't know how the game is played the sixteen year old kid still isn't as prepared as the kid who has affluent parents. And the scores aren't going to reflect ability as much as they reflect whether you come from a household that prepares you for standardized tests.
  - Torgo 12y ago
    The demographic information collected for the test is a proxy for culture, since we really don't have any idea. We just detected if individual test items were gotten wrong substantially more often for any individual demographic categories. In the end it really doesn't matter why people got it wrong, just that statistically it's a strong sign it's a bad question. Yes, the tests help. But they just help you reach your maximum potential. You're not hacking the test, you're hacking yourself. You can get basic prep courses for not much money. Yes some people cannot afford that, but that is a problem with secondary education that test prep even exists. There is no way you can make a test that doesn't discriminate against people that don't get enough sleep or show up hung over or the person didn't figure out the basics of taking standardized tests through primary and secondary education.

Torgo 12y ago

I worked in standardized testing for ten years for SAT's competitor and am very familiar with the SAT as well, and phenomenal effort was made detect and correct for racial, gender and cultural bias. I cranked out crates of statistical analysis using SAS analyzing every demographic breakdown looking for statistical anomalies in answers so they could be excluded from scoring (and mainly preventing pretest items from ever being scored items in the first place.) If blacks (or whites, or women, or Eskimos) performed statistically significantly worse on a question, then it was thrown out of scoring. > Ask an SAT tutor and they'll tell you it's easy to take someone from a decent score to a good one. Yeah, they have a financial interest in promoting their snakeoil. They teach you how to not be a fuckup before you take the test (get good sleep, don't go drinking) and a few general facts about the test to minimize wasting time, they don't teach you how to "beat" the SAT/ACT, that hasn't been a thing since the 80's.

ritchiea 12y ago
I didn't say cultural bias was an issue, you're twisting my words. And the people I know who vouch for the effectiveness of test prep classes don't even teach them anymore, they taught them mostly as a summer jobs or jobs out of college before they found more of a career path job. They have no reason to vouch for the effectiveness of the classes, they are speaking about personal experience. And you honestly don't think the classes help at all? Seeing sample questions, taking sample tests, and learning strategies to handle questions you don't know. You might as well say structured practice doesn't help. And sure you could say "why don't the kids from less affluent families practice just as much?" but if your parent doesn't know how the game is played the sixteen year old kid still isn't as prepared as the kid who has affluent parents. And the scores aren't going to reflect ability as much as they reflect whether you come from a household that prepares you for standardized tests.
- Torgo 12y ago
  The demographic information collected for the test is a proxy for culture, since we really don't have any idea. We just detected if individual test items were gotten wrong substantially more often for any individual demographic categories. In the end it really doesn't matter why people got it wrong, just that statistically it's a strong sign it's a bad question. Yes, the tests help. But they just help you reach your maximum potential. You're not hacking the test, you're hacking yourself. You can get basic prep courses for not much money. Yes some people cannot afford that, but that is a problem with secondary education that test prep even exists. There is no way you can make a test that doesn't discriminate against people that don't get enough sleep or show up hung over or the person didn't figure out the basics of taking standardized tests through primary and secondary education.