5 ms·

If you're talking about the SAT and ACT, I would challenge the idea that they can accurately distinguish between the good and outstanding. At most they provide

If you're talking about the SAT and ACT, I would challenge the idea that they can accurately distinguish between the good and outstanding. At most they provide a rough indication. I know this because I got a much higher score than I deserved to on the ACT because you can read the test pretty easily if you put yourself in the shoes of the people making it (I got a 31 overall and a near-perfect 35 on the science portion without studing)

clankyclanker 6y ago
Further, the only thing that the SAT claims to be able to predict is a student’s first semester performance, and wasn’t found to be a better predictor than other factors on other measures. Of course I can’t find a reference for that right now, but it certainly doesn’t predict college graduation rates best. https://www.forbes.com/sites/prestoncooper2/2018/06/11/what-predicts-college-completion-high-school-gpa-beats-sat-score https://www.forbes.com/sites/prestoncooper2/2018/06/11/what-...
- travisoneill1 6y ago
  Useless. They topped out the SAT buckets at 1100.
jerf 6y ago
No, there are other ways to separate the pack. A perfect score on those tests isn't rare enough, plus you're still going to see a lot of exceptional candidates who got a few wrong. They aren't being used, so they aren't the existing standardized tests, which are being used. But it's not that hard to simply extend the difficulty gradient higher. Stick some math olympiad-type problems on there, for instance.
MiroF 6y ago
> I got a much higher score than I deserved to on the ACT because you can read the test pretty easily if you put yourself in the shoes of the people making it I don't understand why that is some secretly accessible knowledge that is only available to some. FWIW, the ACT always seemed more game-y to me what with its hard time crunches and stuff.
solveit 6y ago
I'm specifically not talking about the SAT and ACT since they're widely used by elite universities. Those tests top out way too early and can't reliably distinguish between the top 1% and the top 0.1% (The fraction of perfect scorers is much smaller than 1% but the standard error of measurement is way too large). The exams I have in mind are those like the science olympiads and the Cambridge STEP. STEP in particular is basically used for this purpose (olympiads are a slightly different beast).
- saagarjha 6y ago
  In the US the appropriate exams would likely be the qualifiers for the International Olympiads.
- TuringNYC 6y ago
  It those were used for admissions, I wouldn’t be surprised if there was a huge market for answers (from somewhere in the production chain), stand-in test takers, etc.
- godelski 6y ago
  Are you trying to get students that can pass specific kinds of tests or students that will be successful? The two aren't strongly correlated. I'd argue successful builds more prestige for the university. And let's be real, everyone knows these tests are purposefully tricky. In the real world people don't give you multiple choices and try to trick you into picking the wrong answer and then laugh at you when you picked the one try tried to get you to pick (politics aside). In classes, unless your professor is an asshole, they aren't trying to trick you. Success depends on a lot of factors well beyond what these tests measure. This is why SAT and ACT don't measure success or even academic success (as in they don't strongly correlate [0][1] with success. This source should decent correlation with first year grades but not second). I'm not sure why one would even think an academic test could determine success. One of the biggest lies many of us have been told is that if you're just smart enough you'll be successful. Sorry, but that's not how it works. [0] https://edpolicyinca.org/publications/predicting-college-success-how-do-different-high-school-assessments-measure-2019 https://edpolicyinca.org/publications/predicting-college-suc... [1] https://www.latimes.com/california/story/2019-12-22/grades-vs-sat-scores-which-is-a-better-predictor-of-college-success https://www.latimes.com/california/story/2019-12-22/grades-v...

clankyclanker 6y ago

Further, the only thing that the SAT claims to be able to predict is a student’s first semester performance, and wasn’t found to be a better predictor than other factors on other measures. Of course I can’t find a reference for that right now, but it certainly doesn’t predict college graduation rates best. https://www.forbes.com/sites/prestoncooper2/2018/06/11/what-predicts-college-completion-high-school-gpa-beats-sat-score https://www.forbes.com/sites/prestoncooper2/2018/06/11/what-...

travisoneill1 6y ago
Useless. They topped out the SAT buckets at 1100.

jerf 6y ago

No, there are other ways to separate the pack. A perfect score on those tests isn't rare enough, plus you're still going to see a lot of exceptional candidates who got a few wrong. They aren't being used, so they aren't the existing standardized tests, which are being used. But it's not that hard to simply extend the difficulty gradient higher. Stick some math olympiad-type problems on there, for instance.

MiroF 6y ago

> I got a much higher score than I deserved to on the ACT because you can read the test pretty easily if you put yourself in the shoes of the people making it I don't understand why that is some secretly accessible knowledge that is only available to some. FWIW, the ACT always seemed more game-y to me what with its hard time crunches and stuff.

solveit 6y ago

I'm specifically not talking about the SAT and ACT since they're widely used by elite universities. Those tests top out way too early and can't reliably distinguish between the top 1% and the top 0.1% (The fraction of perfect scorers is much smaller than 1% but the standard error of measurement is way too large). The exams I have in mind are those like the science olympiads and the Cambridge STEP. STEP in particular is basically used for this purpose (olympiads are a slightly different beast).

saagarjha 6y ago
In the US the appropriate exams would likely be the qualifiers for the International Olympiads.
TuringNYC 6y ago
It those were used for admissions, I wouldn’t be surprised if there was a huge market for answers (from somewhere in the production chain), stand-in test takers, etc.
godelski 6y ago
Are you trying to get students that can pass specific kinds of tests or students that will be successful? The two aren't strongly correlated. I'd argue successful builds more prestige for the university. And let's be real, everyone knows these tests are purposefully tricky. In the real world people don't give you multiple choices and try to trick you into picking the wrong answer and then laugh at you when you picked the one try tried to get you to pick (politics aside). In classes, unless your professor is an asshole, they aren't trying to trick you. Success depends on a lot of factors well beyond what these tests measure. This is why SAT and ACT don't measure success or even academic success (as in they don't strongly correlate [0][1] with success. This source should decent correlation with first year grades but not second). I'm not sure why one would even think an academic test could determine success. One of the biggest lies many of us have been told is that if you're just smart enough you'll be successful. Sorry, but that's not how it works. [0] https://edpolicyinca.org/publications/predicting-college-success-how-do-different-high-school-assessments-measure-2019 https://edpolicyinca.org/publications/predicting-college-suc... [1] https://www.latimes.com/california/story/2019-12-22/grades-vs-sat-scores-which-is-a-better-predictor-of-college-success https://www.latimes.com/california/story/2019-12-22/grades-v...