5 ms·
At the levels we’re talking about, the difference between “perfect score” and “still excellent but imperfect score” can be down to one or two errors on the test
by matthewdgreen 4y ago
At the levels we’re talking about, the difference between “perfect score” and “still excellent but imperfect score” can be down to one or two errors on the test, basically noise. Given the number of applicants and the willingness to drill testing skills, going by scores would eliminate anyone who makes a minor error. The ability to perform perfectly in high-stress situations is a useful (and learnable) skill - like the ability to win quiz games, or perform classical music at concert level - but it is not the same thing as academic potential.
(As an aside: as an advisor of PhD students in computer science, I sometimes have to train students out of these habits. Research doesn’t happen if you’re used to getting the shallow answer more quickly, or if you’re afraid to be wrong.)
- janalsncm 4y agoYes, and that’s one of the reasons why tests are imperfect. That doesn’t mean that introducing any additional variables is warranted, especially given their associated noise. GPA is garbage, for example, which is why I ignore it on resumes. If the test has unacceptably high variance as you mentioned, the solution is to make the test longer not discount it.
- matthewdgreen 4y agoI see no evidence that making tests longer will optimize for the variables that matter in academic performance (which are not 1-dimensional anyway) as opposed to simply optimizing for people who have poured resources into test preparation. You seem to be coming from the perspective that testing must be the best metric, and then trying to develop a methodology based on that assumption. Purely as a thought exercise: it might be helpful to start from the assumption that testing isn’t an option, and think about what you might do instead.