5 ms·

This is the problem with assigning weight to subjective, hand-wavey categories. It becomes an easy way for bias (conscious or unconscious) to creep in. The same

by gundmc 6y ago

This is the problem with assigning weight to subjective, hand-wavey categories. It becomes an easy way for bias (conscious or unconscious) to creep in. The same thing happens with "culture fit" evaluations at tech companies.

rurp 6y ago
> The same thing happens with "culture fit" evaluations at tech companies. I totally agree, this reminded me of culture fit interview questions as well. I think the evidence is very strong that almost any subjective evaluation is going to devolve down to the evaluator picking whatever confirms their subjective biases, then coming up with ex post facto rationalizations.
stickfigure 6y ago
The alternative isn't that rosy - if you judge everyone by (objective) standard tests, Goodhart's Law takes effect, and we breed a generation of highly optimized test takers. Maybe I'm just exposing my Western bias, but I have a child and I don't want his adolescence to be continuous preparation for one academic test after another. Mine wasn't. Combining standardized tests along with bias-ridden subjective measurements may just be the best solution among lots of imperfect solutions. We try to address egregious abuses when we find them, but "fair" will always be a goal, not a destination.
- jkhdigital 6y ago
  False dichotomy; there's another option, which is to use a lottery: http://revisionisthistory.com/episodes/44-the-powerball-revolution http://revisionisthistory.com/episodes/44-the-powerball-revo...
  - Nasrudith 6y ago
    Have we checked for extraterrestrial influnce? Seriously it is Larry Niven's straight out of the Known Space's series Puppeteer plot. In all seriousness it works well if you have a pool sufficiently quality homogenous but would tend towards the mean of the pool.
  - lopmotr 6y ago
    They're obviously not interested in what's better or they would expand to accept them all! But if they grow, they'll dilute their elite status.
- keith___talent 6y ago
  Certainly the test is not the be all end all, other factors are important. Grades, recommendation letters, activities, and an understanding of the students background. However it can help us evaluate who is able to perform in a higher learning environment and that they have the knowledge to study advanced degrees. I had 4 standardized tests in my educational career, 4th grade state test, 10th grade state test, ACT and GRE. Only 2 of those were non-repeatable. How many do students take these days?
- solveit 6y ago
  > if you judge everyone by (objective) standard tests, Goodhart's Law takes effect, and we breed a generation of highly optimized test takers. As opposed to a generation of people highly optimized for a fuzzier metric that nevertheless exists. I know a bunch of graduates from elite universities who all just happened to be really into debate/mock trials/mock UN, competed at the national/international level at obscure sports, founded a school newspaper (we had five new school newspapers every year because everyone wanted to be a founder, few of them last once the founder graduates), "founded" a "business", etc in high school[1]. They're all in consulting/banking/law/ML now. Mildly obfuscating the metric doesn't actually make Goodhart's law go away, it just means you have to be in the right crowd to game the metric. [1] I, on the other hand, actually liked maths and became one of your "highly optimized test takers". Funnily enough, all the elite American schools said no, while Oxford said yes. I am now a PhD student (in maths) at one of the elite American institutions that rejected me as a high school student.[2] If you have a scarce resource, and a system that distributes it, people are going to game the system. The trick is to have better tests so you can't study to the test without actually, you know, learning. Take a look at the SAT (subject) math section. That's the test that should be distinguishing between the student that goes to Harvard and the student that "merely" goes to Georgia Tech. Obviously it can do no such thing, it's way too easy. This is by design. The American system does not want to distinguish between the top 3% and the 0.5%, because then Harvard would have to take the poor genius over the solidly talented but unexceptional rich legacy kid whose dad is worth billions and will donate tens of millions. Now take a look at the MAT, the maths test that tells you if you'll be going to Oxford or Imperial. Or the STEP, the analogous test for Cambridge. [2] Disclaimer: I may be a little bitter to this day. In my defence, I teach the wannabe consultants now, and while some of them are legitimately very competent, boy does the system have false positives.
  - ABeeSea 6y ago
    Those exams are essentially tedious calculus problems. It’s the high school math equivalent of the National spelling bee.
    - lordnacho 6y ago
      Added wrinkle is that for Oxbridge you'll get interviewed as well. I recall being asked interesting things that they didn't cover in class, things that are more deep understanding than parroting syllabus materials.
- pps43 6y ago
  > if you judge everyone by (objective) standard tests That's why I don't like standard tests. When I was studying, entrance exams were oral. You were given three problems (chosen at random), had some time to solve them, then presented your solutions to the examiner. Examiner would then ask follow-up questions. There is no way to prepare for such test other than to learn the subject. No multiple-choice questions you can Kaplan.
- zarkov99 6y ago
  What do you want your child selected on for an elite school then? The bedrock reality is that most children, by definition, won't make it into an elite school. How do we pick?
  - stickfigure 6y ago
    This question is phrased oddly. I want my child to enter an educational environment where he will flourish. It's not obvious to me that having the brand of an 'elite' school is all that significant.
keith___talent 6y ago
We’re about to see an end to standardized tests for college admissions. This is paving the way for discrimination against good students who wish to attend elite universities. Currently elite universities ideas of diversity and “personality” are more important than upholding educational standards. It is simply wrong to evaluate and condemn a person based on race. To justify any kind of discrimination on the idea that you are helping one person, you are hurting another. This an equally problematic, it’s taking us back to times before equal rights and for people to be judged on their abilities. Hoping a brilliant student will apply to a university twice and deepfake their race in one of their admission interview videos.
- kenjackson 6y ago
  Why do good students have more of a right to attend elite colleges than other students? Why don’t we prefer hard working students or students that played the most sports? There’s nothing inherently special about being a good student.
  - pyronik19 6y ago
    Because it can't be elite its filled with people who are mediocre. It's in the name... elite. Everytime you just let someone in on some flimsy diversity grounds you debase degree.
    - collyw 6y ago
      Do you have any evidence of that
    - kenjackson 6y ago
      It’s not in the name. The presumption is that elite means other good students, and not the school itself. When I buy elite golf clubs, it means the clubs are elite not other people who buy it are. Now you can argue part of the reason to attend college is because of your classmates. And then I’d ask is the best experience a bunch of kids who are simply good at school?
- 6y ago
raxxorrax 6y ago
It is an excellent excuse to select for rich parents though.

by gundmc 6y ago

rurp 6y ago

> The same thing happens with "culture fit" evaluations at tech companies. I totally agree, this reminded me of culture fit interview questions as well. I think the evidence is very strong that almost any subjective evaluation is going to devolve down to the evaluator picking whatever confirms their subjective biases, then coming up with ex post facto rationalizations.

stickfigure 6y ago

The alternative isn't that rosy - if you judge everyone by (objective) standard tests, Goodhart's Law takes effect, and we breed a generation of highly optimized test takers. Maybe I'm just exposing my Western bias, but I have a child and I don't want his adolescence to be continuous preparation for one academic test after another. Mine wasn't. Combining standardized tests along with bias-ridden subjective measurements may just be the best solution among lots of imperfect solutions. We try to address egregious abuses when we find them, but "fair" will always be a goal, not a destination.

jkhdigital 6y ago
False dichotomy; there's another option, which is to use a lottery: http://revisionisthistory.com/episodes/44-the-powerball-revolution http://revisionisthistory.com/episodes/44-the-powerball-revo...
- Nasrudith 6y ago
  Have we checked for extraterrestrial influnce? Seriously it is Larry Niven's straight out of the Known Space's series Puppeteer plot. In all seriousness it works well if you have a pool sufficiently quality homogenous but would tend towards the mean of the pool.
- lopmotr 6y ago
  They're obviously not interested in what's better or they would expand to accept them all! But if they grow, they'll dilute their elite status.
keith___talent 6y ago
Certainly the test is not the be all end all, other factors are important. Grades, recommendation letters, activities, and an understanding of the students background. However it can help us evaluate who is able to perform in a higher learning environment and that they have the knowledge to study advanced degrees. I had 4 standardized tests in my educational career, 4th grade state test, 10th grade state test, ACT and GRE. Only 2 of those were non-repeatable. How many do students take these days?
solveit 6y ago
> if you judge everyone by (objective) standard tests, Goodhart's Law takes effect, and we breed a generation of highly optimized test takers. As opposed to a generation of people highly optimized for a fuzzier metric that nevertheless exists. I know a bunch of graduates from elite universities who all just happened to be really into debate/mock trials/mock UN, competed at the national/international level at obscure sports, founded a school newspaper (we had five new school newspapers every year because everyone wanted to be a founder, few of them last once the founder graduates), "founded" a "business", etc in high school[1]. They're all in consulting/banking/law/ML now. Mildly obfuscating the metric doesn't actually make Goodhart's law go away, it just means you have to be in the right crowd to game the metric. [1] I, on the other hand, actually liked maths and became one of your "highly optimized test takers". Funnily enough, all the elite American schools said no, while Oxford said yes. I am now a PhD student (in maths) at one of the elite American institutions that rejected me as a high school student.[2] If you have a scarce resource, and a system that distributes it, people are going to game the system. The trick is to have better tests so you can't study to the test without actually, you know, learning. Take a look at the SAT (subject) math section. That's the test that should be distinguishing between the student that goes to Harvard and the student that "merely" goes to Georgia Tech. Obviously it can do no such thing, it's way too easy. This is by design. The American system does not want to distinguish between the top 3% and the 0.5%, because then Harvard would have to take the poor genius over the solidly talented but unexceptional rich legacy kid whose dad is worth billions and will donate tens of millions. Now take a look at the MAT, the maths test that tells you if you'll be going to Oxford or Imperial. Or the STEP, the analogous test for Cambridge. [2] Disclaimer: I may be a little bitter to this day. In my defence, I teach the wannabe consultants now, and while some of them are legitimately very competent, boy does the system have false positives.
- ABeeSea 6y ago
  Those exams are essentially tedious calculus problems. It’s the high school math equivalent of the National spelling bee.
  - lordnacho 6y ago
    Added wrinkle is that for Oxbridge you'll get interviewed as well. I recall being asked interesting things that they didn't cover in class, things that are more deep understanding than parroting syllabus materials.
pps43 6y ago
> if you judge everyone by (objective) standard tests That's why I don't like standard tests. When I was studying, entrance exams were oral. You were given three problems (chosen at random), had some time to solve them, then presented your solutions to the examiner. Examiner would then ask follow-up questions. There is no way to prepare for such test other than to learn the subject. No multiple-choice questions you can Kaplan.
zarkov99 6y ago
What do you want your child selected on for an elite school then? The bedrock reality is that most children, by definition, won't make it into an elite school. How do we pick?
- stickfigure 6y ago
  This question is phrased oddly. I want my child to enter an educational environment where he will flourish. It's not obvious to me that having the brand of an 'elite' school is all that significant.

keith___talent 6y ago

We’re about to see an end to standardized tests for college admissions. This is paving the way for discrimination against good students who wish to attend elite universities. Currently elite universities ideas of diversity and “personality” are more important than upholding educational standards. It is simply wrong to evaluate and condemn a person based on race. To justify any kind of discrimination on the idea that you are helping one person, you are hurting another. This an equally problematic, it’s taking us back to times before equal rights and for people to be judged on their abilities. Hoping a brilliant student will apply to a university twice and deepfake their race in one of their admission interview videos.

kenjackson 6y ago
Why do good students have more of a right to attend elite colleges than other students? Why don’t we prefer hard working students or students that played the most sports? There’s nothing inherently special about being a good student.
- pyronik19 6y ago
  Because it can't be elite its filled with people who are mediocre. It's in the name... elite. Everytime you just let someone in on some flimsy diversity grounds you debase degree.
  - collyw 6y ago
    Do you have any evidence of that
  - kenjackson 6y ago
    It’s not in the name. The presumption is that elite means other good students, and not the school itself. When I buy elite golf clubs, it means the clubs are elite not other people who buy it are. Now you can argue part of the reason to attend college is because of your classmates. And then I’d ask is the best experience a bunch of kids who are simply good at school?
6y ago

raxxorrax 6y ago

It is an excellent excuse to select for rich parents though.