6 ms·

counter-example: Most systems that work with standardized tests, while they can systematically fail to capture certain types of talent, are fairly objective -

counter-example: Most systems that work with standardized tests, while they can systematically fail to capture certain types of talent, are fairly objective - "if you're in the top 10% on this test, then we take you". The format and subject matter of the test is known in advance, people are free to prepare for it in whatever way they want. Of course, being good at passing tests doesn't mean one will be good at other things, in much the same way that one can be good at Leetcode problems but not good at building and maintaining large scale software systems in a large corporate environment. Still, it remains a 'credible' example of 'how to do meritocracy right', with examples dating back to the Chinese civil service examinations during the Sui and Tang dynasty (from around 593 AD). https://en.wikipedia.org/wiki/Imperial_examination https://en.wikipedia.org/wiki/Imperial_examination

Timon3 1y ago
> Of course, being good at passing tests doesn't mean one will be good at other things, in much the same way that one can be good at Leetcode problems but not good at building and maintaining large scale software systems in a large corporate environment. Still, it remains a 'credible' example of 'how to do meritocracy right', [...] It's important not to forget Goodhart's law in this context: "When a measure becomes a target, it ceases to be a good measure". Unless the tested skill is wholly and directly applicable to the position you're testing for, the results will still be unfairly skewed towards those with more resources - they can afford more time to study for this specific test, and more importantly they have the resources to buy specialized learning material as well as tutoring. Of course that's not to say that merit doesn't factor into the results at all, but it does mean that even examples of 'how to do meritocracy right' show that merit is never the only thing that matters.
- AnthonyMouse 1y ago
  This is presented as an argument against objective metrics, but a) the alternative is subjective metrics, which is even worse, b) "disparate impact" is just another metric subject to Goodhart's law, and c) resources aren't the only thing that determines test scores. If you're poor but determined, you can't afford a high cost test prep course, but you can go to the library. The rich kid has their private tutor come to their house and then saves time that allowed them to be chauffeured to tennis lessons. The poor kid has to take the bus to the library and spend twice as long with the study books and then doesn't get any tennis lessons, but it's possible for someone to do that if they actually care about it. Whereas, how is a low-income white kid supposed to overcome a race quota where every slot for their race was already filled by nepotism?
  - Timon3 1y ago
    > This is presented as an argument against objective metrics No, it's not. It's presented as an argument that measuring objective metrics doesn't mean you're measuring merit. > If you're poor but determined, you can't afford a high cost test prep course, but you can go to the library. The library most likely doesn't have the same specialized learning materials that rich kids can afford, so this doesn't mean the poor kid has equal opportunity. > The rich kid has their private tutor come to their house and then saves time that allowed them to be chauffeured to tennis lessons. The poor kid has to take the bus to the library and spend twice as long with the study books and then doesn't get any tennis lessons, but it's possible for someone to do that if they actually care about it. You're presenting this like it's purely an issue of free time, which is obviously not the case. The poor kid possibly can't reach the library without their guardian, who may or may not have time to drive them there. The poor kid possibly doesn't have anyone to teach them using the learning material there, while the rich kid likely has either their guardians or even specialized tutors for this purpose. Your comment is a wonderful example of how people arguing for meritocracy can ignore reality - the bare minimum is supposed to be enough for the disadvantaged, even though there's a massive difference in effectiveness.
    - AnthonyMouse 1y ago
      > No, it's not. It's presented as an argument that measuring objective metrics doesn't mean you're measuring merit. The goal is to measure merit. Objective metrics are the nearest thing we have to achieving that goal. If there is a better metric, you use that instead. But if the best metric we have isn't perfect, that's no argument for doing something even worse. > The library most likely doesn't have the same specialized learning materials that rich kids can afford, so this doesn't mean the poor kid has equal opportunity. And yet this is still more of an opportunity than being locked out by race quotas. > The poor kid possibly can't reach the library without their guardian, who may or may not have time to drive them there. Libraries are generally in higher density areas with mass transit, and in the worst case you can walk there. Moreover, primary schools generally have libraries and then the kid is already there for school. > The poor kid possibly doesn't have anyone to teach them using the learning material there, while the rich kid likely has either their guardians or even specialized tutors for this purpose. Which is why it takes longer. But the point is that determination has an effect. It's something you can choose rather than something you can't control. Whereas telling people that it's not a level playing field so therefore they shouldn't even try is how you perpetuate the problem forever, if not actively make it worse.

Timon3 1y ago

> Of course, being good at passing tests doesn't mean one will be good at other things, in much the same way that one can be good at Leetcode problems but not good at building and maintaining large scale software systems in a large corporate environment. Still, it remains a 'credible' example of 'how to do meritocracy right', [...] It's important not to forget Goodhart's law in this context: "When a measure becomes a target, it ceases to be a good measure". Unless the tested skill is wholly and directly applicable to the position you're testing for, the results will still be unfairly skewed towards those with more resources - they can afford more time to study for this specific test, and more importantly they have the resources to buy specialized learning material as well as tutoring. Of course that's not to say that merit doesn't factor into the results at all, but it does mean that even examples of 'how to do meritocracy right' show that merit is never the only thing that matters.

AnthonyMouse 1y ago
This is presented as an argument against objective metrics, but a) the alternative is subjective metrics, which is even worse, b) "disparate impact" is just another metric subject to Goodhart's law, and c) resources aren't the only thing that determines test scores. If you're poor but determined, you can't afford a high cost test prep course, but you can go to the library. The rich kid has their private tutor come to their house and then saves time that allowed them to be chauffeured to tennis lessons. The poor kid has to take the bus to the library and spend twice as long with the study books and then doesn't get any tennis lessons, but it's possible for someone to do that if they actually care about it. Whereas, how is a low-income white kid supposed to overcome a race quota where every slot for their race was already filled by nepotism?
- Timon3 1y ago
  > This is presented as an argument against objective metrics No, it's not. It's presented as an argument that measuring objective metrics doesn't mean you're measuring merit. > If you're poor but determined, you can't afford a high cost test prep course, but you can go to the library. The library most likely doesn't have the same specialized learning materials that rich kids can afford, so this doesn't mean the poor kid has equal opportunity. > The rich kid has their private tutor come to their house and then saves time that allowed them to be chauffeured to tennis lessons. The poor kid has to take the bus to the library and spend twice as long with the study books and then doesn't get any tennis lessons, but it's possible for someone to do that if they actually care about it. You're presenting this like it's purely an issue of free time, which is obviously not the case. The poor kid possibly can't reach the library without their guardian, who may or may not have time to drive them there. The poor kid possibly doesn't have anyone to teach them using the learning material there, while the rich kid likely has either their guardians or even specialized tutors for this purpose. Your comment is a wonderful example of how people arguing for meritocracy can ignore reality - the bare minimum is supposed to be enough for the disadvantaged, even though there's a massive difference in effectiveness.
  - AnthonyMouse 1y ago
    > No, it's not. It's presented as an argument that measuring objective metrics doesn't mean you're measuring merit. The goal is to measure merit. Objective metrics are the nearest thing we have to achieving that goal. If there is a better metric, you use that instead. But if the best metric we have isn't perfect, that's no argument for doing something even worse. > The library most likely doesn't have the same specialized learning materials that rich kids can afford, so this doesn't mean the poor kid has equal opportunity. And yet this is still more of an opportunity than being locked out by race quotas. > The poor kid possibly can't reach the library without their guardian, who may or may not have time to drive them there. Libraries are generally in higher density areas with mass transit, and in the worst case you can walk there. Moreover, primary schools generally have libraries and then the kid is already there for school. > The poor kid possibly doesn't have anyone to teach them using the learning material there, while the rich kid likely has either their guardians or even specialized tutors for this purpose. Which is why it takes longer. But the point is that determination has an effect. It's something you can choose rather than something you can't control. Whereas telling people that it's not a level playing field so therefore they shouldn't even try is how you perpetuate the problem forever, if not actively make it worse.