3 ms·
Seems like a classic example of Goodheart’s Law. > When a measure becomes a target, it ceases to be a good measure. It is highly likely effort is being put sp
by trescenzi 2y ago
Seems like a classic example of Goodheart’s Law.
> When a measure becomes a target, it ceases to be a good measure.
It is highly likely effort is being put specifically into meeting benchmarks which in turn means the models are good at meeting benchmarks but not necessarily at what the benchmarks are trying to measure.
Of course ideally they are well designed enough that trying to meet them causes the model to be good at/have the properties they are benchmarking but there’s no guarantee that’ll always be the case.
- brookst 2y agoAgreed, but this is hardly limited to AI benchmarks. All benchmarks suffer from this. It’s almost cute the researchers have discovered that AI is not some wildly different thing.
- voxl 2y agoIt might behoove the reader to remind themselves that goodhearts law is a conjecture, not a law. Similar to Poe's Law, it's an observation about something, but that doesn't make it universal, or some kind of physical, social, or economic law. In other words, you don't get to assume that the tests aren't measuring the right thing, even if AIs are being designed to do well on the tests. That is a claim that demands evidence.