To quote Goodhart's Law: When a measure becomes a target, it ceases to be a good measure. Or, as in the case of LLMs and benchmarks: When a benchmark becomes a

To quote Goodhart's Law: When a measure becomes a target, it ceases to be a good measure. Or, as in the case of LLMs and benchmarks: When a benchmark becomes a target, it ceases to be a good benchmark.