If you read the LLM papers they do that on the standard batch of evaluations: MMLU, hellaswag, winogrande, etc. Every once in a while they have to make up a ne

If you read the LLM papers they do that on the standard batch of evaluations: MMLU, hellaswag, winogrande, etc. Every once in a while they have to make up a new evaluation suite since SOTA is close enough to human-level performance to not be impressive any more. Just read the actual papers and the progress is readily apparent.