3 ms·
MMLU is garbage. A lot of incorrect answers there.
by rafaelero 3y ago
MMLU is garbage. A lot of incorrect answers there.
- bugglebeetle 3y agoAnd yet it’s still a good indicator of general performance. Any model that scores under GPT-4 on that benchmark, but above it in other, tends to be worse overall.