3 ms·
METR themselves say that any estimate >16 is highly suspect because there are too few tasks. I expect benchmarks like ProgramBench will replace METR this year.
by woeirua 5mo ago
METR themselves say that any estimate >16 is highly suspect because there are too few tasks.
I expect benchmarks like ProgramBench will replace METR this year.