3 ms·
We really do need better benchmarks and for models too - Most people use a harness because of its subscription (most companies pay Anthropic) - All model bench
by jiaosdjf 15d ago
We really do need better benchmarks and for models too
- Most people use a harness because of its subscription (most companies pay Anthropic)
- All model benchmarks are biased and gamed, harness benchmarks are too few to matter
- Everyone is just guessing, acting on sample sizes of 1 and trust me bro vibes