3 ms·
Just read through some of the code "benchmarks", and I see why: https://reinvently.co.uk/tools/ed-o-meter/tests/ https://reinvently.co.uk/tools/ed-o-meter/tests
by sambusa_123 1mo ago
Just read through some of the code "benchmarks", and I see why: https://reinvently.co.uk/tools/ed-o-meter/tests/ https://reinvently.co.uk/tools/ed-o-meter/tests/
Most are extremely trivial tasks. I would be surprised if a model from 2 years ago failed these...
- nylonstrung 1mo agoI think most of them would have actually failed, it's only recently that models were any good at using tool calls and harnesses after they started post-training for that