2 ms·
Okay but...closed source harnesses change multiple times per week, sometimes per day. And they also tune the inference of the models behind it. You're using d
by epolanski 9d ago
Okay but...closed source harnesses change multiple times per week, sometimes per day.
And they also tune the inference of the models behind it.
You're using different tooling every day. Hard to benchmark.
- edgyquant 9d agoA benchmark would be a series of prompts + tests that check their results you theoretically run every day