3 ms·
Okay but...closed source harnesses change multiple times per week, sometimes per day. And they also tune the inference of the models behind it. You're using d
by epolanski 2mo ago
Okay but...closed source harnesses change multiple times per week, sometimes per day.
And they also tune the inference of the models behind it.
You're using different tooling every day. Hard to benchmark.
- edgyquant 2mo agoA benchmark would be a series of prompts + tests that check their results you theoretically run every day