3 ms·
Yeah, if the benchmark is actually predictive of the tasks you have then it is trivial to conclude that the cheapest-per-benchmark-task model will be the cheape
by yreg 3mo ago
Yeah, if the benchmark is actually predictive of the tasks you have then it is trivial to conclude that the cheapest-per-benchmark-task model will be the cheapest one for your tasks…
- janalsncm 3mo agoIt might vary between tasks though. A model that’s great at abstract reasoning might be great at writing math proofs but struggle to write software in <insert language>.