2 ms·
Interesting that the coding win-rate vs GPT-4o was only 10% higher. Very cool but clearly this model isn't as much of a slam dunk as the static benchmarks portr
by extr 2y ago
Interesting that the coding win-rate vs GPT-4o was only 10% higher. Very cool but clearly this model isn't as much of a slam dunk as the static benchmarks portray.
However, it does open up an interesting avenue for the future. Could you prompt-cache just the chain-of-thought reasoning bits?
- mewpmewp2 2y agoIt's hard to evaluate those win-rates, because if it's slower, people may have been giving easier problems, which both can solve and picked the faster one.