2 ms·
Llama2 beat ChatGPT 3.5 with a 92.66% win rate to 89.37%, but lost to GPT-4 which got 95.28%. Still pretty amazing though!
by freedomben 3y ago
Llama2 beat ChatGPT 3.5 with a 92.66% win rate to 89.37%, but lost to GPT-4 which got 95.28%. Still pretty amazing though!
- cs702 3y agoNot really close, because performance is logarithmic in training compute. That is, each additional percentage point of performance requires exponentially greater investment in compute during pretraining. Llama 2 was pretrained on 2 trillion tokens -- a significant investment in compute, for sure, but still not enough to get close to GPT-4. And this is only one benchmark.