Looking at the leaderboard shows a clearer picture: https://tatsu-lab.github.io/alpaca_eval/ https://tatsu-lab.github.io/alpaca_eval/ - GPT-4-Turbo: 50.00% -

Looking at the leaderboard shows a clearer picture: https://tatsu-lab.github.io/alpaca_eval/ https://tatsu-lab.github.io/alpaca_eval/ - GPT-4-Turbo: 50.00% - Snorkel (current 2nd, Mistral 7B fine-tune): 34.86% - Yi 34B Chat (current 6th): 29.66% - GPT-4: 23.58% Thoughts: - Just saying that it came 2nd is quite misleading, the difference in score is significant. - Not sure what's up with this benchmark, I've never seen GPT-4-Turbo vs GPT-4 performing so differently. - The Snorkel model is impressive with just 7B parameters. The Yi authors claim that their success is based on good training data cleaning. This seems to be key at least for this benchmark. Snorkel has also always been all about that, using programmatic methods to generate lots of quality training data.