4 ms·
HOW is it compare to phi-2?
by transformi 3y ago
HOW is it compare to phi-2?
- sp332 3y agoHigher on MMLU (62.5 vs 56.7 for phi-2) and GSM8k (61.5 vs 61.1). https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/ https://www.microsoft.com/en-us/research/blog/phi-2-the-surp... The phi-2 numbers are for 5-shot MMLU and 8-shot GSM8k. The blog post doesn't get that specific for Qwen, but it's very likely they tested the same way.