It's an apples and oranges comparison due to different training sets (as well as model sizes), but it seems generally their tests are indicating Phi-2 performin

It's an apples and oranges comparison due to different training sets (as well as model sizes), but it seems generally their tests are indicating Phi-2 performing (in their specialized domains) at level of a more general model 5-10x it's size, not 100x. As far as training data, it seems it's more the quality/focus of the data than the quantity. They refer to "textbook quality data", unlike the LLMs like GPT-3 that are trained on web scrapes etc.