I dont think that's the case, tinyllama has been trained on much more data already (2.5T for 1.1B params, and they aim for 3T) and while it didn't show signs of

I dont think that's the case, tinyllama has been trained on much more data already (2.5T for 1.1B params, and they aim for 3T) and while it didn't show signs of convergence, it also has much worse results than the Phi models. The main takeaway of Microsoft models seems to be the one they got in Textbooks are all you need, that is: data quality gets you very far, even if you use artificial data.