This is mistaken. You use GPT-4 to generate new data using other data sources, for example text books. Asking GPT-4 to create 50 new conversations from a chapt

This is mistaken. You use GPT-4 to generate new data using other data sources, for example text books. Asking GPT-4 to create 50 new conversations from a chapter of a textbook creates higher quality data than most of “the pile” and can extend beyond what GPT-4 has in its existing dataset. This is exactly what MS has done with Phi.