4 ms·
Not sure on the licensing but yes you can do that technically. Phi-1 and therefore phi-1.5 are partially trained on gpt3.5 generated synthetic textbooks.
by IanCal 3y ago
Not sure on the licensing but yes you can do that technically.
Phi-1 and therefore phi-1.5 are partially trained on gpt3.5 generated synthetic textbooks.
- saurik 3y agoThe premise here is specifically not to train it on generated output of the bigger model but to merely use the bigger model to better curate non-generated (and thereby untainted) inputs for the training set of the smaller model.