4 ms·
> What I hear is that training AI on AI outputs is fruitless. To me, thats as good as test as any: when you can use at scale an AI product to inform the trainin
by causalmodels 3y ago
> What I hear is that training AI on AI outputs is fruitless. To me, thats as good as test as any: when you can use at scale an AI product to inform the training of an unrelated AI system I will be impressed.
The phi-1 team did this a few weeks ago.
- ggm 3y agoI'm impressed. Noting this: The results exceeded the researchers' expectations. The team attributes them directly to data quality, as the title of the paper suggests: "Textbooks is all you need," in reference to Google's research on the Transformer breakthrough ("Attention is all you need"). If I have the right context, they selected HIGHLY QUALIFIED inputs. Which probably meant their training set was good as a seed to another system because it had been pre-groomed. But, it's still impressive.