3 ms·
If it was possible: 1) Why wasn't OpenAI doing it themselves? 2) This means we've reached technological singularity if AI models can improve themselves (as in
by SJC_Hacker 2y ago
If it was possible:
1) Why wasn't OpenAI doing it themselves?
2) This means we've reached technological singularity if AI models can improve themselves (as in getting a smarter model, not just compressing existing ones like Deepseek)
- sgillen 2y agoCompressing existing models is exactly what we are talking about here.
- SJC_Hacker 2y agoThen why wasn't OpenAI doing that themselves ?
- nickthegreek 2y agoWho says they arent?
- throwup238 2y agoIt’s not a singularity because the synthetic data generated by the previous frontier model isn’t usually fed directly into the training for the next frontier model - a “discriminator” is applied to select only the highest quality responses. That discriminator could be a field expert or mechanical turk or another model trained to select for higher quality responses (i.e. trained on a dataset of books rather than internet content). As far as I know, OpenAI has been doing this, using both experts and Kenyan workers as well as their own discriminator models. Unfiltered synthetic data is generally used more for distilled models and fine tunes for a specific use case.