4 ms·
the equivalent here would be a child learning from a textbook he has written himself. not sure how effective that would be, if it was his only source of learni
by nick486 2y ago
the equivalent here would be a child learning from a textbook he has written himself.
not sure how effective that would be, if it was his only source of learning.
- throwup238 2y agoWell that’s what the TFA is about. If you indiscriminately ingest synthetic data into training - the child learning from their own textbook - the model collapses. The SOTA is to use a discriminator (often another LLM or ML algo) to select the best output before feeding it into the training data. That’s what OpenAI, Anthropic, et al have been doing. One of them just published a paper about it a few weeks ago.