4 ms·
Right and LLMs will not be able to generate their own high quality training data. There are no perpetual motion machines.
by grajaganDev 2y ago
Right and LLMs will not be able to generate their own high quality training data.
There are no perpetual motion machines.
- throw310822 2y ago> LLMs will not be able to generate their own high quality training data. Humans certainly did. We did not inherit our physics and poetry books from some aliens.
- grajaganDev 2y agoHumans and LLMs are different things. LLMs can not reason - many people seen to believe that they can.
- gatlin 2y agoI can't prove that we did but I don't know that we /didn't/.
- MRtecno98 2y agoLLMs are not humans, nowhere near.
- mike_hearn 2y agoThey already do. All the current leading edge models are heavily trained on synthetic data. It's called textbook learning.
- sebastiennight 2y agoHave you read about this specific model we're talking about? My understanding is that the whole point of R1 is that it was surprisingly effective to train on synthetic data AND to reinforce on the output rather than the whole chain of thought. Which does not require so much human-curated data and is a big part of where the efficiency gain came from.