2 ms·
Fundamentally untrue, and disheartening that it's the top comment. You can't use a model's output to train another model, it leads to complete gibberish (terme
by concinds 3y ago
Fundamentally untrue, and disheartening that it's the top comment.
You can't use a model's output to train another model, it leads to complete gibberish (termed "model collapse"). https://arxiv.org/abs/2305.17493v2 https://arxiv.org/abs/2305.17493v2
And the Llama 2 license allows users to train derivative models, which is what people really care about. https://github.com/facebookresearch/llama/blob/main/LICENSE https://github.com/facebookresearch/llama/blob/main/LICENSE
- rgoldste 3y agoThe truth is between these two. You can use a model’s output to train another model, but it has drawbacks, including model collapse.
- int_19h 3y agoSome of the best LLaMA finetunes today are trained on GPT-4 output. Yes, you cannot do this kind of thing indefinitely and expect endless improvements from "endless training set". But that's a very different problem.