4 ms·
What a lot of people complain about with Llama is the fact that the weights are open but not the training data and training code. That feels like a red herring
by lolinder 2y ago
What a lot of people complain about with Llama is the fact that the weights are open but not the training data and training code. That feels like a red herring to me—code is data and data is code, and we shouldn't require someone to be developing entirely in the open in order for the output to be open source.
The weights are the "preferred form of the work for making modifications to it", to quote the GPL. The rest is just the infrastructure used to produce the work.
Where "open source" is misleading with Llama is that it's restricted to companies under a certain size and has restrictions for what you can and can't do with it. That kind of restriction undermines the freedoms promised by the phrase "open source", and it's concerning to me that people have gotten so fixating on weights vs data when there's a big gap in the freedoms offered on the weights.
- diggan 2y ago> The weights are the "preferred form of the work for making modifications to it", to quote the GPL. The rest is just the infrastructure used to produce the work. I disagree with this, if you want to actually have a fundamental impact on the model, the real work and innovation goes into how the training is done, and the architecture of the model. That's the "source", and what Meta et al is currently trying to protect and keep private. Guess why they're so adamant at keeping the training code secret? Besides that, I agree wholeheartedly with you :)