3 ms·
You could also imagine a battle over something that started with LLaMa, but fine-tuned it using a non-spare method (i.e. changed every single parameter) so it w
by A1kmm 4y ago
You could also imagine a battle over something that started with LLaMa, but fine-tuned it using a non-spare method (i.e. changed every single parameter) so it was very marginally altered in its behaviour.
Even if the base model is copyrightable (possibly a big if), there is a valid question of whether a new model which essentially optimised for something else, but used the base model as a computational shortcut to make it far cheaper to solve an optimisation problem, is still protected by the copyright holder of the base model.
Most of the barrier to creating large language models is the computational cost of training, not coming up with the training set data, so if fine-tuning gets around the copyright issues and allows for better FLOSS-licenced fine-tuned models, that would probably be a good thing (although maybe it will decrease the willingness of companies doing training to release models at all).