3 ms·
An actually open source LLM would be a game changer. We might need a new license that englobes model usage and training, something GPL-like whereby distributing
by omneity 4y ago
An actually open source LLM would be a game changer. We might need a new license that englobes model usage and training, something GPL-like whereby distributing a retrained model requires contributing data back or making it public, but not if you use it privately.
This will definitely accelerate progress in LLM research, productization and safety. Alpaca, vicuna, gpt4all and others are sporadic repesentations of this that could become a continuous improvement process were the LLM and its license truely open source.
An interesting possible side effect of a GPL-like license is that AIs become unlikely to be trained on private data, the usual moat that big tech wouldn't want/just can't make public if it were to use those GPL-like licensed models.
- jupp0r 4y agoAs with original GPL, this would be almost useless in a commercial context.
- e12e 4y agoThere are commercial devices that ship with a Linux kernel?
- ijustlovemath 4y agoI think they mean in terms of enforcement when there's a violation
- Bjartr 4y agoBasically every Android device for starters.
- sp332 4y agoUsing a Linux kernel doesn't mean you have to make your whole project GPL, unless your project is specifically kernel code.
- wongarsu 4y agoNeither would the proposed model license. Just like the kernel's GPL stops at the userspace boundary, the proposed license would only cover the model definition and weights.
- jupp0r 4y agoBut do they train the Linux kernel with their customers data?
- ipsum2 4y agoHuh? There's plenty of open source LLMs. Pythia, GPT-NeoX, GPT-J, GPT-2, BLOOM-176, are ones I can think of off the top of my head. Pythia is the best performing one IIRC.
- bradknowles 4y agoHmm. Would you be able to combine these LLMs? Or are they already supersets of each other?
- MacsHeadroom 4y agoPythia was trained on only 300B tokens and is pretty dumb compared to LLaMA. Pythia 13B is worse than LLaMA-7B and requires double the resources.
- Tepix 4y agoNot all use cases need GPT-4 level performance. I'd argue that even LLaMA-7B is quite limited. Also, new and improved models are being released all the time.
- lhl 4y agoI started keeping a list btw, there are about 20 completely open (Apache 2.0, BSD, MIT, CC-BY) 1B+ parameter foundational LLMs at the moment: https://docs.google.com/spreadsheets/d/1kT4or6b0Fedd-W_jMwYpb63e1ZR3aePczz3zlbJW-Y4/edit?usp=sharing https://docs.google.com/spreadsheets/d/1kT4or6b0Fedd-W_jMwYp...
- buzzscale 4y agoDolly 2.0 is fully open, Apache License and the tuning dataset is employee generated: https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm https://www.databricks.com/blog/2023/04/12/dolly-first-open-...
- gumballindie 4y agoThe solution is simple. We need an updated GPL license that states that the code cant be used in training ais unless the data model is also open source. A coordinated update of all major open source projects and the issue is sorted as it will force the ai folks to open source their models. Or else they’ll have to stick with generating funny cat pictures.
- jacooper 4y agoThe problem isnt the license, the problem is it being fair use or not. If its fair use the license is irrelevant.
- gumballindie 4y agoThat has an even easier fix: no content no problem.