3 ms·

I feel like this is one of those cases where the differences between open source and free software become pretty important.

jillesvangurp 3y ago
Without getting too hung up on the semantics, what matters here is a free exchange of ideas, algorithms, and indeed some code. All this has massively accelerated progress in this space in the last two decades. Instead of sitting on their results, jealously guarding their secret sauce, etc. companies are sharing a lot of detail and are verifying each other's results by replicating them and improving on them. So OpenAI did chat gpt, Meta did llama, Google is doing Bard. Llama sort of got unleashed on the open source community and people are now starting to use that, improve it, benchmark against it with their own models, etc.
- demonshreder 3y ago
  In this case semantics do matter so to keep the improvements in the Software Commons as Commons. Weak / Permissive licenses will ensure that all the voluntarily contributed labour would be enclosed and profitable only to those who can hire an army of engineers. Strong licenses will ensure your call for a free exchange.
  - jillesvangurp 3y ago
    Depends. Overly open licenses can also motivate prevent people to not share things. Case in point here is some of the openai stuff where they shared enough so people can replicate their results but without giving away everything. People have been complaining that that's not open enough. But it's still better than them not sharing anything at all. That model certainly seems to be working well for them. Compare that with e.g. companies in China or elsewhere that are also working on AI that are perhaps sharing a bit less. That's a bit of a black hole in terms of information going in but not coming out. I think the more open people can get the better. But even some weaker licenses are better than nothing. Things are open enough currently for people to independently replicate each other's results. I think that's the important bit.
raxxorraxor 3y ago
Open source is always important. You won't have a good developer pipeline without it. Other engineering disciplines should strive for something similar. Doesn't mean there cannot be closed source software of course. At least for image synthesis, I believe the open models just plainly produce far better results by now since training is crowdsourced.
DebtDeflation 3y ago
In the case of LLMs, I really wish we would stop referring to models where the source code and/or weights were leaked as "open source". Likewise, for those where they were deliberately released but the license restricts them to "non-commercial use" or allows commercial use so long as the use "does not compete" with the company that released them.
- hospitalJail 3y ago
  Yep, first I got nailed with aGPL trying to make embedded products. But hey, I can work around that.... Then OpenAI and GPT3 was a wakeup call that releasing code to view is not the 'end all'. Then Facebook and llama, was a final straw of annoyance. MIT/Apache/GPL or bust. If you use those, you are a person of high class.
- cookieswumchorr 3y ago
  for the leaked part yes. But the license can be whatever, even proprietary, if the source is disclosed it is open-source
JohnFen 3y ago
I honestly feel the opposite. I mean, the more OSS the better, but I don't think that LLMs being open source will do much to mitigate what I see as being the largest risks of LLMs.

jillesvangurp 3y ago

Without getting too hung up on the semantics, what matters here is a free exchange of ideas, algorithms, and indeed some code. All this has massively accelerated progress in this space in the last two decades. Instead of sitting on their results, jealously guarding their secret sauce, etc. companies are sharing a lot of detail and are verifying each other's results by replicating them and improving on them. So OpenAI did chat gpt, Meta did llama, Google is doing Bard. Llama sort of got unleashed on the open source community and people are now starting to use that, improve it, benchmark against it with their own models, etc.

demonshreder 3y ago
In this case semantics do matter so to keep the improvements in the Software Commons as Commons. Weak / Permissive licenses will ensure that all the voluntarily contributed labour would be enclosed and profitable only to those who can hire an army of engineers. Strong licenses will ensure your call for a free exchange.
- jillesvangurp 3y ago
  Depends. Overly open licenses can also motivate prevent people to not share things. Case in point here is some of the openai stuff where they shared enough so people can replicate their results but without giving away everything. People have been complaining that that's not open enough. But it's still better than them not sharing anything at all. That model certainly seems to be working well for them. Compare that with e.g. companies in China or elsewhere that are also working on AI that are perhaps sharing a bit less. That's a bit of a black hole in terms of information going in but not coming out. I think the more open people can get the better. But even some weaker licenses are better than nothing. Things are open enough currently for people to independently replicate each other's results. I think that's the important bit.

raxxorraxor 3y ago

Open source is always important. You won't have a good developer pipeline without it. Other engineering disciplines should strive for something similar. Doesn't mean there cannot be closed source software of course. At least for image synthesis, I believe the open models just plainly produce far better results by now since training is crowdsourced.

DebtDeflation 3y ago

In the case of LLMs, I really wish we would stop referring to models where the source code and/or weights were leaked as "open source". Likewise, for those where they were deliberately released but the license restricts them to "non-commercial use" or allows commercial use so long as the use "does not compete" with the company that released them.

hospitalJail 3y ago
Yep, first I got nailed with aGPL trying to make embedded products. But hey, I can work around that.... Then OpenAI and GPT3 was a wakeup call that releasing code to view is not the 'end all'. Then Facebook and llama, was a final straw of annoyance. MIT/Apache/GPL or bust. If you use those, you are a person of high class.
cookieswumchorr 3y ago
for the leaked part yes. But the license can be whatever, even proprietary, if the source is disclosed it is open-source

JohnFen 3y ago

I honestly feel the opposite. I mean, the more OSS the better, but I don't think that LLMs being open source will do much to mitigate what I see as being the largest risks of LLMs.