5 ms·

Yann Lecun is actively corrupting the term "open source" in relation to AI. If he really is pro open source, he should loudly acknowledge that all the recent mo

by andy99 3y ago

Yann Lecun is actively corrupting the term "open source" in relation to AI. If he really is pro open source, he should loudly acknowledge that all the recent models Meta has released do not qualify and push internally for them to go back to Apache 2.0 licensing and stop with their silly "you can only use it for what we allow" proprietary licenses. For clarity, llama etc are not open source because of the conditions on the license under which the weights are released. It may not be possible to copyright weights or to apply software licenses to them, it hasn't been decided by the courts. But everyone acts like it's possible, as evidenced by the licenses they use. So under that assumption, Meta is not using an open source license with the weights. Edit: for greater clarity, the open source definition has the following clause The license must not restrict anyone from making use of the program in a specific field of endeavor. For example, it may not restrict the program from being used in a business, or from being used for genetic research. Similar things appear in the debian free software guidelines, fsf free software definition, etc. The Meta Llama license lists restrictions on field of endeavour. Ergo, not open source. There are other clauses (the one about the number of users) which would also disqualify it, but the use restrictions are the most egregious.

smcleod 3y ago
The term has been polluted for a long time before AI became well known. Just like "DevOps", "Platform Engineering", "Agile" etc... and like those terms it's not a specific thing (e.g. a specific license like "Apache 2.0", "MIT" etc...) but more a general term without qualification of scope. Do I like it? No. But not liking how people are going to continue using a term shouldn't detract from the importance of the intent. At the end of the day - having LLM tech that has an open source _code_ base is what many people are talking about, the "data" that forms the model isn't always open source and while it would be nice if that was the case - is somewhat similar to open source software vs the data you run in it.
menzoic 3y ago
LLAMA2 is very much open source. You can use it for commercial purposes. The license only restricts companies that have more than 700M monthly active users from using it.
- yumraj 3y ago
  Releasing the trained model does not make it open source. Open source means something else. Calling released trained models open source is akin to releasing the binaries of a software for free, without releasing the source code, and calling it open source.
  - smcleod 3y ago
    It's more like releasing the code as open source (under whatever license is specified) but to make the code useful you need data and that data might be free but not open sourced under any license. If it was trained on the entire internet for example - you can't just bundled up every piece of data on the internet into a zip file to include with the code to interpret it.
    - menzoic 3y ago
      >to make the code useful you need data and that data might be free but not open sourced under any license To use something like Meta’s open sourced LLAMA2 model you don’t need the data. The model is self contained. It’s a compressed lossy form of all the data it was trained on. The weights allow you to continue its training with new data of your choosing.
  - menzoic 3y ago
    > Releasing the trained model does not make it open source. This is a straw man argument as Meta has released the model weights and not just the trained model. When it comes to models the weights are the source.
    - swatcoder 3y ago
      It may be a difference in terminology, but it's definitely no straw man. There's a philosophy of open source based on recipients being able to make any small and large modifications to the source material as they see fit. Just being able to build new projects with something or read what it does not qualify in that context -- it requires the freedom to rework the thing itself to suit your own needs and ends. In the case of these models, that philosophy would have the source training data and the training algorithms themselves available, editable, verifiable, and repeatable. Ignoring the financial challenges, a truly open source model would let you remove, add, edit, or reclassify a sample in the original training data and then build new weights of their won. The released weights are not that kind of open source, and that's what these arguments are generally making the a case for. There are still plenty of grounds from which to critique that argument if you wanted to, but there's no straw man involved in it.
evrydayhustling 3y ago
Cutting edge AI is a world-changing technology that is at immediate risk of being monopolized by 1-2 organizations pushing for global regulation against competitors and individuals who would develop their own. We desperately need "Open" in this context to be a broad, directional movement resisting that lockdown, not a rehash of licensing dogma drama. If you care about agency within the software ecosystem that surrounds you, at least differentiate the main points of Yann's arguments from holy wars about Llama's license.
- andy99 3y ago
  This isn't MIT vs GPL. It's literally one of those few corporations that's in a position to monopolize AI calling a license that says "you can use this at our pleasure" open source. Any open source movement definitively cannot include such licenses. If this was a split over a smaller thing I'd get your point.
  - evrydayhustling 3y ago
    Which clause are you calling "you can use this at our pleasure"? The license is short and there is no such thing. The most restrictive item is whether you had 750M active users in the month before it was released. [1] I'm curious what you are drawing issue with, but it's also under the bridge. Any sane reading of this license and the release shows that you have WAY more rights compared to GPT-4 or other fully closed releases. The availability of these weights and code are propping up an open source community around LLMs, which lends oxygen to other even-more-open endeavors. [1] https://ai.meta.com/llama/license/ https://ai.meta.com/llama/license/
    - lsaferite 3y ago
      I would assume they are referring to this: https://ai.meta.com/llama/use-policy/ https://ai.meta.com/llama/use-policy/
- logtempo 3y ago
  My two cents is that Europe will push, and are pushing, for open-source AI. They're regulating in that way.
megaman821 3y ago
People can and are building things using Llama2 that are not under Meta's control. Every comment is engaging in the definition of open source, but you obviously know what he is referring to. The pipeline to building a functional AI isn't going to perfectly map to just source code.
zer0c00ler 3y ago
Wouldn't open source mean you can recreate the model from scratch by knowing the training data? There is a weird definition of open source that creeps in here. Llama2 is effectively, proprietary compiled software. GPT-4 is proprietary compiled and hosted as SaaS. We have yet to see a true open source LLM... AI/ML is reinventing so many terms it's a bit annoying.
- cactusplant7374 3y ago
  If we did have access to the training data wouldn’t all these organizations be sued for distributing copyright works? It appears all my favorite books have been scanned by Open AI. I am assuming that they didn’t purchase them.
  - seeknotfind 3y ago
    Library corpus sets must be available for purchase for research or machine learning. Is the only way to buy each individual books? People could go to the library to learn. ML should have the same opportunity.
    - cactusplant7374 3y ago
      I support Altman and his team showing up at the library. No argument there.
- JieJie 3y ago
  Quake is Open Source, Quake's data files are not. Llama2 is actually more open than Quake.
asylteltine 3y ago
The source is available, you can get the weights, it’s open source…
aftbit 3y ago
The llama models are more open source than gpt-3.5-turbo. They aren't "free software" in the full definition, but they are available to be fine-tuned, modified, run offline, and generally tinkered with. For AI, "open source" is less about the license and what it allows you to do legally, and more about what the technology allows you to build practically. If/when Meta gets litigious, this might change in a hurry. Right now, there are bigger risks to AI startups (not Google) than worrying about how they'll choose to enforce the llama license.
cushpush 3y ago
No you can't license edge weights any more than you can license concentrations of ethanol in gasoline. Modified DNA can be patented, because it falls into one of the categories "process, machine, composition of matter, manufacture" in that it is a new composition of matter. Edge weights are not that. Edit: Actually, if the training set is IP and the method to train is IP then maybe the weights are licensable, given that it's trade-secret territory. However, ensuring that the edge-weights are not widely disseminated once shared with a licensee is not straightforward and the infeasibility of such makes it an uphill climb. Not like that stops the lawyers from clawing up the tree, tho.

by andy99 3y ago

smcleod 3y ago

The term has been polluted for a long time before AI became well known. Just like "DevOps", "Platform Engineering", "Agile" etc... and like those terms it's not a specific thing (e.g. a specific license like "Apache 2.0", "MIT" etc...) but more a general term without qualification of scope. Do I like it? No. But not liking how people are going to continue using a term shouldn't detract from the importance of the intent. At the end of the day - having LLM tech that has an open source _code_ base is what many people are talking about, the "data" that forms the model isn't always open source and while it would be nice if that was the case - is somewhat similar to open source software vs the data you run in it.

menzoic 3y ago

LLAMA2 is very much open source. You can use it for commercial purposes. The license only restricts companies that have more than 700M monthly active users from using it.

yumraj 3y ago
Releasing the trained model does not make it open source. Open source means something else. Calling released trained models open source is akin to releasing the binaries of a software for free, without releasing the source code, and calling it open source.
- smcleod 3y ago
  It's more like releasing the code as open source (under whatever license is specified) but to make the code useful you need data and that data might be free but not open sourced under any license. If it was trained on the entire internet for example - you can't just bundled up every piece of data on the internet into a zip file to include with the code to interpret it.
  - menzoic 3y ago
    >to make the code useful you need data and that data might be free but not open sourced under any license To use something like Meta’s open sourced LLAMA2 model you don’t need the data. The model is self contained. It’s a compressed lossy form of all the data it was trained on. The weights allow you to continue its training with new data of your choosing.
- menzoic 3y ago
  > Releasing the trained model does not make it open source. This is a straw man argument as Meta has released the model weights and not just the trained model. When it comes to models the weights are the source.
  - swatcoder 3y ago
    It may be a difference in terminology, but it's definitely no straw man. There's a philosophy of open source based on recipients being able to make any small and large modifications to the source material as they see fit. Just being able to build new projects with something or read what it does not qualify in that context -- it requires the freedom to rework the thing itself to suit your own needs and ends. In the case of these models, that philosophy would have the source training data and the training algorithms themselves available, editable, verifiable, and repeatable. Ignoring the financial challenges, a truly open source model would let you remove, add, edit, or reclassify a sample in the original training data and then build new weights of their won. The released weights are not that kind of open source, and that's what these arguments are generally making the a case for. There are still plenty of grounds from which to critique that argument if you wanted to, but there's no straw man involved in it.

evrydayhustling 3y ago

Cutting edge AI is a world-changing technology that is at immediate risk of being monopolized by 1-2 organizations pushing for global regulation against competitors and individuals who would develop their own. We desperately need "Open" in this context to be a broad, directional movement resisting that lockdown, not a rehash of licensing dogma drama. If you care about agency within the software ecosystem that surrounds you, at least differentiate the main points of Yann's arguments from holy wars about Llama's license.

andy99 3y ago
This isn't MIT vs GPL. It's literally one of those few corporations that's in a position to monopolize AI calling a license that says "you can use this at our pleasure" open source. Any open source movement definitively cannot include such licenses. If this was a split over a smaller thing I'd get your point.
- evrydayhustling 3y ago
  Which clause are you calling "you can use this at our pleasure"? The license is short and there is no such thing. The most restrictive item is whether you had 750M active users in the month before it was released. [1] I'm curious what you are drawing issue with, but it's also under the bridge. Any sane reading of this license and the release shows that you have WAY more rights compared to GPT-4 or other fully closed releases. The availability of these weights and code are propping up an open source community around LLMs, which lends oxygen to other even-more-open endeavors. [1] https://ai.meta.com/llama/license/ https://ai.meta.com/llama/license/
  - lsaferite 3y ago
    I would assume they are referring to this: https://ai.meta.com/llama/use-policy/ https://ai.meta.com/llama/use-policy/
logtempo 3y ago
My two cents is that Europe will push, and are pushing, for open-source AI. They're regulating in that way.

megaman821 3y ago

People can and are building things using Llama2 that are not under Meta's control. Every comment is engaging in the definition of open source, but you obviously know what he is referring to. The pipeline to building a functional AI isn't going to perfectly map to just source code.

zer0c00ler 3y ago

Wouldn't open source mean you can recreate the model from scratch by knowing the training data? There is a weird definition of open source that creeps in here. Llama2 is effectively, proprietary compiled software. GPT-4 is proprietary compiled and hosted as SaaS. We have yet to see a true open source LLM... AI/ML is reinventing so many terms it's a bit annoying.

cactusplant7374 3y ago
If we did have access to the training data wouldn’t all these organizations be sued for distributing copyright works? It appears all my favorite books have been scanned by Open AI. I am assuming that they didn’t purchase them.
- seeknotfind 3y ago
  Library corpus sets must be available for purchase for research or machine learning. Is the only way to buy each individual books? People could go to the library to learn. ML should have the same opportunity.
  - cactusplant7374 3y ago
    I support Altman and his team showing up at the library. No argument there.
JieJie 3y ago
Quake is Open Source, Quake's data files are not. Llama2 is actually more open than Quake.

asylteltine 3y ago

The source is available, you can get the weights, it’s open source…

aftbit 3y ago

The llama models are more open source than gpt-3.5-turbo. They aren't "free software" in the full definition, but they are available to be fine-tuned, modified, run offline, and generally tinkered with. For AI, "open source" is less about the license and what it allows you to do legally, and more about what the technology allows you to build practically. If/when Meta gets litigious, this might change in a hurry. Right now, there are bigger risks to AI startups (not Google) than worrying about how they'll choose to enforce the llama license.

cushpush 3y ago

No you can't license edge weights any more than you can license concentrations of ethanol in gasoline. Modified DNA can be patented, because it falls into one of the categories "process, machine, composition of matter, manufacture" in that it is a new composition of matter. Edge weights are not that. Edit: Actually, if the training set is IP and the method to train is IP then maybe the weights are licensable, given that it's trade-secret territory. However, ensuring that the edge-weights are not widely disseminated once shared with a licensee is not straightforward and the infeasibility of such makes it an uphill climb. Not like that stops the lawyers from clawing up the tree, tho.