5 ms·

LLAMA2 is very much open source. You can use it for commercial purposes. The license only restricts companies that have more than 700M monthly active users from

LLAMA2 is very much open source. You can use it for commercial purposes. The license only restricts companies that have more than 700M monthly active users from using it.

yumraj 3y ago
Releasing the trained model does not make it open source. Open source means something else. Calling released trained models open source is akin to releasing the binaries of a software for free, without releasing the source code, and calling it open source.
- smcleod 3y ago
  It's more like releasing the code as open source (under whatever license is specified) but to make the code useful you need data and that data might be free but not open sourced under any license. If it was trained on the entire internet for example - you can't just bundled up every piece of data on the internet into a zip file to include with the code to interpret it.
  - menzoic 3y ago
    >to make the code useful you need data and that data might be free but not open sourced under any license To use something like Meta’s open sourced LLAMA2 model you don’t need the data. The model is self contained. It’s a compressed lossy form of all the data it was trained on. The weights allow you to continue its training with new data of your choosing.
- menzoic 3y ago
  > Releasing the trained model does not make it open source. This is a straw man argument as Meta has released the model weights and not just the trained model. When it comes to models the weights are the source.
  - swatcoder 3y ago
    It may be a difference in terminology, but it's definitely no straw man. There's a philosophy of open source based on recipients being able to make any small and large modifications to the source material as they see fit. Just being able to build new projects with something or read what it does not qualify in that context -- it requires the freedom to rework the thing itself to suit your own needs and ends. In the case of these models, that philosophy would have the source training data and the training algorithms themselves available, editable, verifiable, and repeatable. Ignoring the financial challenges, a truly open source model would let you remove, add, edit, or reclassify a sample in the original training data and then build new weights of their won. The released weights are not that kind of open source, and that's what these arguments are generally making the a case for. There are still plenty of grounds from which to critique that argument if you wanted to, but there's no straw man involved in it.
    - andy99 3y ago
      I don't want to support the guy you replied to, he doesn't understand what open source means, but I belive and have argued it is possible to exercise the freedoms that are part of open source software without access to the training data. See https://www.marble.onl/posts/considerations_for_copyrighting_AI.html https://www.marble.onl/posts/considerations_for_copyrighting...
      - cactusplant7374 3y ago
        Did all of these organizations acquire training data in a legal manner? Seems like we are tip toeing around the real issue here.
    - menzoic 3y ago
      >There's a philosophy of open source based on recipients being able to make any small and large modifications to the source material as they see fit. The reason I see it as a strawman is because the weights are the source of the AI model and they do allow you to modify the model however you please. The data is not owned by Meta, they can’t release it and it’s not required to use or modify the model. This is like wanting a team to release the research that informed product development with their source. The data and training code are not the same as the AI model itself. This is not the same as code that produces a binary, this is more like code that produces another form of code. If you try to replicate the model you won’t be able to because the process is not deterministic. There’s no reason to replicate the model, you would fine tune it instead.
andy99 3y ago
You demonstrably don't understand what open source means, which is basically my point, that people like Lecun have been trying to corrupt the meaning to be equivalent to "source available". I agree with the point you made elsewhere that releasing the data is irrelevant.
- menzoic 3y ago
  > You demonstrably don't understand what open source means, which is basically my point, that people like Lecun have been trying to corrupt the meaning to be equivalent to "source available". You demonstrably don’t understand what it means to open source an AI model. The code is not relevant at all, the same code will not produce the same model weights, it is not deterministic. Also you will need millions of dollars to retrain, retraining is not the goal. The model weights are the source that was opened.
  - dragonwriter 3y ago
    > The model weights are the source that was opened. Model weights are not "the preferred form in which a programmer would modify the program" but more like "the output of a preprocessor or translator" [0], and thus are not "source code", but viewed as code, are more like object code. The training data, training configuration, and program source for the training routines are "source code", without which the model as a whole is not truly open source, just as much as the program source for the inference routines are. [0] for the relevance of the quoted material to whether or not your description meets the Open Source Definition, see the definition itself: https://opensource.org/osd/ https://opensource.org/osd/
    - jph00 3y ago
      I think the model weights are the preferred form in which a programmer would modify the program. We modify models by fine-tuning them, which requires the weights, but not the training data/recipe.
  - squeaky-clean 3y ago
    I think you're misunderstanding their point and mixing it up with other commenters. They are saying everything is available, but there are license restrictions on how you can use it. They're not asking for more code or data. It's source available, but it's not "open source" in the historical usage by programmers. Microsoft could not take their model weights and use it. It's not open source.
amelius 3y ago
It's only open source if you can generate the binary weights from the training data and the training algorithm and training (hyper) parameters, from scratch. Releasing only the binary weights, does not make it open source, just like releasing only binary executables has nothing to do with open source.
- menzoic 3y ago
  > It's only open source if you can generate the binary weights from the training data and the training algorithm and training (hyper) parameters, from scratch. Releasing only the binary weights, does not make it open source, just like releasing only binary executables has nothing to do with open source. The weights aren’t binary. It’s not a compile form, it’s the actual source of the AI model. The same code used to train the model / generate the weights will not produce the same model weights if run again, it is not deterministic. Also you will need millions of dollars to retrain, retraining is not the goal. The model weights are the source that was opened. The weights can be used to modify the model.
  - amelius 3y ago
    > The weights aren’t binary. The weights are typically distributed as a binary blob containing IEEE floating point values, or fixed point values; and these numbers are typically not meant for direct human consumption. > The same code used to train the model / generate the weights will not produce the same model weights if run again, it is not deterministic. This is not true, except in contrived situations like when you use PRNGs and you deliberately throw away their seeds. > Also you will need millions of dollars to retrain This is besides the point.
notfed 3y ago
This comes down to arguing over semantics. Many will argue that "open source" means using a license listed at https://opensource.org/licenses/ https://opensource.org/licenses/ . Even CC0 isn't considered open source by this standard. Even I feel that "700M monthly active user" gives not-so-open vibes, but frankly many of these "open source" licenses have oddly restrictive requirements as well.
dragonwriter 3y ago
> LLAMA2 is very much open source. The inference and training software used to run the model are open source, the concrete model -- that is, the thing for which the weights are the object code -- is not. The concrete model is free-to-use closed source, which is better than an undisclosed blob hiding behind a SaaS service, but still not open source. It's also good that the inference and training code are open source, even though the training data and configuration is not.

LLAMA2 is very much open source. You can use it for commercial purposes. The license only restricts companies that have more than 700M monthly active users from using it.

yumraj 3y ago

Releasing the trained model does not make it open source. Open source means something else. Calling released trained models open source is akin to releasing the binaries of a software for free, without releasing the source code, and calling it open source.

smcleod 3y ago
It's more like releasing the code as open source (under whatever license is specified) but to make the code useful you need data and that data might be free but not open sourced under any license. If it was trained on the entire internet for example - you can't just bundled up every piece of data on the internet into a zip file to include with the code to interpret it.
- menzoic 3y ago
  >to make the code useful you need data and that data might be free but not open sourced under any license To use something like Meta’s open sourced LLAMA2 model you don’t need the data. The model is self contained. It’s a compressed lossy form of all the data it was trained on. The weights allow you to continue its training with new data of your choosing.
menzoic 3y ago
> Releasing the trained model does not make it open source. This is a straw man argument as Meta has released the model weights and not just the trained model. When it comes to models the weights are the source.
- swatcoder 3y ago
  It may be a difference in terminology, but it's definitely no straw man. There's a philosophy of open source based on recipients being able to make any small and large modifications to the source material as they see fit. Just being able to build new projects with something or read what it does not qualify in that context -- it requires the freedom to rework the thing itself to suit your own needs and ends. In the case of these models, that philosophy would have the source training data and the training algorithms themselves available, editable, verifiable, and repeatable. Ignoring the financial challenges, a truly open source model would let you remove, add, edit, or reclassify a sample in the original training data and then build new weights of their won. The released weights are not that kind of open source, and that's what these arguments are generally making the a case for. There are still plenty of grounds from which to critique that argument if you wanted to, but there's no straw man involved in it.
  - andy99 3y ago
    I don't want to support the guy you replied to, he doesn't understand what open source means, but I belive and have argued it is possible to exercise the freedoms that are part of open source software without access to the training data. See https://www.marble.onl/posts/considerations_for_copyrighting_AI.html https://www.marble.onl/posts/considerations_for_copyrighting...
    - cactusplant7374 3y ago
      Did all of these organizations acquire training data in a legal manner? Seems like we are tip toeing around the real issue here.
  - menzoic 3y ago
    >There's a philosophy of open source based on recipients being able to make any small and large modifications to the source material as they see fit. The reason I see it as a strawman is because the weights are the source of the AI model and they do allow you to modify the model however you please. The data is not owned by Meta, they can’t release it and it’s not required to use or modify the model. This is like wanting a team to release the research that informed product development with their source. The data and training code are not the same as the AI model itself. This is not the same as code that produces a binary, this is more like code that produces another form of code. If you try to replicate the model you won’t be able to because the process is not deterministic. There’s no reason to replicate the model, you would fine tune it instead.

andy99 3y ago

You demonstrably don't understand what open source means, which is basically my point, that people like Lecun have been trying to corrupt the meaning to be equivalent to "source available". I agree with the point you made elsewhere that releasing the data is irrelevant.

menzoic 3y ago
> You demonstrably don't understand what open source means, which is basically my point, that people like Lecun have been trying to corrupt the meaning to be equivalent to "source available". You demonstrably don’t understand what it means to open source an AI model. The code is not relevant at all, the same code will not produce the same model weights, it is not deterministic. Also you will need millions of dollars to retrain, retraining is not the goal. The model weights are the source that was opened.
- dragonwriter 3y ago
  > The model weights are the source that was opened. Model weights are not "the preferred form in which a programmer would modify the program" but more like "the output of a preprocessor or translator" [0], and thus are not "source code", but viewed as code, are more like object code. The training data, training configuration, and program source for the training routines are "source code", without which the model as a whole is not truly open source, just as much as the program source for the inference routines are. [0] for the relevance of the quoted material to whether or not your description meets the Open Source Definition, see the definition itself: https://opensource.org/osd/ https://opensource.org/osd/
  - jph00 3y ago
    I think the model weights are the preferred form in which a programmer would modify the program. We modify models by fine-tuning them, which requires the weights, but not the training data/recipe.
- squeaky-clean 3y ago
  I think you're misunderstanding their point and mixing it up with other commenters. They are saying everything is available, but there are license restrictions on how you can use it. They're not asking for more code or data. It's source available, but it's not "open source" in the historical usage by programmers. Microsoft could not take their model weights and use it. It's not open source.

amelius 3y ago

It's only open source if you can generate the binary weights from the training data and the training algorithm and training (hyper) parameters, from scratch. Releasing only the binary weights, does not make it open source, just like releasing only binary executables has nothing to do with open source.

menzoic 3y ago
> It's only open source if you can generate the binary weights from the training data and the training algorithm and training (hyper) parameters, from scratch. Releasing only the binary weights, does not make it open source, just like releasing only binary executables has nothing to do with open source. The weights aren’t binary. It’s not a compile form, it’s the actual source of the AI model. The same code used to train the model / generate the weights will not produce the same model weights if run again, it is not deterministic. Also you will need millions of dollars to retrain, retraining is not the goal. The model weights are the source that was opened. The weights can be used to modify the model.
- amelius 3y ago
  > The weights aren’t binary. The weights are typically distributed as a binary blob containing IEEE floating point values, or fixed point values; and these numbers are typically not meant for direct human consumption. > The same code used to train the model / generate the weights will not produce the same model weights if run again, it is not deterministic. This is not true, except in contrived situations like when you use PRNGs and you deliberately throw away their seeds. > Also you will need millions of dollars to retrain This is besides the point.

notfed 3y ago

This comes down to arguing over semantics. Many will argue that "open source" means using a license listed at https://opensource.org/licenses/ https://opensource.org/licenses/ . Even CC0 isn't considered open source by this standard. Even I feel that "700M monthly active user" gives not-so-open vibes, but frankly many of these "open source" licenses have oddly restrictive requirements as well.

dragonwriter 3y ago

> LLAMA2 is very much open source. The inference and training software used to run the model are open source, the concrete model -- that is, the thing for which the weights are the object code -- is not. The concrete model is free-to-use closed source, which is better than an undisclosed blob hiding behind a SaaS service, but still not open source. It's also good that the inference and training code are open source, even though the training data and configuration is not.