3 ms·

>I don't understand how this isn't a derivative work of the original code The original code is Apache 2 licensed. Derivatives are fine and allowed. This retain

>I don't understand how this isn't a derivative work of the original code The original code is Apache 2 licensed. Derivatives are fine and allowed. This retains the same Apache 2 license as Facebook's code. It's only the model that isn't covered by that permissive Apache 2 license. A model produced by a derivative of the permissively licensed code, or even by the original code itself, is not a derivative or the original non-permissively licensed model produced by the original code and is non-infringing even if it is a bit-perfect replica. > Doesn't the original FB license also apply to the weights? Again, there are different licenses for the code and the model and neither license actually applies to the weights within the model only the actual exact model. If this project produced a bit-for-bit replica of Facebook's model it would still not infringe on that model's license. But it doesn't produce a bit-for-bit replica. Even if Facebook were to re-run their same training code on their same hardware would they could not produce the exact same weights as before since massively parallel matrix multiplications are not deterministic. Benign environmental noise like microscopic fluctuations in temperature make a difference in the outcome.

kristjansson 4y ago
> Apache 2 Isn't the original GPLv3[0]? [0]: https://github.com/facebookresearch/llama/blob/main/LICENSE https://github.com/facebookresearch/llama/blob/main/LICENSE
- lantiga 4y ago
  Correct, the original is GPL 3. To produce this implementation from the LLaMA paper we started from github.com/karpathy/nanoGPT, the LLaMA architecture is really similar to GPT. For instance we added rotary positional encoding starting from the original RoPE repo published with the paper. We finally ran the original model to make sure the two models were numerically.
shagie 4y ago
Can a model itself be copyrighted? https://www.copyright.gov/circs/circ33.pdf https://www.copyright.gov/circs/circ33.pdf > To register a work with the U.S. Copyright Office, you must identify the copyrightable subject matter forming the basis of your claim. To be copyrightable, a work must qualify as an original work of authorship, meaning that it must have been created independently and contain a sufficient amount of creativity. Most works meet these conditions. Some works, however, contain elements that either lack the required creativity or are placed outside the bounds of copyright by the law. This circular highlights different types of noncopyrightable subject matter. For more information, see chapter 300, section 313.3, of the Compendium of U.S. Copyright Office Practices.1 https://www.copyright.gov/comp3/chap300/ch300-copyrightable-authorship.pdf https://www.copyright.gov/comp3/chap300/ch300-copyrightable-... > 313.2 Works That Lack Human Authorship > Similarly, the Office will not register works produced by a machine or mere mechanical process that operates randomly or automatically without any creative input or intervention from a human author. The crucial question is “whether the ‘work’ is basically one of human authorship, with the computer [or other device] merely being an assisting instrument, or whether the traditional elements of authorship in the work (literary, artistic, or musical expression or elements of selection, arrangement, etc.) were actually conceived and executed not by man but by a machine.” > Examples > Reducing or enlarging the size of a preexisting work of authorship. ---- I'm going to go with that models themselves lack human authorship or any sufficient amount of creativity and thus aren't copyrightable. They may fall under trade secret... but they need to be treated as one. This would also mean that licenses based on copyright (most of FOSS licenses) wouldn't be applicable to them.

kristjansson 4y ago

> Apache 2 Isn't the original GPLv3[0]? [0]: https://github.com/facebookresearch/llama/blob/main/LICENSE https://github.com/facebookresearch/llama/blob/main/LICENSE

lantiga 4y ago
Correct, the original is GPL 3. To produce this implementation from the LLaMA paper we started from github.com/karpathy/nanoGPT, the LLaMA architecture is really similar to GPT. For instance we added rotary positional encoding starting from the original RoPE repo published with the paper. We finally ran the original model to make sure the two models were numerically.

shagie 4y ago

Can a model itself be copyrighted? https://www.copyright.gov/circs/circ33.pdf https://www.copyright.gov/circs/circ33.pdf > To register a work with the U.S. Copyright Office, you must identify the copyrightable subject matter forming the basis of your claim. To be copyrightable, a work must qualify as an original work of authorship, meaning that it must have been created independently and contain a sufficient amount of creativity. Most works meet these conditions. Some works, however, contain elements that either lack the required creativity or are placed outside the bounds of copyright by the law. This circular highlights different types of noncopyrightable subject matter. For more information, see chapter 300, section 313.3, of the Compendium of U.S. Copyright Office Practices.1 https://www.copyright.gov/comp3/chap300/ch300-copyrightable-authorship.pdf https://www.copyright.gov/comp3/chap300/ch300-copyrightable-... > 313.2 Works That Lack Human Authorship > Similarly, the Office will not register works produced by a machine or mere mechanical process that operates randomly or automatically without any creative input or intervention from a human author. The crucial question is “whether the ‘work’ is basically one of human authorship, with the computer [or other device] merely being an assisting instrument, or whether the traditional elements of authorship in the work (literary, artistic, or musical expression or elements of selection, arrangement, etc.) were actually conceived and executed not by man but by a machine.” > Examples > Reducing or enlarging the size of a preexisting work of authorship. ---- I'm going to go with that models themselves lack human authorship or any sufficient amount of creativity and thus aren't copyrightable. They may fall under trade secret... but they need to be treated as one. This would also mean that licenses based on copyright (most of FOSS licenses) wouldn't be applicable to them.