3 ms·

The key is if you consider weights source code. I do not think this is a common interpretation. > The labs themselves modify models the same as you are allowed

The key is if you consider weights source code. I do not think this is a common interpretation. > The labs themselves modify models the same as you are allowed to by the license Do the labs do not use source code? It is a bit like arguing that releasing a binary executable is releasing the source code. One could claim developers modify the binary the same as you are allowed to.

NitpickLawyer 1y ago
> Do the labs do not use source code? The weights are part of the source code. When running inference on a model you use the architecture, config files and weights together. All of these are released. Weights are nothing but "hardcoded values". The way you reach those values is irrelevant in the license discussion. Let's take a simple example: I write a chess program that is comprised of a source file with 10 "if" statements, a config file that matches between the variables used in the if statements and a "hardcoded values" file that stores the actual values. It would be a crappy chess program, but I hope you agree that I can release that as open source and no-one would bat an eye. You would also be granted the right to edit those hardcoded values, if you wish so. You'd perhaps make the chess bot better or worse. But you would be allowed to edit it, just like I would. That's the preferred way of modifying it. Me providing the methods that I used to reach those 10 hardcoded values has 0 bearing on my crappy chess bot being open source or not. Do we agree on that? Now instead of 10 values, make it 100billion. Hey, that's an LLM! > It is a bit like arguing that releasing a binary executable is releasing the source code. That's the misconception. Weights are not a binary executable. In other words, there isn't another level above weights that the labs use to "compile" the weights. The weights exist from the beginning to the end, and the labs edit the weights if they want to modify the models. And so can you. There isn't a "compilation" step anywhere in the course of training a model.
- 127 1y ago
  Training is obviously the compilation step.
- jdiff 1y ago
  If you have 10 harcoded values, you have a binary blob, a common feature particularly in hardware drivers that is opaque and commonly considered to not be fully free unless the instructions for deriving it are also included. It's frequently just an executable, occasionally just configuration information, but difficult to change while (assuming no signing shenanigans) still remaining technically possible. The training data is the source code and the training process is the compiler. There's a fairly direct metaphor to be made there. Different compilers can have vastly different impacts on the performance of the compiled executable.
- mejutoco 1y ago
  > The weights are part of the source code. If you will allow me the absurd analogy: my arm is also part of a person (me), but my arm is not a person. My arm does not have its own bank account and pays taxes independently. I get how the weights are not exactly like binary code. Good points. But they are also not source code (from your own quote) > "Source" form shall mean the preferred form for making modifications The weights are not the preferred form of making modifications. At most, one could argue it is the weights + source code. > In other words, there isn't another level above weights that the labs use to "compile" the weights The source and training data? I see your points, and it is an interesting discussion of nuances, but I profoundly disagree that the weights are "the preferred form for making modifications". For these reasons I prefer the term "open weights" for these projects.

NitpickLawyer 1y ago

> Do the labs do not use source code? The weights are part of the source code. When running inference on a model you use the architecture, config files and weights together. All of these are released. Weights are nothing but "hardcoded values". The way you reach those values is irrelevant in the license discussion. Let's take a simple example: I write a chess program that is comprised of a source file with 10 "if" statements, a config file that matches between the variables used in the if statements and a "hardcoded values" file that stores the actual values. It would be a crappy chess program, but I hope you agree that I can release that as open source and no-one would bat an eye. You would also be granted the right to edit those hardcoded values, if you wish so. You'd perhaps make the chess bot better or worse. But you would be allowed to edit it, just like I would. That's the preferred way of modifying it. Me providing the methods that I used to reach those 10 hardcoded values has 0 bearing on my crappy chess bot being open source or not. Do we agree on that? Now instead of 10 values, make it 100billion. Hey, that's an LLM! > It is a bit like arguing that releasing a binary executable is releasing the source code. That's the misconception. Weights are not a binary executable. In other words, there isn't another level above weights that the labs use to "compile" the weights. The weights exist from the beginning to the end, and the labs edit the weights if they want to modify the models. And so can you. There isn't a "compilation" step anywhere in the course of training a model.

127 1y ago
Training is obviously the compilation step.
jdiff 1y ago
If you have 10 harcoded values, you have a binary blob, a common feature particularly in hardware drivers that is opaque and commonly considered to not be fully free unless the instructions for deriving it are also included. It's frequently just an executable, occasionally just configuration information, but difficult to change while (assuming no signing shenanigans) still remaining technically possible. The training data is the source code and the training process is the compiler. There's a fairly direct metaphor to be made there. Different compilers can have vastly different impacts on the performance of the compiled executable.
mejutoco 1y ago
> The weights are part of the source code. If you will allow me the absurd analogy: my arm is also part of a person (me), but my arm is not a person. My arm does not have its own bank account and pays taxes independently. I get how the weights are not exactly like binary code. Good points. But they are also not source code (from your own quote) > "Source" form shall mean the preferred form for making modifications The weights are not the preferred form of making modifications. At most, one could argue it is the weights + source code. > In other words, there isn't another level above weights that the labs use to "compile" the weights The source and training data? I see your points, and it is an interesting discussion of nuances, but I profoundly disagree that the weights are "the preferred form for making modifications". For these reasons I prefer the term "open weights" for these projects.