6 ms·

> Maybe the supporter of the definition could demonstrate practically modifying a ML model without using the original training data, and show that it is just as

> Maybe the supporter of the definition could demonstrate practically modifying a ML model without using the original training data, and show that it is just as easy as with the original data and it does not limit what you can do with it (e.g. demonstrate it can unlearn any parts of the original data as if they were not used). I quite like that comment that was left on the article. I know some models you can tweak the weights, without the source data, but it does seem like you are more restricted without the actual dataset. Personally, the data seems to be part of the source to me, in this case. I mean, the code is derived from the data itself, the weights are the artifact of training. If anything, they should provide the data, the training methodology, the model architecture, the code to train and infer, and the weights could be optional. I mean, the weights basically are equivalent to a built artifact, like the compiled software. And that means commercially, people would pay for the cost of training. I might not have the resources to "compile" it myself, aka, run the training, so maybe I pay a subscription to a service that did.

a2128 2y ago
The usefulness of data here is that you can retrain the model after making changes to its architecture, e.g. seeing if it works better with a different activation function. Of course this is most useful for models small enough that you could train it within a few days on a consumer GPU. When it comes to LLMs only the richest companies would have the adequate resources to retrain.
nextaccountic 2y ago
The source is really the training data plus all code required to train the model. I might not have resources to "compile", and also "compilation" is not deterministic, but those are technical details You could have a programming language whose compiler is a superoptimizer that's very slow and is also stochastic, and it would amount to the same thing in practice.
lolinder 2y ago
A lot of people get hung up on `weights = compiled-artifact` because both are binary representations, but there are major limitations to this comparison. When we're dealing with source code, the cost of getting from source -> binary is minimal. The entire Linux kernel builds in two hours on one modest machine. Since it's cheap to compile and the source code is itself legible, the source code is the preferred form for making modifications. This doesn't work when we try to apply the same reasoning to `training data -> weights`. "Compilation" in this world costs hundreds of millions of dollars per compilation run. Cost of "compilation" alone means that the preferred form for making modifications can't possibly be the training data, even for the company that built the thing in the first place. As for the data itself, it's a far cry from source code—we're talking tens of terrabytes of data at a minimum, which is likewise infeasible to work with on a regular basis. The weights must be the preferred form for making modifications for simple logistics reasons. Importantly, the weights are the preferred form for modifications even for the companies that built them. I think a far more reasonable analogy, to the extent that any are reasonable, is that the training data is all the stuff that the developers of the FOSS software ever learned, and the thousands of computer-hours spent on training are the thousands of man-hours spent coding. The entire point of FOSS is for a few experts to do all that work once and then we all can share and modify the output of those years of work and millions of dollars invested as we see fit, without having to waste all that time and money doing it over again. We don't expect the authors of the Linux kernel to document their every waking thought so we could recreate the process they used to produce the kernel code... we just thank them for the kernel code and contribute to it as best we can.
- smolder 2y ago
  If you can't bootstrap a set of weights from public material, the weights aren't open source, because it's derivative content based on something non-open. Trying to draw an equivalency between code and weights is [edited for temperament, I guess] not right. They are built from the source material supplied to an algorithm. Weights are data, not code. Otherwise, everyone on the internet would be an author, and would have a say in the licensing of the weights.
  - lolinder 2y ago
    > Trying to draw an equivalency between code and weights is ridiculous, because the weights are not written in the same way as source code. By the same logic, the comparison between a compiled artifact and weights fails because the weights are not "compiled" in any meaningful sense. Analogies will always fail, which is why "preferred form for making modifications" is the rod we use, not vague attempts at drawing analogies between completely different development processes. > They are built from the source material supplied to an algorithm. Weights are data, not code. As Lispers know well, code is data and data is code. You can't draw a line in the sand and definitively say that on this side of the line is just code and on that side is just data. In terms of how they behave, weights function as code that is executed by an interpreter that we call an inference engine.
    - dietr1ch 2y ago
      > > Trying to draw an equivalency between code and weights is ridiculous, because the weights are not written in the same way as source code. > By the same logic, the comparison between a compiled artifact and weights fails because the weights are not "compiled" in any meaningful sense. To me the weights map to assembly and the training data+models map to source code+compilers. Sure, you can hand me assembly, and with the assembly I may be able to execute the model/program, but having it does not mean that I can stare at it and learn nor modify it with a reasonable understanding of what's going to change. I got to add that the situation feels even worse than assembly, because assembly, hand-coded or mutilated by an optimizing compiler still does something very specific and deterministic, but the weights on the model makes things equivalent to programming without booleans, but seemingly random numbers and checking for inequalities to get a binary decision.

a2128 2y ago

The usefulness of data here is that you can retrain the model after making changes to its architecture, e.g. seeing if it works better with a different activation function. Of course this is most useful for models small enough that you could train it within a few days on a consumer GPU. When it comes to LLMs only the richest companies would have the adequate resources to retrain.

nextaccountic 2y ago

The source is really the training data plus all code required to train the model. I might not have resources to "compile", and also "compilation" is not deterministic, but those are technical details You could have a programming language whose compiler is a superoptimizer that's very slow and is also stochastic, and it would amount to the same thing in practice.

lolinder 2y ago

A lot of people get hung up on `weights = compiled-artifact` because both are binary representations, but there are major limitations to this comparison. When we're dealing with source code, the cost of getting from source -> binary is minimal. The entire Linux kernel builds in two hours on one modest machine. Since it's cheap to compile and the source code is itself legible, the source code is the preferred form for making modifications. This doesn't work when we try to apply the same reasoning to `training data -> weights`. "Compilation" in this world costs hundreds of millions of dollars per compilation run. Cost of "compilation" alone means that the preferred form for making modifications can't possibly be the training data, even for the company that built the thing in the first place. As for the data itself, it's a far cry from source code—we're talking tens of terrabytes of data at a minimum, which is likewise infeasible to work with on a regular basis. The weights must be the preferred form for making modifications for simple logistics reasons. Importantly, the weights are the preferred form for modifications even for the companies that built them. I think a far more reasonable analogy, to the extent that any are reasonable, is that the training data is all the stuff that the developers of the FOSS software ever learned, and the thousands of computer-hours spent on training are the thousands of man-hours spent coding. The entire point of FOSS is for a few experts to do all that work once and then we all can share and modify the output of those years of work and millions of dollars invested as we see fit, without having to waste all that time and money doing it over again. We don't expect the authors of the Linux kernel to document their every waking thought so we could recreate the process they used to produce the kernel code... we just thank them for the kernel code and contribute to it as best we can.

smolder 2y ago
If you can't bootstrap a set of weights from public material, the weights aren't open source, because it's derivative content based on something non-open. Trying to draw an equivalency between code and weights is [edited for temperament, I guess] not right. They are built from the source material supplied to an algorithm. Weights are data, not code. Otherwise, everyone on the internet would be an author, and would have a say in the licensing of the weights.
- lolinder 2y ago
  > Trying to draw an equivalency between code and weights is ridiculous, because the weights are not written in the same way as source code. By the same logic, the comparison between a compiled artifact and weights fails because the weights are not "compiled" in any meaningful sense. Analogies will always fail, which is why "preferred form for making modifications" is the rod we use, not vague attempts at drawing analogies between completely different development processes. > They are built from the source material supplied to an algorithm. Weights are data, not code. As Lispers know well, code is data and data is code. You can't draw a line in the sand and definitively say that on this side of the line is just code and on that side is just data. In terms of how they behave, weights function as code that is executed by an interpreter that we call an inference engine.
  - dietr1ch 2y ago
    > > Trying to draw an equivalency between code and weights is ridiculous, because the weights are not written in the same way as source code. > By the same logic, the comparison between a compiled artifact and weights fails because the weights are not "compiled" in any meaningful sense. To me the weights map to assembly and the training data+models map to source code+compilers. Sure, you can hand me assembly, and with the assembly I may be able to execute the model/program, but having it does not mean that I can stare at it and learn nor modify it with a reasonable understanding of what's going to change. I got to add that the situation feels even worse than assembly, because assembly, hand-coded or mutilated by an optimizing compiler still does something very specific and deterministic, but the weights on the model makes things equivalent to programming without booleans, but seemingly random numbers and checking for inequalities to get a binary decision.