5 ms·

ML weights are code

jchw 2y ago
I'll accept that if you would like. However, they are not source code if so. They are object code. And open source is about source code, not object code. (And this particular press release isn't about ML weights anyways, at least unless I'm grossly misunderstanding; it is just a dataset. So even failing this, it still doesn't really make any sense.)
- jncfhnb 2y ago
  No it is not object code unless you want to get so stupidly pedantic that you want to argue a Python script in a zip file can’t be considered open source because it’s compressed. The model pickles unpack back to their original form. The picked binary forms are merely for convenience.
  - jchw 2y ago
    Look, please go do research as to what "object code" and "source code" are before saying my argument is "stupidly pedantic". I'm not elaborating because the example you gave has nothing to do with what I said.
    - jncfhnb 2y ago
      Your analogy does not make sense. ML weights are distributed in binary form, like object code, but it is nothing like compiled binary. It’s just temporarily in binary form for convenience. It unpacks directly into its original form. This is not a technicality like “technically up can reverse engineer or modify binary code”. The binary form of model weights is just a fancy zip file format that is useful because they are so large that text is impractical.
      - jchw 2y ago
        Source code is human readable. Object code is not, and produced from some mechanical process. Model weights are not written by hand. You don't manually tweak individual weights. You have to run a training process that has multiple "raw" inputs. Trying to read model weights directly is no better than trying to read object code directly. Heck, reading object code directly is probably easier, because at least it's just machine code at the bottom; I will never be able to comprehend what's going on in an ML model just by reading the weights. The closest thing to "source code" in ML models would be the inputs to the training process, because that's the "source" of the model weights that pops out the other end. If the analogy doesn't make sense, that's because ML models are probably not really code in the same sense that source code and object code. (It may be tempting to look at "ML weights" as source code because of the existence of "closed-weight" API services. Please consider the following: If Amazon offers me a unique database service that I can only use with Amazon Web Services, and then releases a closed-source binary that you can run locally, that is still closed-source, because you don't have the source code.)
        jncfhnb 2y ago
        “Human readable” is not a requirement. Visual programming code breaks down to some obtuse data structure. But with the right tools, it’s easy for humans to interact with it. Visual programming node workflows can be open sourced. ML models are the same. Tooling is required to interact with it. The limits of your human understanding do not determine if something is open source. Otherwise a really complicated traditional program might be argued as not open source. You can individually explore specific vectors and layers of a model and their significance. Produced by a non mechanistic process is not a requirement. I can generate a hello world script with code, and open source the hello world script. It does not matter how it was formed. I do not need to open source the hello world generator either. Data and training code is not source code of the model. That is the source code of a model maker. That’s `make_hello_world.py` not `hello_world.py` The closed source database is not a correct analogy. Excluding unreasonably difficult efforts to decompile the binary, you CANNOT modify the program without expecting it to break. With an ML model, the weights are the PREFERRED method of modifying the program. You do NOT want the original data and training code. That will just be a huge expense to get you what you already have. If you want the model to be different, you take the model weights and change them. Not recreate them differently from scratch. Which is the same for all traditional code. Open source does not mean I provide you with the design documents and testing feedback to demonstrate how the code base got created. It means you get the code base. Recreating the codebase is not something we think about because it doesn’t make sense because we have the code and we have the models.
shevis 2y ago
No, they really aren’t and I’m not sure why I keep seeing this take. ML weights are binary and it’s painfully obvious. They are the end result of a compilation process in which the training data and model code are compiled into the resulting weights. If you can’t even theoretically recreate the weights on your own hardware it isn’t open source.
- jncfhnb 2y ago
  ML weights are not binary. They are modifiable. If produce a program that outputs a hello world file, I can open source the hello world script without open sourcing the hello world generator.
deleted 2y ago
[deleted]
kennyadam 2y ago
Technically, but it feels like you're intentionally missing the point being made. Sure, providing the weights is very useful given the cost of generating them, but you can't exactly learn much by looking through the 'code', make changes and gain an in-depth understanding in the same way you can from the code provided by an actual open source project.
- jncfhnb 2y ago
  You absolutely can and people do all the time. There are mountains of forks and dissections and improvements on open source models.
i_read_news 2y ago
We can also say binaries are code, but if we are being pedantic that likely isn’t the source code that generated the binary (I also doubt the intention of hand writing binary or manually inputting billions of weights). I’d reckon that’s why it’s called open source, not open code or open binary, as the source code that generates the data is distributed. I’d actually just call this for what it is - open weights.
- jncfhnb 2y ago
  Binary is not the equivalent of models. Source code is the equivalent of models. It doesn’t matter if a machine generated source code or a human did for it to be open source code.
  - EnigmaFlare 2y ago
    You keep asserting this but without any reason. Do you have a reason? It seems to go against the general open source idea of source code being convenient for people to modify.
    - jncfhnb 2y ago
      ML weights ARE convenient for people to modify. You can go look at the dozens of modifications of diffusion models being produced, daily, on civit ai. It’s very easy.
      - EnigmaFlare 2y ago
        Would you say that once a model is trained, there's no need to go back and re-train it, even if you want to, say, remove some material from the training set? Anything can be done just with the weights? That's a big surprise to me. Of course people hack binaries too, and binaries are obviously not source code. I once edited a book in PDF form because we didn't have the original Word/whatever document. It's not hard but a PDF still isn't considered to be source code for documentation despite that.

ML weights are code

jchw 2y ago

I'll accept that if you would like. However, they are not source code if so. They are object code. And open source is about source code, not object code. (And this particular press release isn't about ML weights anyways, at least unless I'm grossly misunderstanding; it is just a dataset. So even failing this, it still doesn't really make any sense.)

jncfhnb 2y ago
No it is not object code unless you want to get so stupidly pedantic that you want to argue a Python script in a zip file can’t be considered open source because it’s compressed. The model pickles unpack back to their original form. The picked binary forms are merely for convenience.
- jchw 2y ago
  Look, please go do research as to what "object code" and "source code" are before saying my argument is "stupidly pedantic". I'm not elaborating because the example you gave has nothing to do with what I said.
  - jncfhnb 2y ago
    Your analogy does not make sense. ML weights are distributed in binary form, like object code, but it is nothing like compiled binary. It’s just temporarily in binary form for convenience. It unpacks directly into its original form. This is not a technicality like “technically up can reverse engineer or modify binary code”. The binary form of model weights is just a fancy zip file format that is useful because they are so large that text is impractical.
    - jchw 2y ago
      Source code is human readable. Object code is not, and produced from some mechanical process. Model weights are not written by hand. You don't manually tweak individual weights. You have to run a training process that has multiple "raw" inputs. Trying to read model weights directly is no better than trying to read object code directly. Heck, reading object code directly is probably easier, because at least it's just machine code at the bottom; I will never be able to comprehend what's going on in an ML model just by reading the weights. The closest thing to "source code" in ML models would be the inputs to the training process, because that's the "source" of the model weights that pops out the other end. If the analogy doesn't make sense, that's because ML models are probably not really code in the same sense that source code and object code. (It may be tempting to look at "ML weights" as source code because of the existence of "closed-weight" API services. Please consider the following: If Amazon offers me a unique database service that I can only use with Amazon Web Services, and then releases a closed-source binary that you can run locally, that is still closed-source, because you don't have the source code.)
      - jncfhnb 2y ago
        “Human readable” is not a requirement. Visual programming code breaks down to some obtuse data structure. But with the right tools, it’s easy for humans to interact with it. Visual programming node workflows can be open sourced. ML models are the same. Tooling is required to interact with it. The limits of your human understanding do not determine if something is open source. Otherwise a really complicated traditional program might be argued as not open source. You can individually explore specific vectors and layers of a model and their significance. Produced by a non mechanistic process is not a requirement. I can generate a hello world script with code, and open source the hello world script. It does not matter how it was formed. I do not need to open source the hello world generator either. Data and training code is not source code of the model. That is the source code of a model maker. That’s `make_hello_world.py` not `hello_world.py` The closed source database is not a correct analogy. Excluding unreasonably difficult efforts to decompile the binary, you CANNOT modify the program without expecting it to break. With an ML model, the weights are the PREFERRED method of modifying the program. You do NOT want the original data and training code. That will just be a huge expense to get you what you already have. If you want the model to be different, you take the model weights and change them. Not recreate them differently from scratch. Which is the same for all traditional code. Open source does not mean I provide you with the design documents and testing feedback to demonstrate how the code base got created. It means you get the code base. Recreating the codebase is not something we think about because it doesn’t make sense because we have the code and we have the models.

shevis 2y ago

No, they really aren’t and I’m not sure why I keep seeing this take. ML weights are binary and it’s painfully obvious. They are the end result of a compilation process in which the training data and model code are compiled into the resulting weights. If you can’t even theoretically recreate the weights on your own hardware it isn’t open source.

jncfhnb 2y ago
ML weights are not binary. They are modifiable. If produce a program that outputs a hello world file, I can open source the hello world script without open sourcing the hello world generator.

deleted 2y ago

[deleted]

kennyadam 2y ago

Technically, but it feels like you're intentionally missing the point being made. Sure, providing the weights is very useful given the cost of generating them, but you can't exactly learn much by looking through the 'code', make changes and gain an in-depth understanding in the same way you can from the code provided by an actual open source project.

jncfhnb 2y ago
You absolutely can and people do all the time. There are mountains of forks and dissections and improvements on open source models.

i_read_news 2y ago

We can also say binaries are code, but if we are being pedantic that likely isn’t the source code that generated the binary (I also doubt the intention of hand writing binary or manually inputting billions of weights). I’d reckon that’s why it’s called open source, not open code or open binary, as the source code that generates the data is distributed. I’d actually just call this for what it is - open weights.

jncfhnb 2y ago
Binary is not the equivalent of models. Source code is the equivalent of models. It doesn’t matter if a machine generated source code or a human did for it to be open source code.
- EnigmaFlare 2y ago
  You keep asserting this but without any reason. Do you have a reason? It seems to go against the general open source idea of source code being convenient for people to modify.
  - jncfhnb 2y ago
    ML weights ARE convenient for people to modify. You can go look at the dozens of modifications of diffusion models being produced, daily, on civit ai. It’s very easy.
    - EnigmaFlare 2y ago
      Would you say that once a model is trained, there's no need to go back and re-train it, even if you want to, say, remove some material from the training set? Anything can be done just with the weights? That's a big surprise to me. Of course people hack binaries too, and binaries are obviously not source code. I once edited a book in PDF form because we didn't have the original Word/whatever document. It's not hard but a PDF still isn't considered to be source code for documentation despite that.