4 ms·

A lot of people get hung up on `weights = compiled-artifact` because both are binary representations, but there are major limitations to this comparison. When

A lot of people get hung up on `weights = compiled-artifact` because both are binary representations, but there are major limitations to this comparison. When we're dealing with source code, the cost of getting from source -> binary is minimal. The entire Linux kernel builds in two hours on one modest machine. Since it's cheap to compile and the source code is itself legible, the source code is the preferred form for making modifications. This doesn't work when we try to apply the same reasoning to `training data -> weights`. "Compilation" in this world costs hundreds of millions of dollars per compilation run. Cost of "compilation" alone means that the preferred form for making modifications can't possibly be the training data, even for the company that built the thing in the first place. As for the data itself, it's a far cry from source code—we're talking tens of terrabytes of data at a minimum, which is likewise infeasible to work with on a regular basis. The weights must be the preferred form for making modifications for simple logistics reasons. Importantly, the weights are the preferred form for modifications even for the companies that built them. I think a far more reasonable analogy, to the extent that any are reasonable, is that the training data is all the stuff that the developers of the FOSS software ever learned, and the thousands of computer-hours spent on training are the thousands of man-hours spent coding. The entire point of FOSS is for a few experts to do all that work once and then we all can share and modify the output of those years of work and millions of dollars invested as we see fit, without having to waste all that time and money doing it over again. We don't expect the authors of the Linux kernel to document their every waking thought so we could recreate the process they used to produce the kernel code... we just thank them for the kernel code and contribute to it as best we can.

smolder 2y ago
If you can't bootstrap a set of weights from public material, the weights aren't open source, because it's derivative content based on something non-open. Trying to draw an equivalency between code and weights is [edited for temperament, I guess] not right. They are built from the source material supplied to an algorithm. Weights are data, not code. Otherwise, everyone on the internet would be an author, and would have a say in the licensing of the weights.
- lolinder 2y ago
  > Trying to draw an equivalency between code and weights is ridiculous, because the weights are not written in the same way as source code. By the same logic, the comparison between a compiled artifact and weights fails because the weights are not "compiled" in any meaningful sense. Analogies will always fail, which is why "preferred form for making modifications" is the rod we use, not vague attempts at drawing analogies between completely different development processes. > They are built from the source material supplied to an algorithm. Weights are data, not code. As Lispers know well, code is data and data is code. You can't draw a line in the sand and definitively say that on this side of the line is just code and on that side is just data. In terms of how they behave, weights function as code that is executed by an interpreter that we call an inference engine.
  - dietr1ch 2y ago
    > > Trying to draw an equivalency between code and weights is ridiculous, because the weights are not written in the same way as source code. > By the same logic, the comparison between a compiled artifact and weights fails because the weights are not "compiled" in any meaningful sense. To me the weights map to assembly and the training data+models map to source code+compilers. Sure, you can hand me assembly, and with the assembly I may be able to execute the model/program, but having it does not mean that I can stare at it and learn nor modify it with a reasonable understanding of what's going to change. I got to add that the situation feels even worse than assembly, because assembly, hand-coded or mutilated by an optimizing compiler still does something very specific and deterministic, but the weights on the model makes things equivalent to programming without booleans, but seemingly random numbers and checking for inequalities to get a binary decision.
    - lolinder 2y ago
      This is the analogy that people keep saying, but as I observed above, the key difference is that the company that produces a binary executable doesn't prefer to work with that binary: they work with the source code and re-compile after changing it. In contrast, the weights are the preferred form for modification, even for the company that built it. They only very rarely start a brand new training run from scratch, and when they do so it's not to modify the existing work, it's to make a brand new work that builds on what they learned from the previous model. If the company makes the form of the work that they themselves use as the primary artifact freely available, I'm not sure why we wouldn't call the work open.
kvemkon 2y ago
> The entire Linux kernel builds in two hours on one modest machine. I think it is better to compare with something really big and fast evolving, e.g. Chromium. It will take a day to compile it. (~80000 seconds vs. ~8 seconds for a convenient/old Pascal program.)
- lolinder 2y ago
  Even if we take Chromium, compiling it is still within the logistical capabilities of any individual with a modest computer, and it's still more reasonable to spend a day waiting for the compilation then to try to modify the binary directly. The same cannot be said for LLM weights, as evidenced by the fact that even the enormous megacorps that put these things out tend to follow up by fine tuning the weights (using different training data) rather than retraining from scratch.
  - kvemkon 2y ago
    I'd continue to compare it with how free software appeared. It was barely feasible back then (end of 60th - begin of 70th) to work on software at home on a PC, since there was no PC. Still the software could already be open/free and exchanged between big enough players with big computers. Only later the things changed. But even now there is free software which is nightmare to program on a usual single PC. Thus it is either too early to define "open" for AI. Or "open" must be truly open. Though it remains not practically achievable at home or even at small companies.
SOLAR_FIELDS 2y ago
Is it sufficient to say something is open if it can be reproduced with no external dependencies? If it costs X gazillion dollars to reproduce it, that feels irrelevant to some extent. If it is not reproducible, then it is not open. If it is reproducible, then it is open. Probably there’s some argument to be made here that it’s not actually open if some random dev can’t reproduce it on their own machine over a weekend, but I honestly don’t buy that argument in this realm.
- lolinder 2y ago
  > If it is not reproducible, then it is not open. If it is reproducible, then it is open. You're applying reproducibility unevenly, though. The Linux kernel source code cannot feasibly be reproduced, but it can be copied and modified. The Mistral weights cannot feasibly be reproduced, but they can be copied and modified. Why is the kernel code open source while the Mistral weights are not? Reproducibility is clearly not the deciding factor.
  - fragmede 2y ago
    The Linux kernel is considered Open Source because (among other things) the compiled kernel binary that is used to boot a computer can be reproduced from provided source code. source code -> compile -> kernel binary. That binary is what can be reproduced, given the source code. We don't have the equivalent for Mistral: source code (+ training data) -> training -> weights
    - lolinder 2y ago
      So people have said, but as I've noted I disagree with the characterization that training is equivalent to compilation. Even the companies that can afford to train a foundation model do so once and then fine-tune it from there to modify it. They only start a new training run when they're building a brand new model with totally different characteristics (such as a different parameter count). Training is too expensive for the training data to be the preferred form for making modifications to the work. Given that, the weights themselves are the closest thing these things have to "source code". And this is where the reproducibility argument falls apart: on what basis can we insist that the preferred form for modifying an LLM (the weights) must be reproducible to be open source but the preferred form for modifying a piece of regular software (the code) can be open sourced as is, with none of the processes used to produce the code?
dahart 2y ago
> The entire point of FOSS is for a few experts to do all that work once and then we all can share and modify the output of those years of work and millions of dollars invested as we see fit, without having to waste all that time and money doing it over again. Really? Hmm yeah maybe you’re right, but for some reason, said that way it somehow starts to seem a little disappointing and depressing. Maybe I’m reading it differently than you intended. I always considered the point of FOSS to be about the freedoms to use, study, customize, and share software, like to become an expert, not to avoid becoming an expert. But if the ultimate goal of all that is just a big global application of DRY so that most people rely on the corpus without having to learn as much, I feel like that is in a way antithetical to open source, and it could have a big downside or might end up being a net negative, but I dunno…
raverbashing 2y ago
Completely agree There's a much simpler analogy: a photo You can't have an "Open source photo" because that would require shipping everything (but the camera) that shows up in the photo so that someone could "recreate" the photo It doesn't make sense. A public domain photo is enough
dragonwriter 2y ago
> A lot of people get hung up on `weights = compiled-artifact` because both are binary representations, No, that's not why weights are object code. Binary vs. text is irrelevant. Weights are object code because training data is declarative source code defining the desired behavior of the system and training code is a compiler which takes that source code and produces a system with the desired behavior. Now, the behavior produced is less exactly known from the source code than is the case with traditional programming, but the function is the same. You could have a system where the training and inference codes were open source and the model specified by the weights itself was not — that would be like having a system where software was not open source, but the compiler use to build it and the runtime library it relies on were. But one shouldn't confuse that with an open source model.
- lolinder 2y ago
  This is a much more compelling explanation than any I've seen so far. What do you do with the fact that no one (including the companies who do the initial training) modifies the training data when they want to modify the work? Are the weights not the preferred form for modifying a model?
  - seba_dos1 2y ago
    Compiling Android takes many hours and requires a lot of RAM and disk space, more than I have available on my machines right now. For some things there, binary patching may actually be an easier way to change stuff than compiling the whole thing from source. Now scale that up and consider - at which point such project would start being "FOSS" in your book without actually providing its sources on an appropriate license? The intention behind "preferred form for modification" is to put you as a user on the same level as the copyright holder. This construct works well in a world where compiling is cheap; where it isn't, it may require some refinement to preserve the intention behind it. The copyright holder could decide to modify the learning set before pressing the "start" button, you can't.
    - lolinder 2y ago
Shamar 2y ago
Compiling Emacs on a PDP-11 wasn't cheap either. And by today standards, a PDP-11 is quite comparable with the cost of the server farms used in training. And yet Emacs was released under GPL. So the economic argument is pretty miopic.
- lolinder 2y ago
  People still chose to compile emacs from scratch rather than modify the binary. The source code was the preferred form for modifications. The same is not true of these models. To my knowledge no company has retrained a model from scratch to make a modification to it. They make new models, but these are fundamentally different works with different parameter counts and architectures. When they want to improve on a model that they already built, they fine tune the weights. If that's what companies that own all the IP do, that tells me that the weights themselves are the preferred form for making modifications, which makes them source code under the gpl's definition.
  - Shamar 2y ago
    The problem with hype around LLM is that people without much experience in the field can't think of anything else. So much they forget the basics of the discipline. What do you think cross validation is for? To compare different weights obtained from different initializations, different topologies, different hyper-parameters... all trained from the same training dataset. Even for LLM, have you ever tried to reduce the size of the vocabulary of, say, Llama? No? Yet it's a totally reasonable modification. What's the preferred form to make modifications like this? Can you do it fine tuning llama weights? No. You need training data. That's why training data are the preferred form to make modification, because whatever the AI (hyped or not) it's the only form that let you make all modifications you want.
seba_dos1 2y ago
> The entire point of FOSS is for a few experts to do all that work once and then we all can share and modify the output of those years of work and millions of dollars invested as we see fit, without having to waste all that time and money doing it over again. The entire point of FOSS is to preserve user freedom. Avoiding pointless waste of repeated work is a side effect of applying that freedom. It would feel entirely on point for things that require ungodly amounts of money and resources to even start considering exercising your freedoms on to not be considered FOSS, even if that aspect isn't considered by currently accepted definitions.
- lolinder 2y ago
  Nit: what you're describing is Free Software as put forward by the FSF, not Open Source as put forward by the OSI. I realize I'm the one who used the combo acronym first, but this is a discussion about the OSI, which exists to champion the cynical company-centric version of the movement, and for that version my description is accurate.
  - pxc 2y ago
    It is sad that this definition will represent the first practical split between these two movements in terms of their terminology, if it goes through. The difference between open-source and free software will go from a difference of emphasis to a difference in substance. I suppose I will have to stop writing 'F/OSS'. I'll probably use the term 'open-source' less and less, and maybe stop altogether.
evoke4908 2y ago
> I think a far more reasonable analogy, to the extent that any are reasonable, is that the training data is all the stuff that the developers of the FOSS software ever learned, and the thousands of computer-hours spent on training are the thousands of man-hours spent coding. I think this is a decent point. Is your FOSS project actually open source if your 3D assets were made in Fusion or Adobe? Similarly, how open is a hardware project if you post only finalized STLs? What about with and without Fusion source files? You can still replicate the project. You can even do relatively minor edits to the STL. Is that open or not?
advael 2y ago
Lots of people like that analogy because it's extremely self-congratulatory. It also makes no fucking sense Despite the fact that people keep insisting on the buzzword "AI" to describe these large neural networks, they are more succinctly defined as approximate computer programs. The means by which we create them is through a relatively standardized family of statistical modeling algorithms paired with a dataset they are meant to emulate in their output A computer program that's specified in logic is already a usable representation that can be used to understand every aspect of the functioning code in its entirety, albeit some of it may be hard to understand. You don't need to consult the original programmer at all, let alone read their mind In contrast, a function that is approximated in the manner described needs the training data to replicate or make sense of it, and in fact is even necessary to assess whether the model is cheating at the benchmarks its creators assess it against. The weights themselves are a functional approximation, not a functional description For the purposes of the ethos of free and open source software, it is obvious that training data must be included. However, this argument is also deployed in various other places, like intellectual property disputes, and is equally stupid there. Just because we use the term "learning" to describe these systems doesn't mean it makes sense for the law to treat them as people. It is both nonsensical and harmful to say that no human can be held responsible for what an "AI" model does, but that somehow they are "just like people learning from experience" when it benefits tech companies to believe that
didibus 2y ago
The source is not open, so not open source. It's as simple as that to me. That doesn't mean you can't allow the modification of your weights, but a model is not open source because it lets you modify its weights. If we take the JDK for Java. It is open source, but the actually built JDKs of it are not all free to use or modify. It's quite annoying to build those, patch in source from newer ones into the builds of older ones, cherry pick and all that. So it enables an economy of vendors that do just that, and people willing to pay for the simple act of building the JDK from its open source. I'm not even sure it makes sense to license weights, they're not part of any creative process, I don't think weights should even be copyrightable. Weights are part of the product you sell, maybe a EULA applies, terms of use. It's like with video games, you're not always allowed to modify the binary (cheating), and if you do, you break the EULA and are revoked the right to play the game.

smolder 2y ago

If you can't bootstrap a set of weights from public material, the weights aren't open source, because it's derivative content based on something non-open. Trying to draw an equivalency between code and weights is [edited for temperament, I guess] not right. They are built from the source material supplied to an algorithm. Weights are data, not code. Otherwise, everyone on the internet would be an author, and would have a say in the licensing of the weights.

lolinder 2y ago
> Trying to draw an equivalency between code and weights is ridiculous, because the weights are not written in the same way as source code. By the same logic, the comparison between a compiled artifact and weights fails because the weights are not "compiled" in any meaningful sense. Analogies will always fail, which is why "preferred form for making modifications" is the rod we use, not vague attempts at drawing analogies between completely different development processes. > They are built from the source material supplied to an algorithm. Weights are data, not code. As Lispers know well, code is data and data is code. You can't draw a line in the sand and definitively say that on this side of the line is just code and on that side is just data. In terms of how they behave, weights function as code that is executed by an interpreter that we call an inference engine.
- dietr1ch 2y ago
  > > Trying to draw an equivalency between code and weights is ridiculous, because the weights are not written in the same way as source code. > By the same logic, the comparison between a compiled artifact and weights fails because the weights are not "compiled" in any meaningful sense. To me the weights map to assembly and the training data+models map to source code+compilers. Sure, you can hand me assembly, and with the assembly I may be able to execute the model/program, but having it does not mean that I can stare at it and learn nor modify it with a reasonable understanding of what's going to change. I got to add that the situation feels even worse than assembly, because assembly, hand-coded or mutilated by an optimizing compiler still does something very specific and deterministic, but the weights on the model makes things equivalent to programming without booleans, but seemingly random numbers and checking for inequalities to get a binary decision.
  - lolinder 2y ago
    This is the analogy that people keep saying, but as I observed above, the key difference is that the company that produces a binary executable doesn't prefer to work with that binary: they work with the source code and re-compile after changing it. In contrast, the weights are the preferred form for modification, even for the company that built it. They only very rarely start a brand new training run from scratch, and when they do so it's not to modify the existing work, it's to make a brand new work that builds on what they learned from the previous model. If the company makes the form of the work that they themselves use as the primary artifact freely available, I'm not sure why we wouldn't call the work open.

kvemkon 2y ago

> The entire Linux kernel builds in two hours on one modest machine. I think it is better to compare with something really big and fast evolving, e.g. Chromium. It will take a day to compile it. (~80000 seconds vs. ~8 seconds for a convenient/old Pascal program.)

lolinder 2y ago
Even if we take Chromium, compiling it is still within the logistical capabilities of any individual with a modest computer, and it's still more reasonable to spend a day waiting for the compilation then to try to modify the binary directly. The same cannot be said for LLM weights, as evidenced by the fact that even the enormous megacorps that put these things out tend to follow up by fine tuning the weights (using different training data) rather than retraining from scratch.
- kvemkon 2y ago
  I'd continue to compare it with how free software appeared. It was barely feasible back then (end of 60th - begin of 70th) to work on software at home on a PC, since there was no PC. Still the software could already be open/free and exchanged between big enough players with big computers. Only later the things changed. But even now there is free software which is nightmare to program on a usual single PC. Thus it is either too early to define "open" for AI. Or "open" must be truly open. Though it remains not practically achievable at home or even at small companies.

SOLAR_FIELDS 2y ago

Is it sufficient to say something is open if it can be reproduced with no external dependencies? If it costs X gazillion dollars to reproduce it, that feels irrelevant to some extent. If it is not reproducible, then it is not open. If it is reproducible, then it is open. Probably there’s some argument to be made here that it’s not actually open if some random dev can’t reproduce it on their own machine over a weekend, but I honestly don’t buy that argument in this realm.

lolinder 2y ago
> If it is not reproducible, then it is not open. If it is reproducible, then it is open. You're applying reproducibility unevenly, though. The Linux kernel source code cannot feasibly be reproduced, but it can be copied and modified. The Mistral weights cannot feasibly be reproduced, but they can be copied and modified. Why is the kernel code open source while the Mistral weights are not? Reproducibility is clearly not the deciding factor.
- fragmede 2y ago
  The Linux kernel is considered Open Source because (among other things) the compiled kernel binary that is used to boot a computer can be reproduced from provided source code. source code -> compile -> kernel binary. That binary is what can be reproduced, given the source code. We don't have the equivalent for Mistral: source code (+ training data) -> training -> weights
  - lolinder 2y ago
    So people have said, but as I've noted I disagree with the characterization that training is equivalent to compilation. Even the companies that can afford to train a foundation model do so once and then fine-tune it from there to modify it. They only start a new training run when they're building a brand new model with totally different characteristics (such as a different parameter count). Training is too expensive for the training data to be the preferred form for making modifications to the work. Given that, the weights themselves are the closest thing these things have to "source code". And this is where the reproducibility argument falls apart: on what basis can we insist that the preferred form for modifying an LLM (the weights) must be reproducible to be open source but the preferred form for modifying a piece of regular software (the code) can be open sourced as is, with none of the processes used to produce the code?

dahart 2y ago

> The entire point of FOSS is for a few experts to do all that work once and then we all can share and modify the output of those years of work and millions of dollars invested as we see fit, without having to waste all that time and money doing it over again. Really? Hmm yeah maybe you’re right, but for some reason, said that way it somehow starts to seem a little disappointing and depressing. Maybe I’m reading it differently than you intended. I always considered the point of FOSS to be about the freedoms to use, study, customize, and share software, like to become an expert, not to avoid becoming an expert. But if the ultimate goal of all that is just a big global application of DRY so that most people rely on the corpus without having to learn as much, I feel like that is in a way antithetical to open source, and it could have a big downside or might end up being a net negative, but I dunno…

raverbashing 2y ago

Completely agree There's a much simpler analogy: a photo You can't have an "Open source photo" because that would require shipping everything (but the camera) that shows up in the photo so that someone could "recreate" the photo It doesn't make sense. A public domain photo is enough

dragonwriter 2y ago

> A lot of people get hung up on `weights = compiled-artifact` because both are binary representations, No, that's not why weights are object code. Binary vs. text is irrelevant. Weights are object code because training data is declarative source code defining the desired behavior of the system and training code is a compiler which takes that source code and produces a system with the desired behavior. Now, the behavior produced is less exactly known from the source code than is the case with traditional programming, but the function is the same. You could have a system where the training and inference codes were open source and the model specified by the weights itself was not — that would be like having a system where software was not open source, but the compiler use to build it and the runtime library it relies on were. But one shouldn't confuse that with an open source model.

lolinder 2y ago
This is a much more compelling explanation than any I've seen so far. What do you do with the fact that no one (including the companies who do the initial training) modifies the training data when they want to modify the work? Are the weights not the preferred form for modifying a model?
- seba_dos1 2y ago
  Compiling Android takes many hours and requires a lot of RAM and disk space, more than I have available on my machines right now. For some things there, binary patching may actually be an easier way to change stuff than compiling the whole thing from source. Now scale that up and consider - at which point such project would start being "FOSS" in your book without actually providing its sources on an appropriate license? The intention behind "preferred form for modification" is to put you as a user on the same level as the copyright holder. This construct works well in a world where compiling is cheap; where it isn't, it may require some refinement to preserve the intention behind it. The copyright holder could decide to modify the learning set before pressing the "start" button, you can't.
  - lolinder 2y ago

Shamar 2y ago

Compiling Emacs on a PDP-11 wasn't cheap either. And by today standards, a PDP-11 is quite comparable with the cost of the server farms used in training. And yet Emacs was released under GPL. So the economic argument is pretty miopic.

lolinder 2y ago
People still chose to compile emacs from scratch rather than modify the binary. The source code was the preferred form for modifications. The same is not true of these models. To my knowledge no company has retrained a model from scratch to make a modification to it. They make new models, but these are fundamentally different works with different parameter counts and architectures. When they want to improve on a model that they already built, they fine tune the weights. If that's what companies that own all the IP do, that tells me that the weights themselves are the preferred form for making modifications, which makes them source code under the gpl's definition.
- Shamar 2y ago
  The problem with hype around LLM is that people without much experience in the field can't think of anything else. So much they forget the basics of the discipline. What do you think cross validation is for? To compare different weights obtained from different initializations, different topologies, different hyper-parameters... all trained from the same training dataset. Even for LLM, have you ever tried to reduce the size of the vocabulary of, say, Llama? No? Yet it's a totally reasonable modification. What's the preferred form to make modifications like this? Can you do it fine tuning llama weights? No. You need training data. That's why training data are the preferred form to make modification, because whatever the AI (hyped or not) it's the only form that let you make all modifications you want.

seba_dos1 2y ago

> The entire point of FOSS is for a few experts to do all that work once and then we all can share and modify the output of those years of work and millions of dollars invested as we see fit, without having to waste all that time and money doing it over again. The entire point of FOSS is to preserve user freedom. Avoiding pointless waste of repeated work is a side effect of applying that freedom. It would feel entirely on point for things that require ungodly amounts of money and resources to even start considering exercising your freedoms on to not be considered FOSS, even if that aspect isn't considered by currently accepted definitions.

lolinder 2y ago
Nit: what you're describing is Free Software as put forward by the FSF, not Open Source as put forward by the OSI. I realize I'm the one who used the combo acronym first, but this is a discussion about the OSI, which exists to champion the cynical company-centric version of the movement, and for that version my description is accurate.
- pxc 2y ago
  It is sad that this definition will represent the first practical split between these two movements in terms of their terminology, if it goes through. The difference between open-source and free software will go from a difference of emphasis to a difference in substance. I suppose I will have to stop writing 'F/OSS'. I'll probably use the term 'open-source' less and less, and maybe stop altogether.

evoke4908 2y ago

> I think a far more reasonable analogy, to the extent that any are reasonable, is that the training data is all the stuff that the developers of the FOSS software ever learned, and the thousands of computer-hours spent on training are the thousands of man-hours spent coding. I think this is a decent point. Is your FOSS project actually open source if your 3D assets were made in Fusion or Adobe? Similarly, how open is a hardware project if you post only finalized STLs? What about with and without Fusion source files? You can still replicate the project. You can even do relatively minor edits to the STL. Is that open or not?

advael 2y ago

Lots of people like that analogy because it's extremely self-congratulatory. It also makes no fucking sense Despite the fact that people keep insisting on the buzzword "AI" to describe these large neural networks, they are more succinctly defined as approximate computer programs. The means by which we create them is through a relatively standardized family of statistical modeling algorithms paired with a dataset they are meant to emulate in their output A computer program that's specified in logic is already a usable representation that can be used to understand every aspect of the functioning code in its entirety, albeit some of it may be hard to understand. You don't need to consult the original programmer at all, let alone read their mind In contrast, a function that is approximated in the manner described needs the training data to replicate or make sense of it, and in fact is even necessary to assess whether the model is cheating at the benchmarks its creators assess it against. The weights themselves are a functional approximation, not a functional description For the purposes of the ethos of free and open source software, it is obvious that training data must be included. However, this argument is also deployed in various other places, like intellectual property disputes, and is equally stupid there. Just because we use the term "learning" to describe these systems doesn't mean it makes sense for the law to treat them as people. It is both nonsensical and harmful to say that no human can be held responsible for what an "AI" model does, but that somehow they are "just like people learning from experience" when it benefits tech companies to believe that

didibus 2y ago

The source is not open, so not open source. It's as simple as that to me. That doesn't mean you can't allow the modification of your weights, but a model is not open source because it lets you modify its weights. If we take the JDK for Java. It is open source, but the actually built JDKs of it are not all free to use or modify. It's quite annoying to build those, patch in source from newer ones into the builds of older ones, cherry pick and all that. So it enables an economy of vendors that do just that, and people willing to pay for the simple act of building the JDK from its open source. I'm not even sure it makes sense to license weights, they're not part of any creative process, I don't think weights should even be copyrightable. Weights are part of the product you sell, maybe a EULA applies, terms of use. It's like with video games, you're not always allowed to modify the binary (cheating), and if you do, you break the EULA and are revoked the right to play the game.