3 ms·

No the preferred way of making modifications is the weights _together_ with training (or fine tuning) scripts, and the entire evaluation pipeline to measure per

No the preferred way of making modifications is the weights _together_ with training (or fine tuning) scripts, and the entire evaluation pipeline to measure performance. And the data required to support all of this. When someone joins your data science team your would give them all this code and data. Not just the weights and say - the weights are the source, modify that to improve the model, I look forward to see your MR next week. EDIT: Heck, sometimes the way to make improvements (modifications) is just to improve the data, and not touch the training code at all. It is often one of the most powerful ways. You still need training code though, and evaluation to measure the impact.

NitpickLawyer 1y ago
The license gives you the right to modify the weights, how you do the modification is up to you. The rest is in the realm of IP, know-how, etc. Apples and oranges.
- imiric 1y ago
  Having the right to modify one part of the product is not the same as having the right to modify the entire product. Labeling such projects as open source in the full spirit of the definition is disingenuous. This is similar to the approach taken by some video game studios: release the source code under a permissive license, but not the game assets. Which is better than a proprietary license, but it still presents a hurdle for the final product to be built from source. The open weights approach is much more user hostile, however. Proprietary game assets can at least be purchased, and the final product can be built. With open weights, this is not possible. Nobody can realistically build the same model or similar models from weights alone. They can use the weights and self-host the prebuilt model, but not create revisions of it, which is the whole point of open source. Weights are essentially the bytecode of language models. Sure, you can run and modify it with the right tools, but without the tools used to create it in the first place, the project is not much more useful than publishing binaries.
wizzwizz4 1y ago
You also need the training data, so you can ensure you're not benchmarking on the training set, fine-tuning on the training set (overfitting with extra steps), or otherwise breaking things.
charcircuit 1y ago
It's not about the preferred way. Else open source software would need to give you their IDE setup, CI/CD setup, access to all internal tools, etc. Software like sqlite don't release their full test suite. They paywall the preferred way of making changes, yet they are open source. >The “source code” for a work means the preferred form of the work for making modifications The GPL refers to a form of the artifact being released
- lmm 1y ago
  > open source software would need to give you their IDE setup, CI/CD setup, access to all internal tools, etc. IMO they do. If you can't modify it like a core contributor would, then it's not really open source. Traditional open source projects always included development guides, test configurations etc. > Software like sqlite don't release their full test suite. They paywall the preferred way of making changes, yet they are open source. That's a matter of opinion. IMO sqlite is not true open source, for precisely this reason.

NitpickLawyer 1y ago

The license gives you the right to modify the weights, how you do the modification is up to you. The rest is in the realm of IP, know-how, etc. Apples and oranges.

imiric 1y ago
Having the right to modify one part of the product is not the same as having the right to modify the entire product. Labeling such projects as open source in the full spirit of the definition is disingenuous. This is similar to the approach taken by some video game studios: release the source code under a permissive license, but not the game assets. Which is better than a proprietary license, but it still presents a hurdle for the final product to be built from source. The open weights approach is much more user hostile, however. Proprietary game assets can at least be purchased, and the final product can be built. With open weights, this is not possible. Nobody can realistically build the same model or similar models from weights alone. They can use the weights and self-host the prebuilt model, but not create revisions of it, which is the whole point of open source. Weights are essentially the bytecode of language models. Sure, you can run and modify it with the right tools, but without the tools used to create it in the first place, the project is not much more useful than publishing binaries.

wizzwizz4 1y ago

You also need the training data, so you can ensure you're not benchmarking on the training set, fine-tuning on the training set (overfitting with extra steps), or otherwise breaking things.

charcircuit 1y ago

It's not about the preferred way. Else open source software would need to give you their IDE setup, CI/CD setup, access to all internal tools, etc. Software like sqlite don't release their full test suite. They paywall the preferred way of making changes, yet they are open source. >The “source code” for a work means the preferred form of the work for making modifications The GPL refers to a form of the artifact being released

lmm 1y ago
> open source software would need to give you their IDE setup, CI/CD setup, access to all internal tools, etc. IMO they do. If you can't modify it like a core contributor would, then it's not really open source. Traditional open source projects always included development guides, test configurations etc. > Software like sqlite don't release their full test suite. They paywall the preferred way of making changes, yet they are open source. That's a matter of opinion. IMO sqlite is not true open source, for precisely this reason.