4 ms·

Please don't use the open-source term unless you ship the TBs of data downloaded from Anna's Archive that are required do build it yourself. And dont forget all

Please don't use the open-source term unless you ship the TBs of data downloaded from Anna's Archive that are required do build it yourself. And dont forget all the system prompts to censor the multiple topics that they don't want you to see.

deleted 1y ago
[deleted]
rvnx 1y ago
I don’t know why you got so much downvoted, these models are not open-source/open-recipes. They are censored open weights models. Better than nothing, but far from being Open
- a_vanderbilt 1y ago
  Most people don't really care all that much about the distinction. It comes across to them as linguistic pedantry and they downvote it to show they don't want to hear/read it.
outlore 1y ago
by your definition most of the current open weight models would not qualify
- layer8 1y ago
  That’s why they are called open weight and not open source.
- robotmaxtron 1y ago
  Correct. I agree with them, most of the open weight models are not open source.
someperson 1y ago
Keep fighting the "open weights" terminology fight, because diluting the term open-source for a blob of neural network weights (even inference code is open-source) is not open-source.
mhh__ 1y ago
The system prompt is an inference parameter, no?
Quarrel 1y ago
Is your point really that- "I need to see all data downloaded to make this model, before I can know it is open"? Do you have $XXB worth of GPU time to ingest that data with a state of the art framework to make a model? I don't. Even if I did, I'm not sure FB or Google are in any better position to claim this model is or isn't open beyond the fact that the weights are there. They're giving you a free model. You can evaluate it. You can sue them. But the weights are there. If you dislike the way they license the weights, because the license isn't open enough, then sure, speak up, but because you can't see all the training data??! Wtf.
- ticulatedspline 1y ago
  To many people there's an important distinction between "open source" and "open weights". I agree with the distinction, open source has a particular meaning which is not really here and misuse is worth calling out in order to prevent erosion of the terminology. Historically this would be like calling a free but closed-source application "open source" simply because the application is free.
- layer8 1y ago
  The parent’s point is that open weight is not the same as open source. Rough analogy: SaaS = AI as a service Locally executable closed-source software = open-weight model Open-source software = open-source model (whatever allows to reproduce the model from training data)
- NicuCalcea 1y ago
  I don't have the $XXbn to train a model, but I certainly would like to know what the training data consists of.
- seba_dos1 1y ago
  Do you need to see the source code used to compile this binary before you can know it is open? Do you have enough disk storage and RAM available to compile Chromium on your laptop? I don't.
- nexttk 1y ago
  I agree with OP - the weights are more akin to the binary output from a compiler. You can't see how it works, how it was made, you can't freely manipulate with it, improve it, extend it etc. It's like having a binary of a program. The source code for the model was the training data. The compiler is the tooling that can train a module based on a given set of training data. For me it is not critical for an open source model that it is ONLY distributed in source code form. It is fine that you can also download just the weights. But it should be possible to reproduce the weights - either there should be a tar.gz ball with all the training data, or there needs to be a description/scripts of how one could obtain the training data. It must be reproducible for someone willing to invest the time, compute into it even if 99.999% use only the binary. This is completely analogous to what is normally understood by open source.
NitpickLawyer 1y ago
It's apache2.0, so by definition it's open source. Stop pushing for training data, it'll never happen, and there's literally 0 reason for it to happen (both theoretical and practical). Apache2.0 IS opensource.
- organsnyder 1y ago
  What is the source that's open? Aren't the models themselves more akin to compiled code than to source code?
  - NitpickLawyer 1y ago
    No, not compiled code. Weights are hardcoded values. Code is the combination of model architecture + config + inferencing engine. You run inference based on the architecture (what and when to compute), using some hardcoded values (weights).
    - seba_dos1 1y ago
      JVM bytecode is hardcoded values. Code is the virtual machine implementation + config + operating system it runs on. You run classes based on the virtual machine, using some hardcoded input data generated by javac.
- _flux 1y ago
  No, it's open weight. You wouldn't call applications with only Apache 2.0-licensed binaries "open source". The weights are not the "source code" of the model, they are the "compiled" binary, therefore they are not open source. However, for the sake of argument let's say this release should be called open source. Then what do you call a model that also comes with its training material and tools to reproduce the model? Is it also called open source, and there is no material difference between those two releases? Or perhaps those two different terms should be used for those two different kind of releases? If you say that actually open source releases are impossible now (for mostly copyright reasons I imagine), it doesn't mean that they will be perpetually so. For that glorious future, we can leave them space in the terminology by using the term open weight. It is also the term that should not be misleading to anyone.

deleted 1y ago

[deleted]

rvnx 1y ago

I don’t know why you got so much downvoted, these models are not open-source/open-recipes. They are censored open weights models. Better than nothing, but far from being Open

a_vanderbilt 1y ago
Most people don't really care all that much about the distinction. It comes across to them as linguistic pedantry and they downvote it to show they don't want to hear/read it.

outlore 1y ago

by your definition most of the current open weight models would not qualify

layer8 1y ago
That’s why they are called open weight and not open source.
robotmaxtron 1y ago
Correct. I agree with them, most of the open weight models are not open source.

someperson 1y ago

Keep fighting the "open weights" terminology fight, because diluting the term open-source for a blob of neural network weights (even inference code is open-source) is not open-source.

mhh__ 1y ago

The system prompt is an inference parameter, no?

Quarrel 1y ago

Is your point really that- "I need to see all data downloaded to make this model, before I can know it is open"? Do you have $XXB worth of GPU time to ingest that data with a state of the art framework to make a model? I don't. Even if I did, I'm not sure FB or Google are in any better position to claim this model is or isn't open beyond the fact that the weights are there. They're giving you a free model. You can evaluate it. You can sue them. But the weights are there. If you dislike the way they license the weights, because the license isn't open enough, then sure, speak up, but because you can't see all the training data??! Wtf.

ticulatedspline 1y ago
To many people there's an important distinction between "open source" and "open weights". I agree with the distinction, open source has a particular meaning which is not really here and misuse is worth calling out in order to prevent erosion of the terminology. Historically this would be like calling a free but closed-source application "open source" simply because the application is free.
layer8 1y ago
The parent’s point is that open weight is not the same as open source. Rough analogy: SaaS = AI as a service Locally executable closed-source software = open-weight model Open-source software = open-source model (whatever allows to reproduce the model from training data)
NicuCalcea 1y ago
I don't have the $XXbn to train a model, but I certainly would like to know what the training data consists of.
seba_dos1 1y ago
Do you need to see the source code used to compile this binary before you can know it is open? Do you have enough disk storage and RAM available to compile Chromium on your laptop? I don't.
nexttk 1y ago
I agree with OP - the weights are more akin to the binary output from a compiler. You can't see how it works, how it was made, you can't freely manipulate with it, improve it, extend it etc. It's like having a binary of a program. The source code for the model was the training data. The compiler is the tooling that can train a module based on a given set of training data. For me it is not critical for an open source model that it is ONLY distributed in source code form. It is fine that you can also download just the weights. But it should be possible to reproduce the weights - either there should be a tar.gz ball with all the training data, or there needs to be a description/scripts of how one could obtain the training data. It must be reproducible for someone willing to invest the time, compute into it even if 99.999% use only the binary. This is completely analogous to what is normally understood by open source.

NitpickLawyer 1y ago

It's apache2.0, so by definition it's open source. Stop pushing for training data, it'll never happen, and there's literally 0 reason for it to happen (both theoretical and practical). Apache2.0 IS opensource.

organsnyder 1y ago
What is the source that's open? Aren't the models themselves more akin to compiled code than to source code?
- NitpickLawyer 1y ago
  No, not compiled code. Weights are hardcoded values. Code is the combination of model architecture + config + inferencing engine. You run inference based on the architecture (what and when to compute), using some hardcoded values (weights).
  - seba_dos1 1y ago
    JVM bytecode is hardcoded values. Code is the virtual machine implementation + config + operating system it runs on. You run classes based on the virtual machine, using some hardcoded input data generated by javac.
_flux 1y ago
No, it's open weight. You wouldn't call applications with only Apache 2.0-licensed binaries "open source". The weights are not the "source code" of the model, they are the "compiled" binary, therefore they are not open source. However, for the sake of argument let's say this release should be called open source. Then what do you call a model that also comes with its training material and tools to reproduce the model? Is it also called open source, and there is no material difference between those two releases? Or perhaps those two different terms should be used for those two different kind of releases? If you say that actually open source releases are impossible now (for mostly copyright reasons I imagine), it doesn't mean that they will be perpetually so. For that glorious future, we can leave them space in the terminology by using the term open weight. It is also the term that should not be misleading to anyone.