3 ms·
While I agree, that the term "Open Source model" is misleading, an "open weight model" is much more than closed source executable. It is quite easy to modify th
by snowpid 3y ago
While I agree, that the term "Open Source model" is misleading, an "open weight model" is much more than closed source executable. It is quite easy to modify the model and it's possible to verify and test the model. It looks like "open weight models" are just a new paradigma in the world of CS.
- zingelshuher 3y agoIt's a matter of opinion how much open model should be to be called 'open source'. Looks like some believe they have the right to define it for everybody else to use. Like for software. Have to disagree. Why don't we introduce a separate term: 'open source, training infrastructure and data included for free'? "open weight model" is confusing, because actually the architecture is open too, only data is missing.
- snowpid 3y agothe point of traditional open source software is what you can reproduce your build. This is not possible with open weight models.
- zingelshuher 3y agoIt's a different animal. In general you cannot reproduce the model even having all the training data. There are too many random factors and nobody keeps track of them. Just pushing the training data is done at random from the dataset. This results in some interesting facts. Having the model and the data it's impossible to say if the model was trained on that exactly data. All we can say is that some pieces of that data were used in training, in some cases. Model can be 'watermarked' in hard to detect, stable to quantization and finetuning way. So, you cannot have a reproducible, 'open source' in its strict interpretation, model.