4 ms·

The weights are not "source".

sokoloff 3y ago
That is likely true in the most pedantic sense, but in practice, if I create an algorithm that works by using a series of matrix transformations against a set of carefully chosen (read: "trained") matrices and I open-source only the matrix manipulation code but not the specially chosen matrices, I think there's a fair argument to say that I haven't open-sourced the entire algorithm. In the phrase "the model is open-source in every sense of the word", that, IMO, must include the weights.
- Name_Chawps 3y ago
  I'd be curious to know if you think open source companies should share all of their database records too.
  - sokoloff 3y ago
    I think of two broad categories of database records: transactional data (data created while running the system) and domain data (data created during development and shipped to production as part of the release process). The former type of data I wouldn't expect to ever be open-sourced. The latter type might or might not be, depending on the intent of open-sourcing the related system. If I created a human language translation system that used a SQL database to store the dictionary (domain) data and claimed the system was open-source without shipping the domain data, I think people would rightly say that the system was not fully opened.
deleted 3y ago
[deleted]
flangola7 3y ago
The actual source code is not important. The source code can be printed on a single A4 page, the valuable final product is the weights you get after running the code for fifty million dollars of compute time.
- Name_Chawps 3y ago
  If they start claiming they're "open weights" or "open final product" I'll be up in arms.
  - sokoloff 3y ago
    They are claiming that. They are also delivering on that claim: https://github.com/facebookresearch/segment-anything#model-checkpoints https://github.com/facebookresearch/segment-anything#model-c...
jtsiskin 3y ago
If I autogenerated a huge amount of C based on the weights, that added/multipled variables the same way the existing code+weights does, then would it be “source”?
dragonwriter 3y ago
The weights seem to be under the same license, just distributed separately because it doesn’t make sense for the giant binary artifacts of training to be part of the source repository.

That is likely true in the most pedantic sense, but in practice, if I create an algorithm that works by using a series of matrix transformations against a set of carefully chosen (read: "trained") matrices and I open-source only the matrix manipulation code but not the specially chosen matrices, I think there's a fair argument to say that I haven't open-sourced the entire algorithm. In the phrase "the model is open-source in every sense of the word", that, IMO, must include the weights.

Name_Chawps 3y ago
I'd be curious to know if you think open source companies should share all of their database records too.
- sokoloff 3y ago
  I think of two broad categories of database records: transactional data (data created while running the system) and domain data (data created during development and shipped to production as part of the release process). The former type of data I wouldn't expect to ever be open-sourced. The latter type might or might not be, depending on the intent of open-sourcing the related system. If I created a human language translation system that used a SQL database to store the dictionary (domain) data and claimed the system was open-source without shipping the domain data, I think people would rightly say that the system was not fully opened.

deleted 3y ago

[deleted]

flangola7 3y ago

The actual source code is not important. The source code can be printed on a single A4 page, the valuable final product is the weights you get after running the code for fifty million dollars of compute time.

Name_Chawps 3y ago
If they start claiming they're "open weights" or "open final product" I'll be up in arms.
- sokoloff 3y ago
  They are claiming that. They are also delivering on that claim: https://github.com/facebookresearch/segment-anything#model-checkpoints https://github.com/facebookresearch/segment-anything#model-c...

jtsiskin 3y ago

If I autogenerated a huge amount of C based on the weights, that added/multipled variables the same way the existing code+weights does, then would it be “source”?

dragonwriter 3y ago

The weights seem to be under the same license, just distributed separately because it doesn’t make sense for the giant binary artifacts of training to be part of the source repository.