4 ms·
Reproducibility becomes an important criterion for models though. For normal programs, it is quite easy to decompile an unoptimized binary. Even decompiling an
by samus 3y ago
Reproducibility becomes an important criterion for models though.
For normal programs, it is quite easy to decompile an unoptimized binary. Even decompiling an optimized will lead to source code. To make this harder, an obfuscator has to be used.
A model is different because it relies on its weight, which are quite a bit more difficult to inspect. Way harder than even obfuscated source code. It is magnitudes harder to make statements about which information it might divulge upon careful questioning, or evaluate its biases, if the training data is not available.
- andy99 3y agoEven if you have the data you can't do that stuff any better. The makeup of the training set doesn't really define the behavior in any tractable way. If anything I think it's a distraction and even when it is available people probably pay too much attention to what's in the training data vs actual behavior. Edit to say that I see benefits to having the training data, just that I don't think it's needed to exercise enough freedom to qualify as open source in an analogous way to software. Also to add, training on GPUs is not generally reproducible anyway because of execution order.