3 ms·
'open source' isn't even a meaningful word for LLMs, since they don't have source code in the traditional sense.
by Legend2440 1y ago
'open source' isn't even a meaningful word for LLMs, since they don't have source code in the traditional sense.
- vrighter 1y agoyes they do. training code, dataset, and random seed. given all three, you can recreate the final model (the weights). The set of all three is the source code
- lavoiems 1y agoThat and random race conditions occurring during training.
- herbst 1y agoIf I could theoretically fully rebuild it based on the given data I would call it open source. See the upcoming ETH model, it should be completely reproduceable