7 ms·
> (As an aside, this is why the "open weights are not open source" thing is a complete misunderstanding. The weights themselves along with the documentation giv
by eru 3mo ago
> (As an aside, this is why the "open weights are not open source" thing is a complete misunderstanding. The weights themselves along with the documentation give you enough to fine tune the LLM. You can't rebuild it from scratch, but you can't do this even with the data anyway (because of randomness!))
They could give you the random seeds? (Assuming you carefully train in such a way to remove other sources of randomness, like concurrent execution.)
- alightsoul 3mo agoDo AI labs even keep track of seeds? They don't even keep the checkpoints most of the time.
- eru 3mo agoIt's moot in practice, because there's additional sources of randomness from executing in parallel. Though in principle saving random seeds is a lot less hassle than keeping entire checkpoints around: your random seeds would fit on a floppy disk or even a tweet. The checkpoint is basically as big as the model.