3 ms·
People don’t typically modify distributed binaries. People do typically modify model weights. They are the preferred form to modify model. Saying “build” llam
by jncfhnb 2y ago
People don’t typically modify distributed binaries.
People do typically modify model weights. They are the preferred form to modify model.
Saying “build” llama is just a nonsense comparison to traditional compiled software. “Building llama” is more akin to taking the raw weights as text and putting them into a nice pickle file. Or loading it into an inference engine.
Demanding that you have everything needed to recreate the weights from scratch is like arguing an application cannot be open source unless it also includes the user testing history and design documents.
And of course some idiots don’t understand what a pickled weights file is and claim it’s as useless as a distributed binary if you want to modify the program just because it is technically compiled; not understanding that the point of the pickled file is “convenience” and that it unpacks back to the original form. Like arguing open source software can’t be distributed in zip files.
> Say I have a farm of 100 A100's, could I reproduce the Llama model from scratch today?
Say you have a piece of paper. Can you reproduce `print(“hello world”)` from scratch?