3 ms·
You want to see the source code of the transformer that created the compiled llm model, the dictionary corresponding to token values, etc. Instead, you got the
by Dowwie 3y ago
You want to see the source code of the transformer that created the compiled llm model, the dictionary corresponding to token values, etc. Instead, you got the weights and binary.
I don't know what's going on in this discussion thread, as to whether you are being gaslit or engaging with people who shouldn't comment.
- uoaei 3y agoRight, my interpretation (and I think monocasa's too) is that the training and packaging pipeline is not published therefore the code used to build the model is not open, therefore there is no source available to rebuild the model as Mistral's engineers have done. In essence the output of the training pipeline is "data" (weights), not code. The way those data are read to make things work is the "model". In the (bad) analogy of the JPEG, the data is the coefficients and the model is the encoding/decoding scheme. I'm impressed by the patience monocasa has in the face of so many (seemingly intentionally) obtuse responses.
- artursapek 3y agoThe JPEG analogy makes no sense. A more correct analogy is: they released compiled binaries, not the source code. The difference is though, here compiling from source is prohibitively expensive for the average person. And the source is massive.
- uoaei 3y agoMy analogy makes perfect sense. Also, being able to reverse-engineer the precise build process is an orthogonal concern to having the source code available.