3 ms·
Didn't llama.cpp need to convert the weights file to a new format to support that? The way they're stored in the official file isn't efficient for operating on
by sp332 3y ago
Didn't llama.cpp need to convert the weights file to a new format to support that? The way they're stored in the official file isn't efficient for operating on directly.
- LoganDark 3y agoBecause the original format is the undocumented Python pickle format packed into a zip file. It's kind of ridiculous to attempt to support directly.
- gliptic 3y agoThey already had their own format before that.
- petters 3y agoI don't know about llama.cpp, but yes this method works best if the binary layout on disk is exactly what you use for matrices in memory