4 ms·
If I have the baseline Mixtral-8x7B .pth, how do I convert that to a llamafile compatible .gguf?
by fbdab103 3y ago
If I have the baseline Mixtral-8x7B .pth, how do I convert that to a llamafile compatible .gguf?
- tarruda 3y agoJust download the GGUF from huggingface. @TheBloke always uploads GGUF versions of the popular models
- ekianjo 3y agoUseless as he never updates them post release. You are better served doing it yourself.
- tarruda 3y agoWhere did you get your information from? Just 4 days ago he redid a model after the author updated the EOS token: https://huggingface.co/TheBloke/openchat-3.5-1210-GGUF/discussions/1#657c2f1bc8c6e8dceed9289a https://huggingface.co/TheBloke/openchat-3.5-1210-GGUF/discu...
- ekianjo 3y agoJust make the gguf yourself it takes 5 mins and you dont need thebloke
- fbdab103 3y agoThat was my question. How do I create the gguf?
- onedognight 3y agoUsing the convert.py script in the llama.cpp repo.
- fbdab103 3y agoPerfect, thanks. Don't suppose if you know if the conversion is easily reversible? Some of these models are big, it sucks to carry around the original plus the gguf, but I would hate to be in a situation where the gguf represents a dead end.