3 ms·
How would one go about making a .llamafile for this?
by m3kw9 3y ago
How would one go about making a .llamafile for this?
- simonw 3y agoConvert it to GGML and use a zip tool to add that to a llamafile package. https://huggingface.co/TheBloke?search_models=Zephyr https://huggingface.co/TheBloke?search_models=Zephyr doesn't have a GGML for it yet but I wouldn't be surprised to see one by the end of the day.
- deleted 3y ago[deleted]
- pleks 3y agoAnd it's been uploaded https://huggingface.co/TheBloke/stablelm-zephyr-3b-GGUF/tree/main https://huggingface.co/TheBloke/stablelm-zephyr-3b-GGUF/tree... Getting 20toks/s on M1 mba where as LLaVa I ground to a halt. Very impressed