2 ms·
Convert it to GGML and use a zip tool to add that to a llamafile package. https://huggingface.co/TheBloke?search_models=Zephyr https://huggingface.co/TheBloke?
by simonw 3y ago
Convert it to GGML and use a zip tool to add that to a llamafile package.
https://huggingface.co/TheBloke?search_models=Zephyr https://huggingface.co/TheBloke?search_models=Zephyr doesn't have a GGML for it yet but I wouldn't be surprised to see one by the end of the day.
- deleted 3y ago[deleted]
- pleks 3y agoAnd it's been uploaded https://huggingface.co/TheBloke/stablelm-zephyr-3b-GGUF/tree/main https://huggingface.co/TheBloke/stablelm-zephyr-3b-GGUF/tree... Getting 20toks/s on M1 mba where as LLaVa I ground to a halt. Very impressed