3 ms·
Sorry, I should have been clearer. I meant when you download a gguf file from huggingface, instead of using a model from ollama's library.
by rahimnathwani 2y ago
Sorry, I should have been clearer.
I meant when you download a gguf file from huggingface, instead of using a model from ollama's library.
- monkmartinez 2y agoollama pull hf.co/unsloth/Qwen3-30B-A3B-GGUF:Q4_K_M and the modelfile comes with it. It may have errors in the template or parameters this way. It has to be converted to GGUF/GGML prior to using it this way. You can, of course, convert and create the specific ollama model from bf16 safetensors as well.
- deleted 2y ago[deleted]
- rahimnathwani 2y agoYeah when I do this, the modelfile has only FROM and TEMPLATE. No PARAMETERs: ollama pull hf.co/jedisct1/MiMo-7B-RL-GGUF:Q4_K_M ollama show --modelfile hf.co/jedisct1/MiMo-7B-RL-GGUF:Q4_K_M