3 ms·
You tried a 4-bit quantized version, not the original. qwen3:30b has the same checksum as https://ollama.com/library/qwen3:30b-a3b-q4_K_M https://ollama.com/li
by rahimnathwani 2y ago
You tried a 4-bit quantized version, not the original.
qwen3:30b has the same checksum as https://ollama.com/library/qwen3:30b-a3b-q4_K_M https://ollama.com/library/qwen3:30b-a3b-q4_K_M
- croemer 2y agoWhat is the original? The blog post doesn't state the quantization they benchmarked.
- rahimnathwani 2y agoThis 61GB one: https://ollama.com/library/qwen3:30b-a3b-fp16 https://ollama.com/library/qwen3:30b-a3b-fp16 You can see it's roughly the same size as the one in the official repo (16 files of 4GB each): https://huggingface.co/Qwen/Qwen3-30B-A3B/tree/main https://huggingface.co/Qwen/Qwen3-30B-A3B/tree/main
- int_19h 2y agofp16 is overkill though. 8-bit is the sweet spot before perf degradation starts getting noticeable.
- rahimnathwani 2y agoI haven't yet seen any evals comparing the original Qwen3-30B-A22B with https://ollama.com/library/qwen3:30b-a3b-q8_0 https://ollama.com/library/qwen3:30b-a3b-q8_0