4 ms·
What is the original? The blog post doesn't state the quantization they benchmarked.
by croemer 1y ago
What is the original? The blog post doesn't state the quantization they benchmarked.
- rahimnathwani 1y agoThis 61GB one: https://ollama.com/library/qwen3:30b-a3b-fp16 https://ollama.com/library/qwen3:30b-a3b-fp16 You can see it's roughly the same size as the one in the official repo (16 files of 4GB each): https://huggingface.co/Qwen/Qwen3-30B-A3B/tree/main https://huggingface.co/Qwen/Qwen3-30B-A3B/tree/main
- int_19h 1y agofp16 is overkill though. 8-bit is the sweet spot before perf degradation starts getting noticeable.
- rahimnathwani 1y agoI haven't yet seen any evals comparing the original Qwen3-30B-A22B with https://ollama.com/library/qwen3:30b-a3b-q8_0 https://ollama.com/library/qwen3:30b-a3b-q8_0