3 ms·
fp16 is overkill though. 8-bit is the sweet spot before perf degradation starts getting noticeable.
by int_19h 2y ago
fp16 is overkill though. 8-bit is the sweet spot before perf degradation starts getting noticeable.
- rahimnathwani 2y agoI haven't yet seen any evals comparing the original Qwen3-30B-A22B with https://ollama.com/library/qwen3:30b-a3b-q8_0 https://ollama.com/library/qwen3:30b-a3b-q8_0