3 ms·
i have a Strix Halo machine typically those dense models are too slow on Strix Halo to be practical, expect 5-7 tps you can get an idea by looking at other de
by muyuu 6mo ago
i have a Strix Halo machine
typically those dense models are too slow on Strix Halo to be practical, expect 5-7 tps
you can get an idea by looking at other dense benchmarks here: https://strixhalo.zurkowski.net/experiments https://strixhalo.zurkowski.net/experiments - i'd expect this model to be tested here soon, i don't think i will personally bother
- hedgehog 6mo agoThis one is around 250 t/s prefill and 12.4 generation in my testing.
- anonym29 6mo agosimilar numbers here - slightly higher PP. slightly better peak speed and retention w/ q8_0 kv cache quants too. llama-bench results here, cba to format for hn: https://pastebin.com/raw/zgJeqRbv https://pastebin.com/raw/zgJeqRbv GTR 9 Pro, "performance" profile in BIOS, GTT instead of GART, Fedora 44
- hedgehog 6mo agoIf I did a proper benchmark I think the numbers would be what you got. Minimax M2.7 is also surprisingly not that slow, and in some ways faster as it seems to get things right with less thinking output. (around 140 t/s prefill and 23 t/s generation).
- anonym29 6mo agoThe problem with M2.7 is that it's full GQA, meaning quadratic attention. It does start fast, but by 64k tokens deep, the version I'm running (Unsloth's UD IQ2_XXS) pp512 drops 95% from 261.3 t/s (0 context depth) to 13.1 t/s. q8_0 KV does help, still hitting 57.4 t/s at 64k depth vs 258.3 t/s at 0 depth. TG's retention rates are better, but still approaching single digit even with q8_0 KV cache by 64k depth. That said, it was my favorite model when I valued output quality above all else, at least up until the new Qwen 3.6 27B, which I'm currently playing with. I suspect I will like Qwen 3.6 122B A10B a LOT, maybe even better than M2.7.
- muyuu 6mo agointeresting, might be worth having around although it is still pretty slow
- rpdillon 6mo agoYep, clocking a run right now that's averaging about 8.7t/s. But when I don't mind waiting while I go eat a meal or something, it's not bad! EDIT: I'm running the Unsloth Qwen3.6-27B-Q6_K GGUF on a Corsair Strix Halo 128GB I bought summer 2025. https://huggingface.co/unsloth/Qwen3.6-27B-GGUF/blob/main/Qwen3.6-27B-Q6_K.gguf https://huggingface.co/unsloth/Qwen3.6-27B-GGUF/blob/main/Qw...