3 ms·
2x RTX 4090, Q8, 256k context, 110 t/s
by mirekrusin 8mo ago
2x RTX 4090, Q8, 256k context, 110 t/s
- instagib 8mo ago1 4090, Qwen3.5-35B-A3B-UD-MXFP4_MOE, 64k context, 122 t/s. Llama.cpp
- mirekrusin 8mo agoI believe it's mentioned that MXFP4 performs surprisingly bad, you may want to try other Q4s.