3 ms·
ROLV: 50.5× speedup and 91.4% energy savings Qwen2.5-72B MoE expert FFN (1xB200)
- heggenhougen 8mo agoWe just ran the exact expert FFN slice from Qwen2.5-72B-Instruct (8192×28672, batch 512) on a single NVIDIA B200. Results (ROLV vs vendor-best cuBLAS): - Speedup : 50.5× (4953% faster) - Energy Savings : 91.4% - Tokens/s : 6.42M vs 127k - TFLOPS : 3,018 vs 59.7 - Energy : 64 J vs 742 J - Per-iter : 0.000080 s vs 0.004027 s A_hash and V_hash are identical for both runs (full reproducibility). ROLV_norm_hash: 8dbe5f139fd946d4cd84e8cc612cd9f68cbc87e394457884acc0c5dad56dd8dd This is the real hot path for MoE inference. No synthetic matrices. Comments and questions welcome