3 ms·
If you want the fastest open source implementation on Blackwell and AMD MI355, check out Modular's MAX nightly. You can pip install it super fast, check it out
by chrislattner 6mo ago
If you want the fastest open source implementation on Blackwell and AMD MI355, check out Modular's MAX nightly. You can pip install it super fast, check it out here:
https://www.modular.com/blog/day-zero-launch-fastest-performance-for-gemma-4-on-nvidia-and-amd?utm_campaign=day0&utm_source=hn_chris https://www.modular.com/blog/day-zero-launch-fastest-perform...
-Chris Lattner (yes, affiliated with Modular :-)
- nabakin 6mo agoFaster than TensorRT-LLM on Blackwell? Or do you not consider TensorRT-LLM open source because some dependencies are closed source?
- melodyogonna 6mo agoI reviewed the TensorRT-LLM commit history from the past few days and couldn't find any updates regarding Gemma 4 support. By contrast, here is the reference for MAX:https://github.com/modular/modular/commit/57728b23befed8f3b44e24d67d41b275bd3e49d4 https://github.com/modular/modular/commit/57728b23befed8f3b4...
- nabakin 6mo agoIf OP meant they have the fastest implementation of Gemma 4 on Blackwell at the moment, I guess that is technically true. I doubt that will hold up when TensorRT-LLM finishes their implementation though.
- jjcm 6mo agoWhat % of a speedup should I be expecting vs just running this the standard pytorch approach?