4 ms·
These are valid points but the numbers are still useful as a floor on performance. Given Strix Halo is so much cheaper I'd expect more people to work on improv
by nl 1y ago
These are valid points but the numbers are still useful as a floor on performance.
Given Strix Halo is so much cheaper I'd expect more people to work on improving it, but the NVIDIA tools are better so unclear which has more headroom.
- aseipp 1y agoYeah that's fair. 60 tok/sec on a gpt-oss-120b is certainly nice to know if you should even think about it at all. I'm quite happy with it anyway. The pricing is definitely by far the worst part of all of this. I suspect the GB10 still has more perf left on the table, Blackwell has been a rough launch. But I'm not sure it's $2000 better if you're just looking to get a fun little AI machine to do embeddings/vision/LLMs on?