4 ms·
This is obviously intended for running local AI, but the memory bandwidth of 300 GB/sec seems a limiting factor, since this is what limits single user LLM infer
by HarHarVeryFunny 5mo ago
This is obviously intended for running local AI, but the memory bandwidth of 300 GB/sec seems a limiting factor, since this is what limits single user LLM inference tokens/sec. For comparison the M2 Ultra does up to 800 GB/sec, thanks to having a 1024-bit bus vs the RTX Spark's 256-bit bus.
- m-schuetz 5mo agoWhere did you find the 300GB/s? That's indeed disappointing if true.
- HarHarVeryFunny 5mo agoFrom here: https://www.techpowerup.com/349554/nvidia-announces-rtx-spark-a-supercomputer-grade-processor-for-windows-pcs-with-agentic-user-interfaces https://www.techpowerup.com/349554/nvidia-announces-rtx-spar...