3 ms·
Not quite, what is the vRAM bandwidth of each? The bandwidth is a huge contributor to LLM performance.
by plagiarist 7mo ago
Not quite, what is the vRAM bandwidth of each? The bandwidth is a huge contributor to LLM performance.
- embedding-shape 7mo agoAFAIK, for the unified bandwidth, it depends mostly on the CPU, for M4 Max (I think it's the default today?) it does ~550 GB/s, while GB10 does ~270 GB/s, so about a 2x difference between the two. For comparison, RTX Pro 6000 does 1.8 TB/s, pretty much the same as what a 5090 does, which is probably the fastest/best GPUs a prosumer reasonable could get.
- plagiarist 7mo agoGranted, it won't be competitive against the flagship dGPUs. Nevertheless, that ~2x is a pretty huge difference in similarly priced offerings.