2 ms·
The studio has a lot more ram available to the GPU (up to 192gb) than the a100 (80gb), and iirc at least comparable memory bandwidth -- those are what matter wh
by achierius 2y ago
The studio has a lot more ram available to the GPU (up to 192gb) than the a100 (80gb), and iirc at least comparable memory bandwidth -- those are what matter when you're doing LLM inference, so the studio tends to win out there.
Where the a100 and other similar chips dominate is in training &c, which is mostly a question of flops.
- diggan 2y ago> and iirc at least comparable memory bandwidth I don't think they do. From Wikipedia: > the M2 Pro, M2 Max, and M2 Ultra have approximately 200 GB/s, 400 GB/s, and 800 GB/s respectively From techpowerup: > NVIDIA A100 SXM4 80 GB - Memory bandwidth - 2.04 TB/s Seems to be a magnitude of difference, and that's just the bandwidth.