3 ms·
Is that RAM as fast as A100/H100 VRAM? AFAIR, gpus push ~ 1 TB/s ish
by eurekin 3y ago
Is that RAM as fast as A100/H100 VRAM? AFAIR, gpus push ~ 1 TB/s ish
- jlokier 3y agoThe Apple M2 Ultra memory bandwidth is 800GB/s so it's not a long way off 1TB/s. A100 goes up to 2TB/s in the largest configuration, and H100 claims 3TB/s per GPU node. (These figures keep changing with new variants.) But you can buy several Mac Studios for the price :-) The real use of H100 is for training, as it can pool many GPUs together with a dedicated high bandwidth network. You can't do that with Mac Studios.
- eurekin 3y agoThat figures are really impressive. A single node is still useful for finetuning smaller models. Apple could move the needle in this market a bit