3 ms·
One differentiator is the amount of memory available. Apple's shared pool of memory means the GPU has potentially up to 96gb of ram at its disposal, where discr
by sunflowerfly 3y ago
One differentiator is the amount of memory available. Apple's shared pool of memory means the GPU has potentially up to 96gb of ram at its disposal, where discrete GPU's are often limited to 8, 12, 16, etc. They mentioned in the keynote that the M2 can handle many tasks discrete GPUs cannot simply because they run out of ram.
- incrudible 3y ago$15,000 will buy you an A100 with 80GB of RAM at 2TB/s that will annihilate the biggest Apple chip.
- ElFitz 3y agoSure, but then that’s the price of multiple entire laptops. Or 2 Mac Pros. Not sure what point you’re trying to make here.
- incrudible 3y agoIt is also the performance of 4 Mac Pros. GP was talking about tasks that dedicated GPUs supposedly could not handle. Who is is running models needing 60+ GB of VRAM on their laptops?
- ElFitz 3y agoSome have had fun running some of the larger Llama, Vicuna & Guanaco models on their MacBooks. Mostly to experiment with local inference. Although quantisation has now lowered the required memory, I wouldn’t be surprised if it comes in handy again in the near future. But today that’s quite a niche use case.