5 ms·
The Mac Pro is actually very competitive against similar $7,000 - $10,000 workstations; especially if you need a lot of high bandwidth memory. However, it's no
by aurareturn 3y ago
The Mac Pro is actually very competitive against similar $7,000 - $10,000 workstations; especially if you need a lot of high bandwidth memory.
However, it's not competitive with workstations that are above that because you'll quickly approach workstations that have Epyc and professional Nvidia cards.
We'll need to wait for an Mx "Extreme" SoC, if Apple ever makes one, to compete above the $7,000 price point.
- mort96 3y agoI feel like I'd have a very hard time designing a $7000 system which doesn't completely blow the Mac Pro out of the water in terms of CPU and GPU performance. Do you have some examples of $7000 systems which are beaten by the $7000 Mac Pro's cut-down M2 Ultra in terms of CPU or GPU power? EDIT: I don't get why this is getting downvoted. The M2 Ultra really doesn't have very impressive GPU power compared to even consumer offerings from nvidia and AMD. The competitive $7000 workstations would have to have something weaker than a 4080.
- junofan 3y agoRTX 6000 Ada alone is $7000. Don’t see how you’re “completely blowing the Mac Pro out of the water” at that price point.
- mort96 3y agoThe 4090, which seems like it's blowing the M2 Ultra out of the water, is $1600 though.
- junofan 3y agoGreat for Cyberpunk 2077, but not great if you need more than 24 GiB.
- greedo 3y agoConsidering the RTX 6000 ADA is like $8k, you could add three 4090s for half the cost.
- mort96 3y agoYou know what, that's actually fair. If your workload needs tons of GPU memory, nothing probably beats the Mac Pro's up to 192GB. Except, I guess, for the Mac Studio.
- lostmsu 3y agoIs there a workload that actually needs more than 24GB of VRAM that can't be offloaded to regular RAM. I honestly don't have any examples out of my head. LLM inference is compute bound AFAIK.
- crasm 3y agoLLM inference is GPU bound and VRAM bound. Given quantization however, 2x3090 or 4090 (48GB) is enough VRAM to load 65B quantized llama derivatives (30-40GB). With exllama you can get maybe 20-30 tok/s with dual 4090s on ~1200W. With an M2 Ultra maybe 10 tok/s on ~178W. (I don't have these devices but have been researching LLM inference performance in the interest of buying a machine for them.) Advantage goes to M2 Ultra if you ever might need more than 48GB VRAM. I think it's unlikely nvidia is going to release a significantly higher VRAM consumer card anytime soon, since that's what their A100s are for.
- mort96 3y agoIf only the big ML stuff didn't all use CUDA. Guessing that world won't transition to Metal compute any time soon.
- blackoil 3y agoAre there any such non AI use cases? For AI anyway its all Nvidia.
- greedo 3y agoYou'll still be limited by memory. If Apple were to create a Jade 4c chipset, (two Ultra SoC), you'd only have 384GB of RAM.
- aurareturn 3y agoDepends on what you do. Let's say Apple makes an M3 "Extreme" and sets max memory to 512GB of RAM. That's 512GB of high bandwidth RAM that you'd have to piece together 7 Nvidia H100 80GB cards to match. Each H100 80GB card costs $30,000+. Seven of them would cost $210k alone. Now of course 7 H100s would have more compute power, but if what you need is a lot of high bandwidth RAM such as a large LLM or highly detailed scene rendering, then maybe an M3 "Extreme" would have the best value out there, by far.