3 ms·

Except the RTX 6000 will run circles around the Mac studio in just about every way. Memory bandwidth is literally the only spec where Apple is competitive, and

Except the RTX 6000 will run circles around the Mac studio in just about every way. Memory bandwidth is literally the only spec where Apple is competitive, and while high memory bandwidth is necessary for LLMs to perform well, many people strangely don't understand that memory bandwidth alone is not sufficient.

F7F7F7 2mo ago
It better because you’ll need a few of them to run some larger models (I’ll be just as vague citing which models).
- angoragoats 2mo ago
  I was not speaking about specific models so I didn’t feel the need to cite any. Not sure why the backhanded insult was necessary. You need multiple Mac Studios to run the largest models as well, so neither is a one-size-fits-all device. If you can show me a model for which a Mac is faster than the RTX 6000 then I’ll be happy to update or retract my statement.
speedgoose 2mo ago
Energy efficiency is in another league with the Mac Studio for local workloads. I can run agents using deepseek v4 flash or Qwen 3.8 on my m3 ultra and it will be lukewarm and the fan will eventually start blowing softly.
- angoragoats 2mo ago
  I’ve run the numbers on this, and an optimized nVidia build can meet or beat Apple platforms in terms of tokens per watt, which is the most important efficiency metric if what you care about is using the least amount of energy to generate a given response. Yes, the Mac might get lukewarm, but it will take 2-3+ times longer to do the same task.
  - speedgoose 2mo ago
    Yes I agree that at full speed with parallel workloads, the tokens per watt are better using Nvidia GPU servers. Also while the pre fill performance sucks, the Mac isn’t that slow and can use much better model compared to a similarly priced Nvidia workstation so it’s not really taking much longer in practice. It’s taking longer than in the cloud for sure. At least a local computer uses the local energy grid that is pretty clean and not fossil energy.
    - angoragoats 2mo ago
      Okay, so you said “Energy efficiency is in another league with the Mac Studio”; I assume you meant that it was more efficient than an nVidia GPU setup. But when pressed, you agreed that’s not actually the case. So it’s not actually that much different in terms of energy efficiency. > Also while the pre fill performance sucks, the Mac isn’t that slow and can use much better model compared to a similarly priced Nvidia workstation so it’s not really taking much longer in practice. This is exactly backwards; assuming you mean that the Mac has more RAM so you can use a larger model, the mac is going to be _even slower_ since the fastest Mac isn’t as fast as the average nVidia setup. My point is that comparing apples to apples (no pun intended), an nVidia setup is both faster and (possibly with some tweaking) more efficient than a Mac.
aurareturn 2mo ago
Mac studio wins in memory capacity, price, perf/watt and value. RTX 6000 wins in performance, if your model can fit into the VRAM. There are very obvious and clear advantages to a Mac Studio. It's an entire system for one and you're getting a world class CPU as well.
- angoragoats 2mo ago
  > Mac studio wins in memory capacity, price, perf/watt and value. [citation needed]. I have personally specced out and built an nvidia GPU-based machine which after some optimization, handily beat the Mac Studio in terms of tokens/watt for LLM inference with most models. This was in the M2 Ultra era, and I haven't run the numbers for the later generations, but nvidia's cards have gotten faster just as Apple's CPUs/GPUs have, so I would guess that it's still possible to do. > RTX 6000 wins in performance, if your model can fit into the VRAM. "if your model can fit into the VRAM" can be true for the Mac as well. > There are very obvious and clear advantages to a Mac Studio. There are certain advantages for sure, depending on your use case. They may _seem_ to be obvious, but as evidenced above, I believe that many people overestimate the Mac's superiority on the metrics you cite when comparing a Mac vs. a dedicated GPU for LLM inference.
  - seanmcdirmid 2mo ago
    > "if your model can fit into the VRAM" can be true for the Mac as well. It is much more likely for your model to fit in large unified memory of a Mac than the smaller more limited memory of a GPU. Even going with two 5090s, you now have to shard your model and that is a PITA. But it turns out that MoE is the solution both for running models on macs of limited computer power means (not as fast as GPUs), and on multiple GPUs that require sharding the model.
    - angoragoats 2mo ago
      > It is much more likely for your model to fit in large unified memory of a Mac than the smaller more limited memory of a GPU. I bristle at general statements like this when it obviously depends on the specific Mac and GPU in question. But yes, comparing a maxed out M5 Ultra with an RTX 6000, the Mac has much more memory. > Even going with two 5090s, you now have to shard your model and that is a PITA. Every modern tool does this for you automatically. It is absolutely not a pain in the least (e.g. llama.cpp ships with pipeline parallelism enabled by default).

F7F7F7 2mo ago

It better because you’ll need a few of them to run some larger models (I’ll be just as vague citing which models).

angoragoats 2mo ago
I was not speaking about specific models so I didn’t feel the need to cite any. Not sure why the backhanded insult was necessary. You need multiple Mac Studios to run the largest models as well, so neither is a one-size-fits-all device. If you can show me a model for which a Mac is faster than the RTX 6000 then I’ll be happy to update or retract my statement.

speedgoose 2mo ago

Energy efficiency is in another league with the Mac Studio for local workloads. I can run agents using deepseek v4 flash or Qwen 3.8 on my m3 ultra and it will be lukewarm and the fan will eventually start blowing softly.

angoragoats 2mo ago
I’ve run the numbers on this, and an optimized nVidia build can meet or beat Apple platforms in terms of tokens per watt, which is the most important efficiency metric if what you care about is using the least amount of energy to generate a given response. Yes, the Mac might get lukewarm, but it will take 2-3+ times longer to do the same task.
- speedgoose 2mo ago
  Yes I agree that at full speed with parallel workloads, the tokens per watt are better using Nvidia GPU servers. Also while the pre fill performance sucks, the Mac isn’t that slow and can use much better model compared to a similarly priced Nvidia workstation so it’s not really taking much longer in practice. It’s taking longer than in the cloud for sure. At least a local computer uses the local energy grid that is pretty clean and not fossil energy.
  - angoragoats 2mo ago
    Okay, so you said “Energy efficiency is in another league with the Mac Studio”; I assume you meant that it was more efficient than an nVidia GPU setup. But when pressed, you agreed that’s not actually the case. So it’s not actually that much different in terms of energy efficiency. > Also while the pre fill performance sucks, the Mac isn’t that slow and can use much better model compared to a similarly priced Nvidia workstation so it’s not really taking much longer in practice. This is exactly backwards; assuming you mean that the Mac has more RAM so you can use a larger model, the mac is going to be _even slower_ since the fastest Mac isn’t as fast as the average nVidia setup. My point is that comparing apples to apples (no pun intended), an nVidia setup is both faster and (possibly with some tweaking) more efficient than a Mac.

aurareturn 2mo ago

Mac studio wins in memory capacity, price, perf/watt and value. RTX 6000 wins in performance, if your model can fit into the VRAM. There are very obvious and clear advantages to a Mac Studio. It's an entire system for one and you're getting a world class CPU as well.

angoragoats 2mo ago
> Mac studio wins in memory capacity, price, perf/watt and value. [citation needed]. I have personally specced out and built an nvidia GPU-based machine which after some optimization, handily beat the Mac Studio in terms of tokens/watt for LLM inference with most models. This was in the M2 Ultra era, and I haven't run the numbers for the later generations, but nvidia's cards have gotten faster just as Apple's CPUs/GPUs have, so I would guess that it's still possible to do. > RTX 6000 wins in performance, if your model can fit into the VRAM. "if your model can fit into the VRAM" can be true for the Mac as well. > There are very obvious and clear advantages to a Mac Studio. There are certain advantages for sure, depending on your use case. They may _seem_ to be obvious, but as evidenced above, I believe that many people overestimate the Mac's superiority on the metrics you cite when comparing a Mac vs. a dedicated GPU for LLM inference.
- seanmcdirmid 2mo ago
  > "if your model can fit into the VRAM" can be true for the Mac as well. It is much more likely for your model to fit in large unified memory of a Mac than the smaller more limited memory of a GPU. Even going with two 5090s, you now have to shard your model and that is a PITA. But it turns out that MoE is the solution both for running models on macs of limited computer power means (not as fast as GPUs), and on multiple GPUs that require sharding the model.
  - angoragoats 2mo ago
    > It is much more likely for your model to fit in large unified memory of a Mac than the smaller more limited memory of a GPU. I bristle at general statements like this when it obviously depends on the specific Mac and GPU in question. But yes, comparing a maxed out M5 Ultra with an RTX 6000, the Mac has much more memory. > Even going with two 5090s, you now have to shard your model and that is a PITA. Every modern tool does this for you automatically. It is absolutely not a pain in the least (e.g. llama.cpp ships with pipeline parallelism enabled by default).