4 ms·

This is just a little under the price of NVidia's DGX Spark with CUDA or a Mac with 128GB and twice the memory bandwidth. The point of Strix Halo used to be tha

by c7b 3mo ago

This is just a little under the price of NVidia's DGX Spark with CUDA or a Mac with 128GB and twice the memory bandwidth. The point of Strix Halo used to be that it was half the price of those way more capable machines. You'd be crazy to buy the AMD chip at this price. But the hardware market is generally crazy right now, so I'm sure this will sell as well, unfortunately.

xandrius 3mo ago
Personally, I'm totally ok to have a competitor to Nvidia, regardless of whether they are under the price or not.
- c7b 3mo ago
  But ideally they would be competitive, right? If your goal is LLM or Diffusion inference or - god forbid - training, you're going to get way better performance on DGX Spark. The difference is more stark than 250 vs 273 GB/s bandwidth delta would suggest. Now I think it's totally fine to have a less capable offering, and the Strix Halo is still a mighty capable machine for inference on mid-size MoEs. At 2k it was a tinkerer's dream. But the performance difference should be reflected in the price. This is roughly a doubling of the price compared to less than a year ago without adding any notable features, it's appalling.
p1esk 3mo ago
where do you see "twice the memory bandwidth"?
- ahmadyan 3mo ago
  He is making things up. It is the same bandwidth as DGX Spark (256 GB/s vs 273 GB/s) and far behind M3 Ultra (~819 GB/s)
  - c7b 3mo ago
    You're the one making things up. An M3 Ultra with 128GB RAM doesn't exist, the M3 Max has 410GB/s bandwidth [0]. I was of course talking about the M4 Max with 546GB/s, which was closer to twice the price of a Strix Halo mini PC in a typical configuration when it was still available. And memory bandwidth isn't everything, NVidia's lead in software is substantial, look up any tests comparing them side-by-side. [0] https://en.wikipedia.org/wiki/Apple_M3 https://en.wikipedia.org/wiki/Apple_M3 [1] https://en.wikipedia.org/wiki/Apple_M4 https://en.wikipedia.org/wiki/Apple_M4
    - ahmadyan 3mo ago
      Google is your friend: https://www.apple.com/mac-studio/specs/ https://www.apple.com/mac-studio/specs/ Apple M3 Ultra chip 819GB/s memory bandwidth They have 96, 128, 256, and 512GB variants my friend.
      - deleted 3mo ago
        [deleted]
      - icedchai 3mo ago
        They stopped selling anything greater than the 96 gig variant a while back. If you want more RAM, be prepared to bid high on ebay.
    - actionfromafar 3mo ago
      Well, some exist, but I guess they don't make any new ones for now.
- c7b 3mo ago
  The M4 Max with 128GB RAM has 546GB/s memory bandwidth [0], compared to Strix Halo's 250 (on the label, I've yet to see a benchmark that tops 220). It's not available at 128GB RAM anymore, at least in my shop, but when it was not so long ago it was about 4,7k, or a little over twice the price of a cheaper Strix Halo PC (around 2,2k a few months ago). [0] https://en.wikipedia.org/wiki/Apple_M4 https://en.wikipedia.org/wiki/Apple_M4
tencentshill 3mo ago
The Nvidia DGX are sitting on the shelf unsold at $4.5k.
adrian_b 3mo ago
Only the top variants of M5 Max have more than twice the memory bandwidth of AMD, but those are much more expensive, i.e. over $10,000. M5 Pro has as slightly higher memory bandwidth, but currently it is available only with up to 64 GB of DRAM. Even with this small memory, it is more expensive than either AMD or NVIDIA, especially when you do not want a puny SSD, but a normal-sized SSD, i.e. big enough to put there an LLM if you want to compute a quantization yourself (i.e. more than $5600 with a 4 TB SSD & 64 GB DRAM). If you want to do LLM inference, I do not see Apple as a competitive solution, as their price is much, much higher, while also having limited expansion for SSDs, where you need a lot of space if you want to store a few LLMs. The only thing that is correct is that the AMD Strix Halo system used to be much cheaper than NVIDIA, but now it has the same price. The CPU of Strix Halo is better than that of NVIDIA, but the NVIDIA GPU is likely to be better than the AMD GPU and CUDA is guaranteed to work fine.

by c7b 3mo ago

xandrius 3mo ago

Personally, I'm totally ok to have a competitor to Nvidia, regardless of whether they are under the price or not.

c7b 3mo ago
But ideally they would be competitive, right? If your goal is LLM or Diffusion inference or - god forbid - training, you're going to get way better performance on DGX Spark. The difference is more stark than 250 vs 273 GB/s bandwidth delta would suggest. Now I think it's totally fine to have a less capable offering, and the Strix Halo is still a mighty capable machine for inference on mid-size MoEs. At 2k it was a tinkerer's dream. But the performance difference should be reflected in the price. This is roughly a doubling of the price compared to less than a year ago without adding any notable features, it's appalling.

p1esk 3mo ago

where do you see "twice the memory bandwidth"?

ahmadyan 3mo ago
He is making things up. It is the same bandwidth as DGX Spark (256 GB/s vs 273 GB/s) and far behind M3 Ultra (~819 GB/s)
- c7b 3mo ago
  You're the one making things up. An M3 Ultra with 128GB RAM doesn't exist, the M3 Max has 410GB/s bandwidth [0]. I was of course talking about the M4 Max with 546GB/s, which was closer to twice the price of a Strix Halo mini PC in a typical configuration when it was still available. And memory bandwidth isn't everything, NVidia's lead in software is substantial, look up any tests comparing them side-by-side. [0] https://en.wikipedia.org/wiki/Apple_M3 https://en.wikipedia.org/wiki/Apple_M3 [1] https://en.wikipedia.org/wiki/Apple_M4 https://en.wikipedia.org/wiki/Apple_M4
  - ahmadyan 3mo ago
    Google is your friend: https://www.apple.com/mac-studio/specs/ https://www.apple.com/mac-studio/specs/ Apple M3 Ultra chip 819GB/s memory bandwidth They have 96, 128, 256, and 512GB variants my friend.
    - deleted 3mo ago
      [deleted]
    - icedchai 3mo ago
      They stopped selling anything greater than the 96 gig variant a while back. If you want more RAM, be prepared to bid high on ebay.
  - actionfromafar 3mo ago
    Well, some exist, but I guess they don't make any new ones for now.
c7b 3mo ago
The M4 Max with 128GB RAM has 546GB/s memory bandwidth [0], compared to Strix Halo's 250 (on the label, I've yet to see a benchmark that tops 220). It's not available at 128GB RAM anymore, at least in my shop, but when it was not so long ago it was about 4,7k, or a little over twice the price of a cheaper Strix Halo PC (around 2,2k a few months ago). [0] https://en.wikipedia.org/wiki/Apple_M4 https://en.wikipedia.org/wiki/Apple_M4

tencentshill 3mo ago

The Nvidia DGX are sitting on the shelf unsold at $4.5k.

adrian_b 3mo ago

Only the top variants of M5 Max have more than twice the memory bandwidth of AMD, but those are much more expensive, i.e. over $10,000. M5 Pro has as slightly higher memory bandwidth, but currently it is available only with up to 64 GB of DRAM. Even with this small memory, it is more expensive than either AMD or NVIDIA, especially when you do not want a puny SSD, but a normal-sized SSD, i.e. big enough to put there an LLM if you want to compute a quantization yourself (i.e. more than $5600 with a 4 TB SSD & 64 GB DRAM). If you want to do LLM inference, I do not see Apple as a competitive solution, as their price is much, much higher, while also having limited expansion for SSDs, where you need a lot of space if you want to store a few LLMs. The only thing that is correct is that the AMD Strix Halo system used to be much cheaper than NVIDIA, but now it has the same price. The CPU of Strix Halo is better than that of NVIDIA, but the NVIDIA GPU is likely to be better than the AMD GPU and CUDA is guaranteed to work fine.