3 ms·

I had been hoping that these would be a bit faster than the 9950X because of the different memory architecture, but it appears that due to the lower power desig

I had been hoping that these would be a bit faster than the 9950X because of the different memory architecture, but it appears that due to the lower power design point the AI Max+ 395 loses across the board, by large margins. So I guess these really are niche products for ML users only, and people with generic workloads that want more than the 9950X offers are shopping for a Threadripper.

dijit 1y ago
Sounds about right. I’m struggling to justify the cost of a Threadripper (let alone pro!) for a AAA game studio though. I wonder who can justify these machines. High frequency trading? data science? shouldn’t that be done on servers?
- jeffbee 1y ago
  Yeah I don't get it either. To get marginally more resources than the 9950X you have to make a significant leap in price to a $1500+ CPU on a $1000 motherboard.
  - wpm 1y ago
    No, you get it. This is pure market segmentation. If you need that little bit extra, you’re forced to compromise, or to open your wallet big time, and AMD is betting on people who “really” need that slightly extra oomph to pay.
  - toast0 1y ago
    The premium is about getting more i/o. More memory channel, a lot more lanes. Maybe also a very high power limit. If you need that in a single system, you gotta pay up. Lower tier SP6 processors are actually pretty reasonably priced, boards are still spendy though.
  - sliken 1y ago
    Doubly so when a $650 Epyc motherboard and a $650 Epyc CPU gets you 3x the threadipper's bandwidth and 1.5 the thread ripper pro's bandwidth. Seems like threadripper is too low volume to be price competitive with Epyc and there's a relatively price insensitive workstation market out there.
- kadoban 1y ago
  Threadripper very rarely seems to make any sense. The only times it seems like you want it are for huge memory support/bandwidth and/or a huge number of pcie slots. But it's not cheap or supported enough compared to epyc to really make sense to me any time I've been specing out a system along those lines.
  - StrangeDoctor 1y ago
    I bought a threadripper pro system out of desperation, trying to get secondhand PCIe 80G A100s to run locally. The huge rebar allocations confused/crashed every Intel/AMD system I had access to. I think the Xeon systems should have worked and that it was actually a motherboard bios issue, but I had seen a photo of it running in a threadripper and prayed I wasn’t digging an even deeper hole.
    - kadoban 1y ago
      Yeah, that makes sense if you just have ~proof that some configuration works and want to just be done with it.
      - jeffbee 1y ago
        This is why a business like Puget Systems, or a line like HP Z Workstations, persist. You know in advance that your rig will work.
    - sliken 1y ago
      I've been tempted, but had a hard time finding a case where I needed more than the 9950x but less than a single socket epyc. Especially since the epyc motherboards are cheaper, CPUs are cheaper, and the Epycs have 3x the memory bandwidth of the thread ripper and 1.5x the memory bandwidth of the Threadripper pro.
rtkwe 1y ago
It also seems like the tools aren't there to fully utilize them. Unless I misunderstood he was running off CPU only for all the test so there's still the iGPU and NPU performance that's not been utilized in these tests.
- geerlingguy 1y ago
  No, only a couple initial tests with Ollama used CPU. I ran most tests on Vulkan / iGPU, and some on ROCm (read further down the thread). I found it difficult to install ROCm on Fedora 42 but after upgrading to Rawhide it was easy, so I re-tested everything with ROCm vs Vulkan. Ollama, for some silly reason, doesn't support Vulkan even though I've used a fork many times to get full GPU acceleration with it on Pi, Ampere, and even this AMD system... (moral of the story just stick with llama.cpp).
  - edwinjones 1y ago
    Sadly, the reason they give is subjectively terrible: https://x.com/ollama/status/1952783981000446029 https://x.com/ollama/status/1952783981000446029 No experimental flag option, no "you can use the fork that works fine but we don't have capacity to support this" just a hard "no, we think it's unreliable". I guess they just want you to drop them and use llama.cpp.
    - geerlingguy 1y ago
      Yeah, my conspiracy theory is Nvidia is somehow influencing the decision. If you can do Vulkan with Ollama, it opens up people to using Intel/AMD/other iGPUs and you might not be incentivized to buy an Nvidia GPU. ROCm support is not wonderful. It's certainly worse for an end user to deal with than Vulkan, which usually 'just works'.
      - edwinjones 1y ago
        I agree. AMD should just go all in on vulkan I think, The ROCm compatibility list is terrible compared to...every modern device and probably some ancient gpus that can be made to work with vulkan as well. Considering they created mantle, you would think it would be the obvious move too.
sliken 1y ago
Across the board, by a large margin? Phoronix ran 200 benchmarks on the 9950x vs 395x max and found a difference of less than 5%. Not bad considering the average power use was 91 watts vs 154 watts. If you need the memory bandwidth the strix halo looks good, if you are cache friendly and don't care about using almost double the power than the 9950x is a better deal.
- jeffbee 1y ago
  The way Phoronix weights an average score for a machine is ridiculous, because there aren't any users who do all of machine learning, fluid dynamics, video compression, database hosting, and software development, and games on the same machine. I looked at the applications that matter to me and the 9950X wins by 40% in those.

dijit 1y ago

Sounds about right. I’m struggling to justify the cost of a Threadripper (let alone pro!) for a AAA game studio though. I wonder who can justify these machines. High frequency trading? data science? shouldn’t that be done on servers?

jeffbee 1y ago
Yeah I don't get it either. To get marginally more resources than the 9950X you have to make a significant leap in price to a $1500+ CPU on a $1000 motherboard.
- wpm 1y ago
  No, you get it. This is pure market segmentation. If you need that little bit extra, you’re forced to compromise, or to open your wallet big time, and AMD is betting on people who “really” need that slightly extra oomph to pay.
- toast0 1y ago
  The premium is about getting more i/o. More memory channel, a lot more lanes. Maybe also a very high power limit. If you need that in a single system, you gotta pay up. Lower tier SP6 processors are actually pretty reasonably priced, boards are still spendy though.
- sliken 1y ago
  Doubly so when a $650 Epyc motherboard and a $650 Epyc CPU gets you 3x the threadipper's bandwidth and 1.5 the thread ripper pro's bandwidth. Seems like threadripper is too low volume to be price competitive with Epyc and there's a relatively price insensitive workstation market out there.
kadoban 1y ago
Threadripper very rarely seems to make any sense. The only times it seems like you want it are for huge memory support/bandwidth and/or a huge number of pcie slots. But it's not cheap or supported enough compared to epyc to really make sense to me any time I've been specing out a system along those lines.
- StrangeDoctor 1y ago
  I bought a threadripper pro system out of desperation, trying to get secondhand PCIe 80G A100s to run locally. The huge rebar allocations confused/crashed every Intel/AMD system I had access to. I think the Xeon systems should have worked and that it was actually a motherboard bios issue, but I had seen a photo of it running in a threadripper and prayed I wasn’t digging an even deeper hole.
  - kadoban 1y ago
    Yeah, that makes sense if you just have ~proof that some configuration works and want to just be done with it.
    - jeffbee 1y ago
      This is why a business like Puget Systems, or a line like HP Z Workstations, persist. You know in advance that your rig will work.
  - sliken 1y ago
    I've been tempted, but had a hard time finding a case where I needed more than the 9950x but less than a single socket epyc. Especially since the epyc motherboards are cheaper, CPUs are cheaper, and the Epycs have 3x the memory bandwidth of the thread ripper and 1.5x the memory bandwidth of the Threadripper pro.

rtkwe 1y ago

It also seems like the tools aren't there to fully utilize them. Unless I misunderstood he was running off CPU only for all the test so there's still the iGPU and NPU performance that's not been utilized in these tests.

geerlingguy 1y ago
No, only a couple initial tests with Ollama used CPU. I ran most tests on Vulkan / iGPU, and some on ROCm (read further down the thread). I found it difficult to install ROCm on Fedora 42 but after upgrading to Rawhide it was easy, so I re-tested everything with ROCm vs Vulkan. Ollama, for some silly reason, doesn't support Vulkan even though I've used a fork many times to get full GPU acceleration with it on Pi, Ampere, and even this AMD system... (moral of the story just stick with llama.cpp).
- edwinjones 1y ago
  Sadly, the reason they give is subjectively terrible: https://x.com/ollama/status/1952783981000446029 https://x.com/ollama/status/1952783981000446029 No experimental flag option, no "you can use the fork that works fine but we don't have capacity to support this" just a hard "no, we think it's unreliable". I guess they just want you to drop them and use llama.cpp.
  - geerlingguy 1y ago
    Yeah, my conspiracy theory is Nvidia is somehow influencing the decision. If you can do Vulkan with Ollama, it opens up people to using Intel/AMD/other iGPUs and you might not be incentivized to buy an Nvidia GPU. ROCm support is not wonderful. It's certainly worse for an end user to deal with than Vulkan, which usually 'just works'.
    - edwinjones 1y ago
      I agree. AMD should just go all in on vulkan I think, The ROCm compatibility list is terrible compared to...every modern device and probably some ancient gpus that can be made to work with vulkan as well. Considering they created mantle, you would think it would be the obvious move too.

sliken 1y ago

Across the board, by a large margin? Phoronix ran 200 benchmarks on the 9950x vs 395x max and found a difference of less than 5%. Not bad considering the average power use was 91 watts vs 154 watts. If you need the memory bandwidth the strix halo looks good, if you are cache friendly and don't care about using almost double the power than the 9950x is a better deal.

jeffbee 1y ago
The way Phoronix weights an average score for a machine is ridiculous, because there aren't any users who do all of machine learning, fluid dynamics, video compression, database hosting, and software development, and games on the same machine. I looked at the applications that matter to me and the 9950X wins by 40% in those.