5 ms·

Now all it needs is more memory bandwidth, because those two memory channels on the consumer AM5 socket are pathetic given the compute this will deliver, and es

Now all it needs is more memory bandwidth, because those two memory channels on the consumer AM5 socket are pathetic given the compute this will deliver, and especially in comparison with even the most basic ASi. I moved to an M2 Max from a chunky Zen setup and it's a revelation how much the memory bandwidth improvement accelerates intensive data work. Also for heavy-ish multitasking the Zen setup's narrow memory pipe would often choke.

AnthonyMouse 2y ago
There are very few applications that are actually memory bandwidth bound but aren't more suited to a GPU than a CPU. The reason people have been looking at Apple Silicon for LLMs in particular is that even though they are more suited to GPUs, they also require a lot of VRAM and NVIDIA charges an extortionate amount of money for GPUs with a lot of VRAM. What AMD should really do if they want to steal NVIDIA's thunder is to sell consumer GPUs with 64-128GB of VRAM.
- The_Colonel 2y ago
  > What AMD should really do if they want to steal NVIDIA's thunder is to sell consumer GPUs with 64-128GB of VRAM. For gaming, it's an overkill (esp. given the price), it would be useful only for LLM enthusiasts which isn't that big of a market.
  - AnthonyMouse 2y ago
    Then how come so many people are buying GPUs with that amount of VRAM for extraordinary amounts of money? Nobody said it was for gaming. Though of course consumers could use it for both.
    - kimixa 2y ago
      But AMD already offer GPUs with that amount of VRAM for extraordinary amounts of money, so there'll be no change? And from what I hear they're selling every one they're producing, even to the level where there's a ~6 month waiting list. So even if they reduced the price, the only difference would be that the list would get longer and AMD got less money. No more devices would actually make into people's hands, being production limited.
      - AnthonyMouse 2y ago
        The production limit isn't in the amount of VRAM. They could offer any of their existing consumer GPUs in versions with more VRAM with prices that add twice their own cost for the extra VRAM (but still far less than the datacenter versions) and thereby sell the same GPUs for higher margins. This might even leave gamers with more GPUs, because right now there are people buying e.g. four 24GB consumer GPUs to run a large model, and they could instead buy one 96GB GPU and leave three more for gamers.
        kimixa 2y ago
        You're not really saying "They're not offering a product" - so much "They're not offering a product at a price I want", which is really a very different statement. They're intentionally differentiated in the market and marked up due to the relatively higher R&D costs from targeting a much smaller market niche, with it's own demands on development. I doubt you'll just be running games on them, after all. If that is your niche, AMD is effectively saying they don't want your custom at that price. No point selling products that lose money, after all. AMD have done it before - arguably are doing it right now as their graphics BU lost money last year if you exclude APUs and consoles - and it went badly each time (despite what some people say about "Market Share!" on some internet forums).
        AnthonyMouse 2y ago
        That strategy doesn't win though. The enterprise is going to buy the enterprise GPU anyway because they're spending someone else's money and it's a tax write off and the enterprise GPUs are faster and have ECC memory in addition to having more VRAM. But giving something for hobbyists to play with is how you build an ecosystem. So not only do they get higher margins on their consumer GPUs, they get more code written for their architecture, which lets them sell more of the expensive GPUs. Not doing that is making a mistake.
cesarb 2y ago
> because those two memory channels on the consumer AM5 socket There are actually four memory channels in AM5, because DDR5 doubles the number of channels.
- pezezin 2y ago
  But each channel is half the width, so we are back at the same point.

AnthonyMouse 2y ago

There are very few applications that are actually memory bandwidth bound but aren't more suited to a GPU than a CPU. The reason people have been looking at Apple Silicon for LLMs in particular is that even though they are more suited to GPUs, they also require a lot of VRAM and NVIDIA charges an extortionate amount of money for GPUs with a lot of VRAM. What AMD should really do if they want to steal NVIDIA's thunder is to sell consumer GPUs with 64-128GB of VRAM.

The_Colonel 2y ago
> What AMD should really do if they want to steal NVIDIA's thunder is to sell consumer GPUs with 64-128GB of VRAM. For gaming, it's an overkill (esp. given the price), it would be useful only for LLM enthusiasts which isn't that big of a market.
- AnthonyMouse 2y ago
  Then how come so many people are buying GPUs with that amount of VRAM for extraordinary amounts of money? Nobody said it was for gaming. Though of course consumers could use it for both.
  - kimixa 2y ago
    But AMD already offer GPUs with that amount of VRAM for extraordinary amounts of money, so there'll be no change? And from what I hear they're selling every one they're producing, even to the level where there's a ~6 month waiting list. So even if they reduced the price, the only difference would be that the list would get longer and AMD got less money. No more devices would actually make into people's hands, being production limited.
    - AnthonyMouse 2y ago
      The production limit isn't in the amount of VRAM. They could offer any of their existing consumer GPUs in versions with more VRAM with prices that add twice their own cost for the extra VRAM (but still far less than the datacenter versions) and thereby sell the same GPUs for higher margins. This might even leave gamers with more GPUs, because right now there are people buying e.g. four 24GB consumer GPUs to run a large model, and they could instead buy one 96GB GPU and leave three more for gamers.
      - kimixa 2y ago
        You're not really saying "They're not offering a product" - so much "They're not offering a product at a price I want", which is really a very different statement. They're intentionally differentiated in the market and marked up due to the relatively higher R&D costs from targeting a much smaller market niche, with it's own demands on development. I doubt you'll just be running games on them, after all. If that is your niche, AMD is effectively saying they don't want your custom at that price. No point selling products that lose money, after all. AMD have done it before - arguably are doing it right now as their graphics BU lost money last year if you exclude APUs and consoles - and it went badly each time (despite what some people say about "Market Share!" on some internet forums).
        AnthonyMouse 2y ago
        That strategy doesn't win though. The enterprise is going to buy the enterprise GPU anyway because they're spending someone else's money and it's a tax write off and the enterprise GPUs are faster and have ECC memory in addition to having more VRAM. But giving something for hobbyists to play with is how you build an ecosystem. So not only do they get higher margins on their consumer GPUs, they get more code written for their architecture, which lets them sell more of the expensive GPUs. Not doing that is making a mistake.

cesarb 2y ago

> because those two memory channels on the consumer AM5 socket There are actually four memory channels in AM5, because DDR5 doubles the number of channels.

pezezin 2y ago
But each channel is half the width, so we are back at the same point.