4 ms·

>Unifying it again, like with AMD's APUs, means either compromises for the CPU or for the GPU. There's no free lunch. I think the lunch here (it still ain't fr

>Unifying it again, like with AMD's APUs, means either compromises for the CPU or for the GPU. There's no free lunch. I think the lunch here (it still ain't free) is that RAM speed means nothing if you don't have enough RAM in the first place, and this is a compromise solution to that practical problem.

Rinzler89 3y ago
>if you don't have enough RAM in the first place Enough RAM for what task exactly? System RAM is plentiful and cheap nowadays(unless you buy Apple). I got new a laptop with 32GB RAM for about 750 Euros. But the speeds are too low for high-end gamming or LLM training for the poor APU.
- numpad0 3y ago
  Enough RAM for LLM. There are GPUs faster than M2 Ultra but can't run LLMs normally, which make that speed a moot point for LLM use-cases.
Dylan16807 3y ago
The real problem is a lack of competition in GPU production. GDDR is not very expensive. You should be able to get a GPU with a mid-level chip and tons of memory, but it's just not offered. Instead, please pay triple or quadruple the price of a high end gaming GPU to get a model with double the memory and basically the same core. The level of markup is astonishing. I can go from 8GB to 16GB on AMD for $60, but going from 24GB to 48GB costs $3000. And nvidia isn't better.
- zozbot234 3y ago
  > You should be able to get a GPU with a mid-level chip and tons of memory, but it's just not offered. Apple unified memory is the easiest way to get exactly that. There is a markup on memory upgrades but it's quite reasonable, not at all extreme.
  - Dylan16807 3y ago
    But I can't put that GPU into my existing machine, so I'm still paying $3000 extra if I don't want that Apple machine to be my computer.
- deleted 3y ago
  [deleted]
- nsteel 3y ago
  This isn't my area but won't it be quite expensive to use that GDDR? The PHY and the controller are complicated and you've got max 2GB devices, so if you want more memory you need a wider bus. That requires more beachfront and therefore a bigger die. That must make it expensive once you go beyond what you can fit on your small, cheap chip. Do their 24GB+ cards really use GDDR?* And you need to ensure you don't shoot yourself in the foot by making anything (relatively) cheap that could be useful for AI... *Edit: wow yeh, they do! A 384-bit interface on some! Sounds hot.
  - Dylan16807 3y ago
    The upgrade is actually remarkably similar. RX7600 and RX7600XT have 8GB or 16GB attached to a 128-bit bus. RX7900XT and W7900 have 24GB or 48GB attached to a 384-bit bus. Neither upgrade changes the bus width, only the memory chips. The two upgraded models even use the same memory chips, as far as I can tell! GDDR6, 18000MT/s, 2GB per 16 pins. I couldn't confirm chip count on the W7900 but it's the same density.
    - nsteel 3y ago
      Perhaps I should have been clearer and said "if you want more memory than the 16GB model you need a wider bus" but this confirms what I described above. - cheap gfx die with a 128-bit memory interface. - vastly more expensive gfx die with a 384-bit memory interface. Essentially there's a cheap upgrade option available for each die, swapping the 1GB memory chips for 2GB memory chips (GDDR doesn't support a mix). If you have the 16GB model and you want more memory, there are no bigger memory chips available, and so you need a wider bus and that's going to cost significantly more to produce and hence they charge more. As a side note, I would expect the the GDDR chips to be x32, rather than 16.
      - Dylan16807 3y ago
        > Essentially there's a cheap upgrade option available for each die Hmm, I think you missed what my actual point was. You can buy a card with the cheap die for $270. You can buy a card with the expensive die for $1000. So far, so good. The cost of just the memory upgrade for the cheap die is $60. The cost of just the memory upgrade for the expensive die is $3000. None of that $3000 is going toward upgrading the bus width. Maybe one percent of it goes toward upgrading the circuit board. It's a huge market segmentation fee. > As a side note, I would expect the the GDDR chips to be x32, rather than 16. The pictures I've found show the 7600 using 4 ram chips and the 7600 XT using 8 ram chips.
- paulmd 3y ago
  what exactly do you expect them to do about the routing? 384b gpus are as big as memory buses get in the modern era and that gives you 24gb capacity or 48gb when clamshelled. Higher-density 3GB modules that would allow 36GB/72GB have been repeatedly pushed back, which has kinda screwed over consoles as well - they are in the same bind with "insufficient" VRAM on MS and while sony is less awful they didn't give a VRAM increase with PS5 Pro either. GB202 might be going to 512b bus which is unprecedented in the modern era (nobody has done it since the Hawaii/GCN2 days) but that's really as big as anybody cares to route right now. What do you propose for going past that? Ironically AMD actually does have the capability to experiment and put bigger memory buses on smaller dies. And the MCM packaging actually does give you some physical fanout that makes the routing easier. But again, ultimately there is just no appetite for going to 512b or 768b buses at a design level, for very good technical reasons. like it just is what it is - GDDR is just not very dense, this is what you can get out of it. Production of HBM-based gpus is constrained by stacking capacity, which is the same reason people can't get enough datacenter cards in the first place. Higher-density LPDDR does exist, but you still have to route it, and bandwidth goes down quite a bit. And that's the Apple Silicon approach, which solves your problem but unfortunately a lot of people just flatly reject any offering from the Fruit Company for interpersonal reasons.
  - Dylan16807 3y ago
    > what exactly do you expect them to do about the routing? 384b gpus are as big as memory buses get in the modern era and that gives you 24gb capacity or 48gb when clamshelled. I'm not asking for more than that. I'm asking for that to be available on a mainstream consumer model. Most of the mid-tier GPUS from AMD and nVidia have 256 bit memory busses. I want 32GB on that bus at a reasonable price. Let's say $150 or $200 more than the 16GB version of the same model. I appreciate the information about higher densities, though.

Rinzler89 3y ago

>if you don't have enough RAM in the first place Enough RAM for what task exactly? System RAM is plentiful and cheap nowadays(unless you buy Apple). I got new a laptop with 32GB RAM for about 750 Euros. But the speeds are too low for high-end gamming or LLM training for the poor APU.

numpad0 3y ago
Enough RAM for LLM. There are GPUs faster than M2 Ultra but can't run LLMs normally, which make that speed a moot point for LLM use-cases.

Dylan16807 3y ago

The real problem is a lack of competition in GPU production. GDDR is not very expensive. You should be able to get a GPU with a mid-level chip and tons of memory, but it's just not offered. Instead, please pay triple or quadruple the price of a high end gaming GPU to get a model with double the memory and basically the same core. The level of markup is astonishing. I can go from 8GB to 16GB on AMD for $60, but going from 24GB to 48GB costs $3000. And nvidia isn't better.

zozbot234 3y ago
> You should be able to get a GPU with a mid-level chip and tons of memory, but it's just not offered. Apple unified memory is the easiest way to get exactly that. There is a markup on memory upgrades but it's quite reasonable, not at all extreme.
- Dylan16807 3y ago
  But I can't put that GPU into my existing machine, so I'm still paying $3000 extra if I don't want that Apple machine to be my computer.
deleted 3y ago
[deleted]
nsteel 3y ago
This isn't my area but won't it be quite expensive to use that GDDR? The PHY and the controller are complicated and you've got max 2GB devices, so if you want more memory you need a wider bus. That requires more beachfront and therefore a bigger die. That must make it expensive once you go beyond what you can fit on your small, cheap chip. Do their 24GB+ cards really use GDDR?* And you need to ensure you don't shoot yourself in the foot by making anything (relatively) cheap that could be useful for AI... *Edit: wow yeh, they do! A 384-bit interface on some! Sounds hot.
- Dylan16807 3y ago
  The upgrade is actually remarkably similar. RX7600 and RX7600XT have 8GB or 16GB attached to a 128-bit bus. RX7900XT and W7900 have 24GB or 48GB attached to a 384-bit bus. Neither upgrade changes the bus width, only the memory chips. The two upgraded models even use the same memory chips, as far as I can tell! GDDR6, 18000MT/s, 2GB per 16 pins. I couldn't confirm chip count on the W7900 but it's the same density.
  - nsteel 3y ago
    Perhaps I should have been clearer and said "if you want more memory than the 16GB model you need a wider bus" but this confirms what I described above. - cheap gfx die with a 128-bit memory interface. - vastly more expensive gfx die with a 384-bit memory interface. Essentially there's a cheap upgrade option available for each die, swapping the 1GB memory chips for 2GB memory chips (GDDR doesn't support a mix). If you have the 16GB model and you want more memory, there are no bigger memory chips available, and so you need a wider bus and that's going to cost significantly more to produce and hence they charge more. As a side note, I would expect the the GDDR chips to be x32, rather than 16.
    - Dylan16807 3y ago
      > Essentially there's a cheap upgrade option available for each die Hmm, I think you missed what my actual point was. You can buy a card with the cheap die for $270. You can buy a card with the expensive die for $1000. So far, so good. The cost of just the memory upgrade for the cheap die is $60. The cost of just the memory upgrade for the expensive die is $3000. None of that $3000 is going toward upgrading the bus width. Maybe one percent of it goes toward upgrading the circuit board. It's a huge market segmentation fee. > As a side note, I would expect the the GDDR chips to be x32, rather than 16. The pictures I've found show the 7600 using 4 ram chips and the 7600 XT using 8 ram chips.
paulmd 3y ago
what exactly do you expect them to do about the routing? 384b gpus are as big as memory buses get in the modern era and that gives you 24gb capacity or 48gb when clamshelled. Higher-density 3GB modules that would allow 36GB/72GB have been repeatedly pushed back, which has kinda screwed over consoles as well - they are in the same bind with "insufficient" VRAM on MS and while sony is less awful they didn't give a VRAM increase with PS5 Pro either. GB202 might be going to 512b bus which is unprecedented in the modern era (nobody has done it since the Hawaii/GCN2 days) but that's really as big as anybody cares to route right now. What do you propose for going past that? Ironically AMD actually does have the capability to experiment and put bigger memory buses on smaller dies. And the MCM packaging actually does give you some physical fanout that makes the routing easier. But again, ultimately there is just no appetite for going to 512b or 768b buses at a design level, for very good technical reasons. like it just is what it is - GDDR is just not very dense, this is what you can get out of it. Production of HBM-based gpus is constrained by stacking capacity, which is the same reason people can't get enough datacenter cards in the first place. Higher-density LPDDR does exist, but you still have to route it, and bandwidth goes down quite a bit. And that's the Apple Silicon approach, which solves your problem but unfortunately a lot of people just flatly reject any offering from the Fruit Company for interpersonal reasons.
- Dylan16807 3y ago
  > what exactly do you expect them to do about the routing? 384b gpus are as big as memory buses get in the modern era and that gives you 24gb capacity or 48gb when clamshelled. I'm not asking for more than that. I'm asking for that to be available on a mainstream consumer model. Most of the mid-tier GPUS from AMD and nVidia have 256 bit memory busses. I want 32GB on that bus at a reasonable price. Let's say $150 or $200 more than the 16GB version of the same model. I appreciate the information about higher densities, though.