7 ms·

Old becomes new, the SGI O2 had (off chip) a unified memory model for performance reasons. Not a CS guy, but it seems to me that NUMA like architecture has to

Old becomes new, the SGI O2 had (off chip) a unified memory model for performance reasons. Not a CS guy, but it seems to me that NUMA like architecture has to come back. Large RAM on chip (balancing a thermal budget between #ofcores vs ram), a much larger RAM off chip and even more RAM through a fast interconnect on a single kernel image. Like the Origin 300 had.

Rinzler89 3y ago
UMA in the SGI machines (and gaming consoles) made sense because all the memory chips at that time were equally slow, or fast, depending how you wanna look at it. PC HW split the video memory from system memory once GDDRAM become so much faster than system RAM, but GDDRAM has too high latency for CPUs and DDR has too low bandwidth for GPUs, so the separation made sense for each's strengths and still does to this day. Unifying it again, like with AMD's APUs, means either compromises for the CPU or for the GPU. There's no free lunch. Currently AMD APUs on the PC use unified DDRAM so CPU performance is top but GPU/NPU perforce is bottlenecked. If they were to use unified GDDRAM like in the PS5/Xbox then GPU/NPU performance would be top and CPU performance would be bottlenecked.
- Dalewyn 3y ago
  >Unifying it again, like with AMD's APUs, means either compromises for the CPU or for the GPU. There's no free lunch. I think the lunch here (it still ain't free) is that RAM speed means nothing if you don't have enough RAM in the first place, and this is a compromise solution to that practical problem.
  - Rinzler89 3y ago
    >if you don't have enough RAM in the first place Enough RAM for what task exactly? System RAM is plentiful and cheap nowadays(unless you buy Apple). I got new a laptop with 32GB RAM for about 750 Euros. But the speeds are too low for high-end gamming or LLM training for the poor APU.
    - numpad0 3y ago
      Enough RAM for LLM. There are GPUs faster than M2 Ultra but can't run LLMs normally, which make that speed a moot point for LLM use-cases.
  - Dylan16807 3y ago
    The real problem is a lack of competition in GPU production. GDDR is not very expensive. You should be able to get a GPU with a mid-level chip and tons of memory, but it's just not offered. Instead, please pay triple or quadruple the price of a high end gaming GPU to get a model with double the memory and basically the same core. The level of markup is astonishing. I can go from 8GB to 16GB on AMD for $60, but going from 24GB to 48GB costs $3000. And nvidia isn't better.
    - zozbot234 3y ago
      > You should be able to get a GPU with a mid-level chip and tons of memory, but it's just not offered. Apple unified memory is the easiest way to get exactly that. There is a markup on memory upgrades but it's quite reasonable, not at all extreme.
      - Dylan16807 3y ago
        But I can't put that GPU into my existing machine, so I'm still paying $3000 extra if I don't want that Apple machine to be my computer.
    - deleted 3y ago
      [deleted]
    - nsteel 3y ago
      This isn't my area but won't it be quite expensive to use that GDDR? The PHY and the controller are complicated and you've got max 2GB devices, so if you want more memory you need a wider bus. That requires more beachfront and therefore a bigger die. That must make it expensive once you go beyond what you can fit on your small, cheap chip. Do their 24GB+ cards really use GDDR?* And you need to ensure you don't shoot yourself in the foot by making anything (relatively) cheap that could be useful for AI... *Edit: wow yeh, they do! A 384-bit interface on some! Sounds hot.
      - Dylan16807 3y ago
        The upgrade is actually remarkably similar. RX7600 and RX7600XT have 8GB or 16GB attached to a 128-bit bus. RX7900XT and W7900 have 24GB or 48GB attached to a 384-bit bus. Neither upgrade changes the bus width, only the memory chips. The two upgraded models even use the same memory chips, as far as I can tell! GDDR6, 18000MT/s, 2GB per 16 pins. I couldn't confirm chip count on the W7900 but it's the same density.
- numpad0 3y ago
  I suspect there are difficulties with DRAM latency and/or signal integrity with APUs and RAM-expandable GPUs. Wasted ALUs are wasted if you'd be stalling deep SIMD pipelines all the time.
- fulafel 3y ago
  The fancier and more expensive SGIs had higher bandwisth non UMA memory systems on the GPUs. The thing about bandwidth is that you can just make wider buses with more of the same memory chips in parallel.
aidenn0 3y ago
Several PC graphics standards attempted to offer high-speed access to system memory (though I think only VLB offered direct access to the memory controller at the same speed as the CPU). Not having to have dedicated GPU memory has obvious advantages, but it's hard to get right.
Dylan16807 3y ago
> Old becomes new I disagree. This is like pointing at a 2024 model hatchback and saying "old becomes new" because you can cite a hatchback from 50 years ago. There's a bevy of ways to isolate or combine memory pools, and mainstream hardware has consistently used many of these methods the entire time.

Rinzler89 3y ago

UMA in the SGI machines (and gaming consoles) made sense because all the memory chips at that time were equally slow, or fast, depending how you wanna look at it. PC HW split the video memory from system memory once GDDRAM become so much faster than system RAM, but GDDRAM has too high latency for CPUs and DDR has too low bandwidth for GPUs, so the separation made sense for each's strengths and still does to this day. Unifying it again, like with AMD's APUs, means either compromises for the CPU or for the GPU. There's no free lunch. Currently AMD APUs on the PC use unified DDRAM so CPU performance is top but GPU/NPU perforce is bottlenecked. If they were to use unified GDDRAM like in the PS5/Xbox then GPU/NPU performance would be top and CPU performance would be bottlenecked.

Dalewyn 3y ago
>Unifying it again, like with AMD's APUs, means either compromises for the CPU or for the GPU. There's no free lunch. I think the lunch here (it still ain't free) is that RAM speed means nothing if you don't have enough RAM in the first place, and this is a compromise solution to that practical problem.
- Rinzler89 3y ago
  >if you don't have enough RAM in the first place Enough RAM for what task exactly? System RAM is plentiful and cheap nowadays(unless you buy Apple). I got new a laptop with 32GB RAM for about 750 Euros. But the speeds are too low for high-end gamming or LLM training for the poor APU.
  - numpad0 3y ago
    Enough RAM for LLM. There are GPUs faster than M2 Ultra but can't run LLMs normally, which make that speed a moot point for LLM use-cases.
- Dylan16807 3y ago
  The real problem is a lack of competition in GPU production. GDDR is not very expensive. You should be able to get a GPU with a mid-level chip and tons of memory, but it's just not offered. Instead, please pay triple or quadruple the price of a high end gaming GPU to get a model with double the memory and basically the same core. The level of markup is astonishing. I can go from 8GB to 16GB on AMD for $60, but going from 24GB to 48GB costs $3000. And nvidia isn't better.
  - zozbot234 3y ago
    > You should be able to get a GPU with a mid-level chip and tons of memory, but it's just not offered. Apple unified memory is the easiest way to get exactly that. There is a markup on memory upgrades but it's quite reasonable, not at all extreme.
    - Dylan16807 3y ago
      But I can't put that GPU into my existing machine, so I'm still paying $3000 extra if I don't want that Apple machine to be my computer.
  - deleted 3y ago
    [deleted]
  - nsteel 3y ago
    This isn't my area but won't it be quite expensive to use that GDDR? The PHY and the controller are complicated and you've got max 2GB devices, so if you want more memory you need a wider bus. That requires more beachfront and therefore a bigger die. That must make it expensive once you go beyond what you can fit on your small, cheap chip. Do their 24GB+ cards really use GDDR?* And you need to ensure you don't shoot yourself in the foot by making anything (relatively) cheap that could be useful for AI... *Edit: wow yeh, they do! A 384-bit interface on some! Sounds hot.
    - Dylan16807 3y ago
      The upgrade is actually remarkably similar. RX7600 and RX7600XT have 8GB or 16GB attached to a 128-bit bus. RX7900XT and W7900 have 24GB or 48GB attached to a 384-bit bus. Neither upgrade changes the bus width, only the memory chips. The two upgraded models even use the same memory chips, as far as I can tell! GDDR6, 18000MT/s, 2GB per 16 pins. I couldn't confirm chip count on the W7900 but it's the same density.
numpad0 3y ago
I suspect there are difficulties with DRAM latency and/or signal integrity with APUs and RAM-expandable GPUs. Wasted ALUs are wasted if you'd be stalling deep SIMD pipelines all the time.
fulafel 3y ago
The fancier and more expensive SGIs had higher bandwisth non UMA memory systems on the GPUs. The thing about bandwidth is that you can just make wider buses with more of the same memory chips in parallel.

aidenn0 3y ago

Several PC graphics standards attempted to offer high-speed access to system memory (though I think only VLB offered direct access to the memory controller at the same speed as the CPU). Not having to have dedicated GPU memory has obvious advantages, but it's hard to get right.

Dylan16807 3y ago

> Old becomes new I disagree. This is like pointing at a 2024 model hatchback and saying "old becomes new" because you can cite a hatchback from 50 years ago. There's a bevy of ways to isolate or combine memory pools, and mainstream hardware has consistently used many of these methods the entire time.