5 ms·

Although I think the premise of the article is wrong, it is nicely written. A major nitpick: unified memory is being massively over hyped. There is a reason GP

Although I think the premise of the article is wrong, it is nicely written. A major nitpick: unified memory is being massively over hyped. There is a reason GPUs have their own memory bus -- contention. CPU/GPUs fighting over access to memory causes massive disruption to very parallel computation. Even if Intel/nVidia resolved their fight over inter-CPU connectivity or we're talking POWER and nVidia using NV-LINK, you still need extra memory ports to keep things fed. The more cores and the faster the GPU the more memory bandwidth required. I expect to see future Mac Pro M1 series machines with multiple CPU sockets -- at which point memory isn't unified any more, and all the regular CC-NUMA tricks will be used. But it won't be a big deal.

forrestthewoods 6y ago
> unified memory is being massively over hyped. There is a reason GPUs have their own memory bus -- contention. CPU/GPUs fighting over access to memory causes massive disruption to very parallel computation. Not sure I totally agree with this. Game console have unified memory architecture and it’s a beloved feature. It greatly simplifies things and allows the CPU to far more easily use results computed by the GPU with complicated sync commands or frame delays. Maybe unified architecture is less valuable for non-interactive programs. I’m not sure. This is a fair bit outside of my wheelhouse. Memory access is definitely one of the biggest bottlenecks. So I fully agree with the general concern. And you may even be right that the unified architecture isn’t that interesting. But I’m not so sure it’s the problem you think it is.
- angry_octet 6y ago
  It does indeed make it easier to code. But you may have noticed that Xbox X has moved to having different speed memories with different memory controllers, even if logically contiguous and uniformly accessible. The CPU pool has a narrower bus and can't be used for graphics objects -- you have to queue for copy to the graphics pool. The driver then schedules a deconflicted copy. The PS5 architecture also has different speeds -- but the slow access is to SSD! It has manual management of streaming resources from SSD to RAM, but also allows direct SSD reads -- but that is a trick, because the SSD controller has a huge RAM cache too. So I'd say we're actually moving away from UMA in general. I think that memory aware scheduling is going to be the next win -- online learning to understand memory access patterns and scheduling compute and cache fill. Fancy cache algorithms used to take too much logic (and slow cache fill logic down), but for SSD->RAM you can do lots of prediction based on program state.
  - forrestthewoods 6y ago
    Interesting. Sadly I’ve not gotten to dev with a PS5 or XSX yet. I’d subscribe to your newsletter.
floatboth 6y ago
Apple is super fucking rich, I wouldn't be surprised if they are ready to pay TSMC enormous money to make monster monolithic dies, and Mac Pro could be single-socket only, packing >128 cores into one socket.
- angry_octet 6y ago
  You need lots of memory too. Maybe not an issue for the desktop. If they doubled the SOC for 16 cores (8 hot/8 cool) and 32GB of RAM that would be a decent Mac Pro. But plenty of people want >32GB of memory, I routinely use machines with 256GB. No way you can get enough RAM into the SOC. Large core counts are even harder, because speed of light means you need a new on socket switched architecture, memory on the other side is slow to access etc. TANSTAAFL.
  - floatboth 6y ago
    They probably won't use DDR-on-package for Mac Pro tier chips, so RAM could be whatever they want, they could put an 8-channel DDR4 controller onto the SoC, or even DDR5 if they don't rush it to market ASAP. But it would be cool if someone did a tiered HBM+DDR system already.. (AFAIK Fujitsu A64FX is HBM-only) maybe the Mac Pro could be that. I kinda wish AMD did that though.
socialdemocrat 6y ago
This article discuss exactly what you are talking about and why it is not an issue: https://debugger.medium.com/why-is-apples-m1-chip-so-fast-3262b158cba2 https://debugger.medium.com/why-is-apples-m1-chip-so-fast-32...
- angry_octet 6y ago
  I did indeed read that, but it has nothing real to say about it. Instead it says this: > Apple uses memory which serves both large chunks of data and serves it fast. In computer speak that is called low latency and high throughput. Thus the need to be connected to separate types of memory is removed. That is just hand waving. It is possible to produce such memory, but it involves ultra wide busses, far wider than optimal for filling CPU caches, and preferably directly connected to the GPU rather than a multi-master bus or switch. There is the possibility that Apple has built a very fancy memory interposer that leverages the short distances in the SOC to present the memory both wide (to GPU) and narrow (for filling a queue of L2 misses), so that cache fills pause while GPUs read/write. That would be a highly interesting piece of logic. But of course it can't scale outside of the SOC.
  - socialdemocrat 6y ago
    Well yeah they are exploiting the fact that they have an SoC. And they don’t need to scale outside the SoC. There is still space to add a lot more memory.

forrestthewoods 6y ago

> unified memory is being massively over hyped. There is a reason GPUs have their own memory bus -- contention. CPU/GPUs fighting over access to memory causes massive disruption to very parallel computation. Not sure I totally agree with this. Game console have unified memory architecture and it’s a beloved feature. It greatly simplifies things and allows the CPU to far more easily use results computed by the GPU with complicated sync commands or frame delays. Maybe unified architecture is less valuable for non-interactive programs. I’m not sure. This is a fair bit outside of my wheelhouse. Memory access is definitely one of the biggest bottlenecks. So I fully agree with the general concern. And you may even be right that the unified architecture isn’t that interesting. But I’m not so sure it’s the problem you think it is.

angry_octet 6y ago
It does indeed make it easier to code. But you may have noticed that Xbox X has moved to having different speed memories with different memory controllers, even if logically contiguous and uniformly accessible. The CPU pool has a narrower bus and can't be used for graphics objects -- you have to queue for copy to the graphics pool. The driver then schedules a deconflicted copy. The PS5 architecture also has different speeds -- but the slow access is to SSD! It has manual management of streaming resources from SSD to RAM, but also allows direct SSD reads -- but that is a trick, because the SSD controller has a huge RAM cache too. So I'd say we're actually moving away from UMA in general. I think that memory aware scheduling is going to be the next win -- online learning to understand memory access patterns and scheduling compute and cache fill. Fancy cache algorithms used to take too much logic (and slow cache fill logic down), but for SSD->RAM you can do lots of prediction based on program state.
- forrestthewoods 6y ago
  Interesting. Sadly I’ve not gotten to dev with a PS5 or XSX yet. I’d subscribe to your newsletter.

floatboth 6y ago

Apple is super fucking rich, I wouldn't be surprised if they are ready to pay TSMC enormous money to make monster monolithic dies, and Mac Pro could be single-socket only, packing >128 cores into one socket.

angry_octet 6y ago
You need lots of memory too. Maybe not an issue for the desktop. If they doubled the SOC for 16 cores (8 hot/8 cool) and 32GB of RAM that would be a decent Mac Pro. But plenty of people want >32GB of memory, I routinely use machines with 256GB. No way you can get enough RAM into the SOC. Large core counts are even harder, because speed of light means you need a new on socket switched architecture, memory on the other side is slow to access etc. TANSTAAFL.
- floatboth 6y ago
  They probably won't use DDR-on-package for Mac Pro tier chips, so RAM could be whatever they want, they could put an 8-channel DDR4 controller onto the SoC, or even DDR5 if they don't rush it to market ASAP. But it would be cool if someone did a tiered HBM+DDR system already.. (AFAIK Fujitsu A64FX is HBM-only) maybe the Mac Pro could be that. I kinda wish AMD did that though.

socialdemocrat 6y ago

This article discuss exactly what you are talking about and why it is not an issue: https://debugger.medium.com/why-is-apples-m1-chip-so-fast-3262b158cba2 https://debugger.medium.com/why-is-apples-m1-chip-so-fast-32...

angry_octet 6y ago
I did indeed read that, but it has nothing real to say about it. Instead it says this: > Apple uses memory which serves both large chunks of data and serves it fast. In computer speak that is called low latency and high throughput. Thus the need to be connected to separate types of memory is removed. That is just hand waving. It is possible to produce such memory, but it involves ultra wide busses, far wider than optimal for filling CPU caches, and preferably directly connected to the GPU rather than a multi-master bus or switch. There is the possibility that Apple has built a very fancy memory interposer that leverages the short distances in the SOC to present the memory both wide (to GPU) and narrow (for filling a queue of L2 misses), so that cache fills pause while GPUs read/write. That would be a highly interesting piece of logic. But of course it can't scale outside of the SOC.
- socialdemocrat 6y ago
  Well yeah they are exploiting the fact that they have an SoC. And they don’t need to scale outside the SoC. There is still space to add a lot more memory.