4 ms·

> Moving high bandwidth RAM to be shared between the CPU & GPU There's a really weird amount of focus on this. Unified memory has been a thing for almost half

> Moving high bandwidth RAM to be shared between the CPU & GPU There's a really weird amount of focus on this. Unified memory has been a thing for almost half a decade now. It become a thing as soon as AMD stuck the CPU & GPU behind the same IOMMU ( https://en.wikipedia.org/wiki/Heterogeneous_System_Architecture https://en.wikipedia.org/wiki/Heterogeneous_System_Architect... ) Now you may ask, but why hasn't anyone leverage this before? And the answer is they have, but it's usually not worth doing because usually the integrated GPU isn't worth using for anything. As in, the market for HPC software that doesn't benefit from a discreet GPU is vanishingly small. So nobody really cares about it. But there are applications of this. Things like Intel's QuickSync should be internally leveraging this on the relevant CPUs, for example. So those sorts of frameworks (like MLKit) will just be a bit faster and cheaper to spin up. The rise in consumer software that does "HPC-like" workloads (eg, facial recognition, voice recognition, etc..) make this more interesting, but there's nothing new here, either. It's certainly not playing any meaningful role in the M1's current performance. The bigger question here would be to what extent does Apple push this. Like is Apple going to try and push integrated GPUs for the iMac Pro? What about the Mac Pro?

tjalfi 6y ago
Unified memory is quite a bit older than that; the SGI O2[0] had it in 1996. [0] https://en.wikipedia.org/wiki/SGI_O2 https://en.wikipedia.org/wiki/SGI_O2
- rasz 6y ago
  so did original Macintosh, and ZX Spectrum. Unified memory is not a good thing, makes the CPU and GPU fight for access.
  - SigmundA 6y ago
    However it also means zero copy between them too, there are benefits.
  - compiler-guy 6y ago
    The M1's memory is ported in such a way that there isn't contention for the memory. At least according to the article.
    - kllrnohj 6y ago
      Well, the article is basically just wrong on that, kinda. It is a single memory controller, which means the CPU & GPU are fighting for bandwidth. It's not an exclusive lock, no, but you will see a very sharp decline in memcpy performance if you're also hammering the GPU with blit commands. Which is the same as basically all integrated GPUs on all modern (or even kinda modern) CPUs. You don't usually have such heavy CPU memory bandwidth and GPU memory bandwidth workloads simultaneously, so it's mostly fine in practice, but there is technically resource contention there.
      - compiler-guy 6y ago
        Is this from real world observation, or just looking at the diagrams or what? Because by all accounts, these chips don’t seem to encounter any issues with the ui or encoding movies or whatever. Which would strongly imply that memory contention between the gpu and cpu is not a significant issue. If you have actual data that backs up your position, I would love to see it.
        kllrnohj 6y ago
        As I said, in most scenarios this doesn't matter, so I'm not sure why you're pointing at an average scenario as some sort of counter-argument? There's a single memory controller on the M1. The CPU, GPU, neural net engine, etc... all share that single controller (hence how "unified memory" is achieved). Given the theoretical maximum throughput of the M1's memory controller is 68GB/s, and that the CPU can hit around 68GB/s in a memcpy, I'm not sure what you're expecting? If you hammer the GPU at the same time, it must by design share that single 68GB/s pipe to the memory. There's not a secondary dedicated pipe for it to use. So the bandwidth must necessarily be split in a multi-use scenario, there's no other option here. Think of it like a network switch. You can put 5 computers behind a gigabit switch and 99% of the time nobody will ever have an issue. At any given point you can speedtest on one of them and see a full 1gbit/s on it. But if you speedtest on all of them simultaneously you of course won't see each one getting 1gbit/s since there's only a single 1gbit/s connection upstream. Same thing here, just with a 68GB/s 8x16-bit channel LPDDR4X memory controller. The only way you can get full-throughput CPU & GPU memory performance is if you physically dedicated memory modules to the CPU & GPU independently (such as with a typical desktop PC using discrete graphics). Otherwise they must compete for the shared resource by definition of being shared.
ogre_codes 6y ago
> There's a really weird amount of focus on this. Considering how much emphasis Apple put on it, I don't think it is weird at all. Apple made big claims about their CPU and attributed a significant chunk of that performance gain to their unified memory. Their claims about the CPU have proved largely accurate. Why would they fabricate the reasons behind that performance? It's not like they are throwing off competitors here.
- fomine3 6y ago
  It seems that some analysis overestimates UMA. It won't accelarate simple CPU performance and won't reduce RAM usage unless its data is also used on GPU. Possibly is they also call stacked RAM as "UMA" rather than shared RAM between CPU and GPU?
  - ogre_codes 6y ago
    What I find odd is how someone rushes out to "debunk" nearly everything Apple has said that they did to optimize the M1. It's clear the M1 is fast, so clearly some of the things they did to optimize it worked. Why Apple would lie about what those optimizations are is a head scratcher. It strikes me as odd that so many people claim there is zero benefit. I'm left wondering if people think Apple is lying about all of the actual reasons the CPU is fast and have presented this as a smokescreen. Makes zero sense to me.
    - kllrnohj 6y ago
      > Why Apple would lie about what those optimizations are is a head scratcher. Indeed it is. But marketing isn't really known for being technically accurate, why would Apple's be any exception? > It strikes me as odd that so many people claim there is zero benefit. That's not the claim at all. The claim is that unified memory isn't new, and there's a decent chance you already had it. For example, every Macbook Air of the last few years has been unified memory. The M1's unified memory is therefore a continuation of the existing norms & not something different. The M1's IPC is something different. The M1's cache latency is something different. The M1's 8-wide decoder is something different. There's a lot about the M1 that's different. Unified memory just isn't one of them, and unified memory still just doesn't improve CPU performance. All those CPU benchmarks that M1 is tearing up? It'd put up identical scores without unified memory, since none of those benchmarks involve copying/moving data between the CPU and a non-CPU coprocessor like the GPU or neural processor.
vvanders 6y ago
The whole focus on this makes no sense to me as well. UMA has been standard architecture for mobile SoCs since the dawn of time(aside from a few oddballs). Ditto tiling GPUs and the few other things I've seen called out. Heck the XBox 360 used UMA[1] and the PS3 didn't[2]. It really didn't play into the systems performance directly(other than you might do some crazy tricks like store audio data in VRAM and stream it back). With UMA you can get into cases where heavy CPU reads can impact other parts of the system because the memory controller is shared. [1] https://en.wikipedia.org/wiki/Xbox_360_technical_specifications#Memory_and_system_bandwidth https://en.wikipedia.org/wiki/Xbox_360_technical_specificati... [2] https://en.wikipedia.org/wiki/PlayStation_3_technical_specifications#Central_processing_unit https://en.wikipedia.org/wiki/PlayStation_3_technical_specif...
- kllrnohj 6y ago
  I've seen a few very rare edge cases where you can do cool tricks with UMA on mobile (which as you say have been UMA for years). For example on Android for VR you can have the sensor data stored in a chunk of memory that can also be read by the GPU, so your last-second time warp can have that smidge less latency by sampling the latest sensor data as hot off the sensor as it gets, without even bouncing off of the CPU. ( https://developer.android.com/ndk/reference/group/sensor#asensormanager_createhardwarebufferdirectchannel https://developer.android.com/ndk/reference/group/sensor#ase... ) But the best VR experiences are still done on non-UMA desktop PCs with discrete graphics so... At some point the slight efficiency wins of UMA are trumped by just the raw horsepower you get from multiple big dies.

tjalfi 6y ago

Unified memory is quite a bit older than that; the SGI O2[0] had it in 1996. [0] https://en.wikipedia.org/wiki/SGI_O2 https://en.wikipedia.org/wiki/SGI_O2

rasz 6y ago
so did original Macintosh, and ZX Spectrum. Unified memory is not a good thing, makes the CPU and GPU fight for access.
- SigmundA 6y ago
  However it also means zero copy between them too, there are benefits.
- compiler-guy 6y ago
  The M1's memory is ported in such a way that there isn't contention for the memory. At least according to the article.
  - kllrnohj 6y ago
    Well, the article is basically just wrong on that, kinda. It is a single memory controller, which means the CPU & GPU are fighting for bandwidth. It's not an exclusive lock, no, but you will see a very sharp decline in memcpy performance if you're also hammering the GPU with blit commands. Which is the same as basically all integrated GPUs on all modern (or even kinda modern) CPUs. You don't usually have such heavy CPU memory bandwidth and GPU memory bandwidth workloads simultaneously, so it's mostly fine in practice, but there is technically resource contention there.
    - compiler-guy 6y ago
      Is this from real world observation, or just looking at the diagrams or what? Because by all accounts, these chips don’t seem to encounter any issues with the ui or encoding movies or whatever. Which would strongly imply that memory contention between the gpu and cpu is not a significant issue. If you have actual data that backs up your position, I would love to see it.
      - kllrnohj 6y ago
        As I said, in most scenarios this doesn't matter, so I'm not sure why you're pointing at an average scenario as some sort of counter-argument? There's a single memory controller on the M1. The CPU, GPU, neural net engine, etc... all share that single controller (hence how "unified memory" is achieved). Given the theoretical maximum throughput of the M1's memory controller is 68GB/s, and that the CPU can hit around 68GB/s in a memcpy, I'm not sure what you're expecting? If you hammer the GPU at the same time, it must by design share that single 68GB/s pipe to the memory. There's not a secondary dedicated pipe for it to use. So the bandwidth must necessarily be split in a multi-use scenario, there's no other option here. Think of it like a network switch. You can put 5 computers behind a gigabit switch and 99% of the time nobody will ever have an issue. At any given point you can speedtest on one of them and see a full 1gbit/s on it. But if you speedtest on all of them simultaneously you of course won't see each one getting 1gbit/s since there's only a single 1gbit/s connection upstream. Same thing here, just with a 68GB/s 8x16-bit channel LPDDR4X memory controller. The only way you can get full-throughput CPU & GPU memory performance is if you physically dedicated memory modules to the CPU & GPU independently (such as with a typical desktop PC using discrete graphics). Otherwise they must compete for the shared resource by definition of being shared.

ogre_codes 6y ago

> There's a really weird amount of focus on this. Considering how much emphasis Apple put on it, I don't think it is weird at all. Apple made big claims about their CPU and attributed a significant chunk of that performance gain to their unified memory. Their claims about the CPU have proved largely accurate. Why would they fabricate the reasons behind that performance? It's not like they are throwing off competitors here.

fomine3 6y ago
It seems that some analysis overestimates UMA. It won't accelarate simple CPU performance and won't reduce RAM usage unless its data is also used on GPU. Possibly is they also call stacked RAM as "UMA" rather than shared RAM between CPU and GPU?
- ogre_codes 6y ago
  What I find odd is how someone rushes out to "debunk" nearly everything Apple has said that they did to optimize the M1. It's clear the M1 is fast, so clearly some of the things they did to optimize it worked. Why Apple would lie about what those optimizations are is a head scratcher. It strikes me as odd that so many people claim there is zero benefit. I'm left wondering if people think Apple is lying about all of the actual reasons the CPU is fast and have presented this as a smokescreen. Makes zero sense to me.
  - kllrnohj 6y ago
    > Why Apple would lie about what those optimizations are is a head scratcher. Indeed it is. But marketing isn't really known for being technically accurate, why would Apple's be any exception? > It strikes me as odd that so many people claim there is zero benefit. That's not the claim at all. The claim is that unified memory isn't new, and there's a decent chance you already had it. For example, every Macbook Air of the last few years has been unified memory. The M1's unified memory is therefore a continuation of the existing norms & not something different. The M1's IPC is something different. The M1's cache latency is something different. The M1's 8-wide decoder is something different. There's a lot about the M1 that's different. Unified memory just isn't one of them, and unified memory still just doesn't improve CPU performance. All those CPU benchmarks that M1 is tearing up? It'd put up identical scores without unified memory, since none of those benchmarks involve copying/moving data between the CPU and a non-CPU coprocessor like the GPU or neural processor.

vvanders 6y ago

The whole focus on this makes no sense to me as well. UMA has been standard architecture for mobile SoCs since the dawn of time(aside from a few oddballs). Ditto tiling GPUs and the few other things I've seen called out. Heck the XBox 360 used UMA[1] and the PS3 didn't[2]. It really didn't play into the systems performance directly(other than you might do some crazy tricks like store audio data in VRAM and stream it back). With UMA you can get into cases where heavy CPU reads can impact other parts of the system because the memory controller is shared. [1] https://en.wikipedia.org/wiki/Xbox_360_technical_specifications#Memory_and_system_bandwidth https://en.wikipedia.org/wiki/Xbox_360_technical_specificati... [2] https://en.wikipedia.org/wiki/PlayStation_3_technical_specifications#Central_processing_unit https://en.wikipedia.org/wiki/PlayStation_3_technical_specif...

kllrnohj 6y ago
I've seen a few very rare edge cases where you can do cool tricks with UMA on mobile (which as you say have been UMA for years). For example on Android for VR you can have the sensor data stored in a chunk of memory that can also be read by the GPU, so your last-second time warp can have that smidge less latency by sampling the latest sensor data as hot off the sensor as it gets, without even bouncing off of the CPU. ( https://developer.android.com/ndk/reference/group/sensor#asensormanager_createhardwarebufferdirectchannel https://developer.android.com/ndk/reference/group/sensor#ase... ) But the best VR experiences are still done on non-UMA desktop PCs with discrete graphics so... At some point the slight efficiency wins of UMA are trumped by just the raw horsepower you get from multiple big dies.