4 ms·

I don’t see the reason for the focus on the amount of memory given the potential theoretical advantages of a unified memory pool that extends beyond the DRAM. P

I don’t see the reason for the focus on the amount of memory given the potential theoretical advantages of a unified memory pool that extends beyond the DRAM. PCIe gen 4 SSDs have obscene data rates and tons of IOPs. In fact the whole SSD is now an extension of your RAM is a selling point of the new consoles. Is it as fast as having more DRAM in all possible scenarios? No, but with good memory management the real world performance might very well be identical to a system with the same capabilities and even greater than traditional memory systems with larger non-unified pools.

voldacar 6y ago
>Is it as fast as having more DRAM in all possible scenarios? Is it as fast as having more DRAM in any scenario?
- zepto 6y ago
  Obviously so based on Apple’s numbers.
  - dogma1138 6y ago
    Not just Apple we have multiple other examples of when a unified memory pool outperforms larger non unified memory pools just due to the overhead of having to copy memory manage multiple memory address spaces with all the translation and lookups you need to do etc. We have cheap consumer SSDs today offering 5GB/s of read performance and more than enough I/O to satisfy caching.
- dogma1138 6y ago
  Yes, and based on non Apple systems it’s faster than having much larger pools of non-unified memory. Essentially all the big compute vendors moving towards memory unification and cache coherent interfaces.
  - voldacar 6y ago
    Ok I misread the numbers somehow. This is interesting - I wonder what it means for the future of conventional desktop computing, when considering peripheral chips like GPUs that already have DMA capability
    - dogma1138 6y ago
      GPUs are going above DMA, DMA still needs memcopy and or address space translations which adds latency slows things down, with unified memory in theory you have a single address space and cache coherency across everything which means you can operate at the maximum bandwidth of all interfaces without wasting any cycles.
guenthert 6y ago
Well, DRAM still has 2+ orders of magnitude lower latency than SSDs. Whether that matters clearly depends on your application. On a desktop system switching tabs in your browser, swapping the content in from SSD will be fine. A ML application on a server accessing memory quasi randomly won't.
- dogma1138 6y ago
  Using SSDs for ML is what vendors like NVIDIA and AMD are trying to solve because this is a fallacy as you will never have as much RAM as you want / need compared to the size of your datasets. https://developer.nvidia.com/blog/gpudirect-storage/ https://developer.nvidia.com/blog/gpudirect-storage/ It’s the same fallacy as RAM size impacts video editing, it doesn’t. 8K RED raw is 4.374 terabytes per hour... at that point it doesn’t matter if you have 8, 16 or even 256GB of RAM there if your storage is too slow to support it you’ll drop frames and or have huge seek times, more memory won’t really help you there is no amount of prefetch you can do to bridge gaps that big. There is a huge advantage of having a unified memory architecture where the application can address a single memory address space to access your data and you do not need to perform memory copies, allocations and all that stuff between your storage, CPU and GPU...
  - nix23 6y ago
    >It’s the same fallacy as RAM size impacts video editing, it doesn’t. Computers are used for much more then just Video Editing...for Fluid Simulation Ram definitely has a impact. And Talking about SSD (QLC)...if they run out of SLC-Cache they are sometimes slower then HDD's. And i bet your Video editing is faster if you have 4TB of RAM.
  - zaphirplane 6y ago
    Wouldn’t having more ram give the process more runway to prefetch data into ram before it’s needed ? Without having new prefetch evicting unprocessed older prefetched data
    - dogma1138 6y ago
ajay-b 6y ago
Does shelf life play a role in this? My understanding was that SSD has a short lifespan [0] and so a focus on memory might be better for long term planning, and reliability. [0.] https://www.solarwindsmsp.com/blog/ssd-lifespan https://www.solarwindsmsp.com/blog/ssd-lifespan
- dogma1138 6y ago
  Not in any realistic sense. Also the big culprit in shortening the life of an SSD are writes, unified memory doesn’t mean that you have to write more often to the SSD in fact it means the opposite, unified memory isn’t a page file.
mpweiher 6y ago
This is absolutely a factor, though one I didn't focus on in this post. The sorts of large binary blobs that can take a lot of memory tend to be, well, large and contiguous, so perfect for moving between SSD and RAM at maximum speed. While I haven't seen benchmarks yet, the claim was 2x relative to their current systems, IIRC, which were already crazy fast at > 2GB/s. So it would probably do OK even just plain swapping, but really, really well moving these large blobs in and out, for example using mmap() and madvise().
- dogma1138 6y ago
  PCIe Gen4 gives you sequential 5GB/s reads and even random reads at well above 4GB/s, this isn’t a page file on an ATA133 drive. In theory this can also be extended over TB/USB4 and ofc it can be extended over the network (however that’s the slowest part) this going to be quite a big thing especially for use cases like video editing if they offer a true unified memory as a lot of the latency doesn’t necessarily comes from the interface latency but from memcopy and translating between multiple address spaces.

voldacar 6y ago

>Is it as fast as having more DRAM in all possible scenarios? Is it as fast as having more DRAM in any scenario?

zepto 6y ago
Obviously so based on Apple’s numbers.
- dogma1138 6y ago
  Not just Apple we have multiple other examples of when a unified memory pool outperforms larger non unified memory pools just due to the overhead of having to copy memory manage multiple memory address spaces with all the translation and lookups you need to do etc. We have cheap consumer SSDs today offering 5GB/s of read performance and more than enough I/O to satisfy caching.
dogma1138 6y ago
Yes, and based on non Apple systems it’s faster than having much larger pools of non-unified memory. Essentially all the big compute vendors moving towards memory unification and cache coherent interfaces.
- voldacar 6y ago
  Ok I misread the numbers somehow. This is interesting - I wonder what it means for the future of conventional desktop computing, when considering peripheral chips like GPUs that already have DMA capability
  - dogma1138 6y ago
    GPUs are going above DMA, DMA still needs memcopy and or address space translations which adds latency slows things down, with unified memory in theory you have a single address space and cache coherency across everything which means you can operate at the maximum bandwidth of all interfaces without wasting any cycles.

guenthert 6y ago

Well, DRAM still has 2+ orders of magnitude lower latency than SSDs. Whether that matters clearly depends on your application. On a desktop system switching tabs in your browser, swapping the content in from SSD will be fine. A ML application on a server accessing memory quasi randomly won't.

dogma1138 6y ago
Using SSDs for ML is what vendors like NVIDIA and AMD are trying to solve because this is a fallacy as you will never have as much RAM as you want / need compared to the size of your datasets. https://developer.nvidia.com/blog/gpudirect-storage/ https://developer.nvidia.com/blog/gpudirect-storage/ It’s the same fallacy as RAM size impacts video editing, it doesn’t. 8K RED raw is 4.374 terabytes per hour... at that point it doesn’t matter if you have 8, 16 or even 256GB of RAM there if your storage is too slow to support it you’ll drop frames and or have huge seek times, more memory won’t really help you there is no amount of prefetch you can do to bridge gaps that big. There is a huge advantage of having a unified memory architecture where the application can address a single memory address space to access your data and you do not need to perform memory copies, allocations and all that stuff between your storage, CPU and GPU...
- nix23 6y ago
  >It’s the same fallacy as RAM size impacts video editing, it doesn’t. Computers are used for much more then just Video Editing...for Fluid Simulation Ram definitely has a impact. And Talking about SSD (QLC)...if they run out of SLC-Cache they are sometimes slower then HDD's. And i bet your Video editing is faster if you have 4TB of RAM.
- zaphirplane 6y ago
  Wouldn’t having more ram give the process more runway to prefetch data into ram before it’s needed ? Without having new prefetch evicting unprocessed older prefetched data
  - dogma1138 6y ago

ajay-b 6y ago

Does shelf life play a role in this? My understanding was that SSD has a short lifespan [0] and so a focus on memory might be better for long term planning, and reliability. [0.] https://www.solarwindsmsp.com/blog/ssd-lifespan https://www.solarwindsmsp.com/blog/ssd-lifespan

dogma1138 6y ago
Not in any realistic sense. Also the big culprit in shortening the life of an SSD are writes, unified memory doesn’t mean that you have to write more often to the SSD in fact it means the opposite, unified memory isn’t a page file.

mpweiher 6y ago

This is absolutely a factor, though one I didn't focus on in this post. The sorts of large binary blobs that can take a lot of memory tend to be, well, large and contiguous, so perfect for moving between SSD and RAM at maximum speed. While I haven't seen benchmarks yet, the claim was 2x relative to their current systems, IIRC, which were already crazy fast at > 2GB/s. So it would probably do OK even just plain swapping, but really, really well moving these large blobs in and out, for example using mmap() and madvise().

dogma1138 6y ago
PCIe Gen4 gives you sequential 5GB/s reads and even random reads at well above 4GB/s, this isn’t a page file on an ATA133 drive. In theory this can also be extended over TB/USB4 and ofc it can be extended over the network (however that’s the slowest part) this going to be quite a big thing especially for use cases like video editing if they offer a true unified memory as a lot of the latency doesn’t necessarily comes from the interface latency but from memcopy and translating between multiple address spaces.