4 ms·

Apple's ridiculously expensive fast & wide on-package memory is a big reason why the M series chips are so fast. Expect AMD & Intel to adopt a similar strateg

Apple's ridiculously expensive fast & wide on-package memory is a big reason why the M series chips are so fast. Expect AMD & Intel to adopt a similar strategy in their high end chips soon.

jsheard 3y ago
Intel is already there at the ultra-high-end, the Xeon Max is a hybrid with 64GB of HBM2 on-package and an external DDR5 bus supporting up to 4TB of socketed RAM. The HBM2 can be configured to work as L4 cache for the DDR5, or it can boot without any DDR5 at all and run entirely from the HBM2. https://www.servethehome.com/intel-xeon-max-9480-deep-dive-intel-has-64gb-hbm2e-onboard-like-a-gpu-or-ai-accelerator/ https://www.servethehome.com/intel-xeon-max-9480-deep-dive-i...
- bryanlarsen 3y ago
  MI300X also has on-package RAM, but that's even further from a laptop chip than Xeon Max. :)
- jauntywundrkind 3y ago
  Intel's Lakefield also comes to mind, pretty underrated chip imo. The first x86 big/small heterogenous chip, using Intel's Foveros to stack memory, CPU, and platform chip into one package. It made for a very simple design that just needed power. LPDDR4X-4266 doesn't crazy fast but in 2020 it was pretty swift. https://www.anandtech.com/show/15877/intel-hybrid-cpu-lakefield-all-you-need-to-know/3 https://www.anandtech.com/show/15877/intel-hybrid-cpu-lakefi... Alas the 1+4 Skylake+atom were all pretty meh chips & Intel hasn't tried again. Conceptually I really.liked it, and it was doing great against the Qualcomm 8cx competitor for the tablets/2-in-1s it was in, but people wanted more.
- bastard_op 3y ago
  I remember seeing this from STH as well, and my first thought was "I want one of these in my laptop!" It'd be perfect then to max ram in a laptop with 2 sodimm slots, or one of these 64gb lpcamm's. Ideally with a thinner laptop in mind, but probably not with the heatsink for THAT massive xeon. I really hope the hbm memory on-die trickles down into pro-sumer processor chips eventually too.
jeswin 3y ago
> Apple's ridiculously expensive fast & wide on-package memory is a big reason why the M series chips are so fast Increasing memory speeds have had only a marginal impact on common workloads (not AI) in the PC World - this has been a well discussed topic. Why would it be different on the Mac?
- __MatrixMan__ 3y ago
  probably something to do with rounded rectangles
- cogman10 3y ago
  Memory speed usually refers to memory bandwidth, which does have a limited amount of impact on day to day apps as they don't usually require a lot of bandwidth. Memory Latency, on the other hand, can have a huge impact on performance. That's the reason chips have had steadily increasing cache sizes over the decades along with the introduction of additional caching layers. What Apple's M series improves by putting the memory on chip is both bandwidth and latency. The latency, however, is what will impact app performance more than anything else. Latency improvements are the reason memory controllers were moved from motherboard northbridges into the CPU IC. Shortening that distance means a lot to shortening latency.
  - vlovich123 3y ago
    Amusingly the top-line latency number is higher than for X86, but I agree with you because that number hides the better performance in other latency metrics that are probably more important.
- vlovich123 3y ago
  It's not about memory bandwidth. [1] shows that M1 is 66x faster at pointer chasing than an equivalent AMD. You can look at [2] and [3] to see that TLB penalty is also lower at any given test depth which is important because TLB misses aren't that infrequent and they're serial in your ability to access any memory that's not in your cache. It's not an unreasonable hypothesis that this is because of on-package memory which means you can run with tighter thresholds since your memory traces are shorter. Their unified memory architecture also means that GPU is way more effective since you never need to copy memory from the CPU to GPU. That's probably not a huge deal for normal apps, but having rendering the screen not competing with any app resources for memory probably isn't nothing. Neither the CPU nor the GPU can fully utilize the available memory bandwidth [4] The insane memory bandwidth is to also support the other coprocessors like the media engine & to ensure that you can do a bunch of tasks in parallel without them contending with each for resources. Those workloads do benefit greatly from memory bandwidth & the "artist" community is hugely important to Apple. The reason M is overall faster is more because of things like better ILP (e.g. fixed-width RISC encoding vs compression-like CISC means easier to keep the pipeline fed). Apple software for the most part is better engineered for performance & the M chip is optimized for that performance profile (e.g. C# and Java vs Swift/ObjC - Rc is pretty conclusively faster & requires less memory overall not to mention that the M chip has specific optimizations to further reduce the traditional cost of Rc that they have to pay because they don't typically distinguish Rc from Arc as Rust does). The final truth is that Apple throws a lot of money at this problem as well & buy up the latest processing node to stay ahead of the competition. [1] https://www.linkedin.com/pulse/apple-m-cpus-probably-much-faster-than-we-realize-yaron-shani/ https://www.linkedin.com/pulse/apple-m-cpus-probably-much-fa... [2] https://www.anandtech.com/show/16252/mac-mini-apple-m1-tested https://www.anandtech.com/show/16252/mac-mini-apple-m1-teste... [3] https://www.anandtech.com/show/17047/the-intel-12th-gen-core-i912900k-review-hybrid-performance-brings-hybrid-complexity/6 https://www.anandtech.com/show/17047/the-intel-12th-gen-core... [4] https://www.anandtech.com/show/17024/apple-m1-max-performance-review/2 https://www.anandtech.com/show/17024/apple-m1-max-performanc...
raverbashing 3y ago
Yeah. But even as a NUMA area, upgradability would win a lot of brownie points here (of course modern Apple does not give a flapping bird about it) Not sure how it works on the iMac/MacPro
- avidphantasm 3y ago
  Would love to see a Mac Pro with 8 of these…

Apple's ridiculously expensive fast & wide on-package memory is a big reason why the M series chips are so fast. Expect AMD & Intel to adopt a similar strategy in their high end chips soon.

jsheard 3y ago

Intel is already there at the ultra-high-end, the Xeon Max is a hybrid with 64GB of HBM2 on-package and an external DDR5 bus supporting up to 4TB of socketed RAM. The HBM2 can be configured to work as L4 cache for the DDR5, or it can boot without any DDR5 at all and run entirely from the HBM2. https://www.servethehome.com/intel-xeon-max-9480-deep-dive-intel-has-64gb-hbm2e-onboard-like-a-gpu-or-ai-accelerator/ https://www.servethehome.com/intel-xeon-max-9480-deep-dive-i...

bryanlarsen 3y ago
MI300X also has on-package RAM, but that's even further from a laptop chip than Xeon Max. :)
jauntywundrkind 3y ago
Intel's Lakefield also comes to mind, pretty underrated chip imo. The first x86 big/small heterogenous chip, using Intel's Foveros to stack memory, CPU, and platform chip into one package. It made for a very simple design that just needed power. LPDDR4X-4266 doesn't crazy fast but in 2020 it was pretty swift. https://www.anandtech.com/show/15877/intel-hybrid-cpu-lakefield-all-you-need-to-know/3 https://www.anandtech.com/show/15877/intel-hybrid-cpu-lakefi... Alas the 1+4 Skylake+atom were all pretty meh chips & Intel hasn't tried again. Conceptually I really.liked it, and it was doing great against the Qualcomm 8cx competitor for the tablets/2-in-1s it was in, but people wanted more.
bastard_op 3y ago
I remember seeing this from STH as well, and my first thought was "I want one of these in my laptop!" It'd be perfect then to max ram in a laptop with 2 sodimm slots, or one of these 64gb lpcamm's. Ideally with a thinner laptop in mind, but probably not with the heatsink for THAT massive xeon. I really hope the hbm memory on-die trickles down into pro-sumer processor chips eventually too.

jeswin 3y ago

> Apple's ridiculously expensive fast & wide on-package memory is a big reason why the M series chips are so fast Increasing memory speeds have had only a marginal impact on common workloads (not AI) in the PC World - this has been a well discussed topic. Why would it be different on the Mac?

__MatrixMan__ 3y ago
probably something to do with rounded rectangles
cogman10 3y ago
Memory speed usually refers to memory bandwidth, which does have a limited amount of impact on day to day apps as they don't usually require a lot of bandwidth. Memory Latency, on the other hand, can have a huge impact on performance. That's the reason chips have had steadily increasing cache sizes over the decades along with the introduction of additional caching layers. What Apple's M series improves by putting the memory on chip is both bandwidth and latency. The latency, however, is what will impact app performance more than anything else. Latency improvements are the reason memory controllers were moved from motherboard northbridges into the CPU IC. Shortening that distance means a lot to shortening latency.
- vlovich123 3y ago
  Amusingly the top-line latency number is higher than for X86, but I agree with you because that number hides the better performance in other latency metrics that are probably more important.
vlovich123 3y ago
It's not about memory bandwidth. [1] shows that M1 is 66x faster at pointer chasing than an equivalent AMD. You can look at [2] and [3] to see that TLB penalty is also lower at any given test depth which is important because TLB misses aren't that infrequent and they're serial in your ability to access any memory that's not in your cache. It's not an unreasonable hypothesis that this is because of on-package memory which means you can run with tighter thresholds since your memory traces are shorter. Their unified memory architecture also means that GPU is way more effective since you never need to copy memory from the CPU to GPU. That's probably not a huge deal for normal apps, but having rendering the screen not competing with any app resources for memory probably isn't nothing. Neither the CPU nor the GPU can fully utilize the available memory bandwidth [4] The insane memory bandwidth is to also support the other coprocessors like the media engine & to ensure that you can do a bunch of tasks in parallel without them contending with each for resources. Those workloads do benefit greatly from memory bandwidth & the "artist" community is hugely important to Apple. The reason M is overall faster is more because of things like better ILP (e.g. fixed-width RISC encoding vs compression-like CISC means easier to keep the pipeline fed). Apple software for the most part is better engineered for performance & the M chip is optimized for that performance profile (e.g. C# and Java vs Swift/ObjC - Rc is pretty conclusively faster & requires less memory overall not to mention that the M chip has specific optimizations to further reduce the traditional cost of Rc that they have to pay because they don't typically distinguish Rc from Arc as Rust does). The final truth is that Apple throws a lot of money at this problem as well & buy up the latest processing node to stay ahead of the competition. [1] https://www.linkedin.com/pulse/apple-m-cpus-probably-much-faster-than-we-realize-yaron-shani/ https://www.linkedin.com/pulse/apple-m-cpus-probably-much-fa... [2] https://www.anandtech.com/show/16252/mac-mini-apple-m1-tested https://www.anandtech.com/show/16252/mac-mini-apple-m1-teste... [3] https://www.anandtech.com/show/17047/the-intel-12th-gen-core-i912900k-review-hybrid-performance-brings-hybrid-complexity/6 https://www.anandtech.com/show/17047/the-intel-12th-gen-core... [4] https://www.anandtech.com/show/17024/apple-m1-max-performance-review/2 https://www.anandtech.com/show/17024/apple-m1-max-performanc...

raverbashing 3y ago

Yeah. But even as a NUMA area, upgradability would win a lot of brownie points here (of course modern Apple does not give a flapping bird about it) Not sure how it works on the iMac/MacPro

avidphantasm 3y ago
Would love to see a Mac Pro with 8 of these…