3 ms·

> Apple's ridiculously expensive fast & wide on-package memory is a big reason why the M series chips are so fast Increasing memory speeds have had only a marg

by jeswin 2y ago

> Apple's ridiculously expensive fast & wide on-package memory is a big reason why the M series chips are so fast Increasing memory speeds have had only a marginal impact on common workloads (not AI) in the PC World - this has been a well discussed topic. Why would it be different on the Mac?

__MatrixMan__ 2y ago
probably something to do with rounded rectangles
cogman10 2y ago
Memory speed usually refers to memory bandwidth, which does have a limited amount of impact on day to day apps as they don't usually require a lot of bandwidth. Memory Latency, on the other hand, can have a huge impact on performance. That's the reason chips have had steadily increasing cache sizes over the decades along with the introduction of additional caching layers. What Apple's M series improves by putting the memory on chip is both bandwidth and latency. The latency, however, is what will impact app performance more than anything else. Latency improvements are the reason memory controllers were moved from motherboard northbridges into the CPU IC. Shortening that distance means a lot to shortening latency.
- vlovich123 2y ago
  Amusingly the top-line latency number is higher than for X86, but I agree with you because that number hides the better performance in other latency metrics that are probably more important.
vlovich123 2y ago
It's not about memory bandwidth. [1] shows that M1 is 66x faster at pointer chasing than an equivalent AMD. You can look at [2] and [3] to see that TLB penalty is also lower at any given test depth which is important because TLB misses aren't that infrequent and they're serial in your ability to access any memory that's not in your cache. It's not an unreasonable hypothesis that this is because of on-package memory which means you can run with tighter thresholds since your memory traces are shorter. Their unified memory architecture also means that GPU is way more effective since you never need to copy memory from the CPU to GPU. That's probably not a huge deal for normal apps, but having rendering the screen not competing with any app resources for memory probably isn't nothing. Neither the CPU nor the GPU can fully utilize the available memory bandwidth [4] The insane memory bandwidth is to also support the other coprocessors like the media engine & to ensure that you can do a bunch of tasks in parallel without them contending with each for resources. Those workloads do benefit greatly from memory bandwidth & the "artist" community is hugely important to Apple. The reason M is overall faster is more because of things like better ILP (e.g. fixed-width RISC encoding vs compression-like CISC means easier to keep the pipeline fed). Apple software for the most part is better engineered for performance & the M chip is optimized for that performance profile (e.g. C# and Java vs Swift/ObjC - Rc is pretty conclusively faster & requires less memory overall not to mention that the M chip has specific optimizations to further reduce the traditional cost of Rc that they have to pay because they don't typically distinguish Rc from Arc as Rust does). The final truth is that Apple throws a lot of money at this problem as well & buy up the latest processing node to stay ahead of the competition. [1] https://www.linkedin.com/pulse/apple-m-cpus-probably-much-faster-than-we-realize-yaron-shani/ https://www.linkedin.com/pulse/apple-m-cpus-probably-much-fa... [2] https://www.anandtech.com/show/16252/mac-mini-apple-m1-tested https://www.anandtech.com/show/16252/mac-mini-apple-m1-teste... [3] https://www.anandtech.com/show/17047/the-intel-12th-gen-core-i912900k-review-hybrid-performance-brings-hybrid-complexity/6 https://www.anandtech.com/show/17047/the-intel-12th-gen-core... [4] https://www.anandtech.com/show/17024/apple-m1-max-performance-review/2 https://www.anandtech.com/show/17024/apple-m1-max-performanc...
- neonsunset 2y ago
  No. M1/2/3 has data memory-dependent prefetcher[0] which can speculatively chase pointers in contiguous memory ranges, reducing indirection cost provided the data there is homogenous, looks like pointers and prefetch attempts are successful per cache line. It also has extra good indirect load predictor. These have little to do with memory packaging and a lot to do with cache and prefetching architecture. It does have one of the lowest latencies for atomics[1][2] which is something Swift cares about, but the overall core design just as much benefits other, less indirection and synchronization heavy languages (side note: Swift and Java are closer with each other in defaulting to virtual dispatch, where-as C# sits in the middle of the road with static dispatch by default but some code heavily using interfaces and virtual methods which do make such calls virtual to an extent (JIT has DPGO to devirtualize, AOT can uncoinditionally devirtualize in certain scenarios too, similar to what Swift's WMO does)). [0] https://gofetch.fail/files/gofetch.pdf https://gofetch.fail/files/gofetch.pdf [1] https://dougallj.github.io/applecpu/measurements/firestorm/DMB_ISH.html https://dougallj.github.io/applecpu/measurements/firestorm/D... [2] https://dougallj.github.io/applecpu/measurements/firestorm/CASAL_64.html https://dougallj.github.io/applecpu/measurements/firestorm/C...

by jeswin 2y ago

__MatrixMan__ 2y ago

probably something to do with rounded rectangles

cogman10 2y ago

Memory speed usually refers to memory bandwidth, which does have a limited amount of impact on day to day apps as they don't usually require a lot of bandwidth. Memory Latency, on the other hand, can have a huge impact on performance. That's the reason chips have had steadily increasing cache sizes over the decades along with the introduction of additional caching layers. What Apple's M series improves by putting the memory on chip is both bandwidth and latency. The latency, however, is what will impact app performance more than anything else. Latency improvements are the reason memory controllers were moved from motherboard northbridges into the CPU IC. Shortening that distance means a lot to shortening latency.

vlovich123 2y ago
Amusingly the top-line latency number is higher than for X86, but I agree with you because that number hides the better performance in other latency metrics that are probably more important.

vlovich123 2y ago

It's not about memory bandwidth. [1] shows that M1 is 66x faster at pointer chasing than an equivalent AMD. You can look at [2] and [3] to see that TLB penalty is also lower at any given test depth which is important because TLB misses aren't that infrequent and they're serial in your ability to access any memory that's not in your cache. It's not an unreasonable hypothesis that this is because of on-package memory which means you can run with tighter thresholds since your memory traces are shorter. Their unified memory architecture also means that GPU is way more effective since you never need to copy memory from the CPU to GPU. That's probably not a huge deal for normal apps, but having rendering the screen not competing with any app resources for memory probably isn't nothing. Neither the CPU nor the GPU can fully utilize the available memory bandwidth [4] The insane memory bandwidth is to also support the other coprocessors like the media engine & to ensure that you can do a bunch of tasks in parallel without them contending with each for resources. Those workloads do benefit greatly from memory bandwidth & the "artist" community is hugely important to Apple. The reason M is overall faster is more because of things like better ILP (e.g. fixed-width RISC encoding vs compression-like CISC means easier to keep the pipeline fed). Apple software for the most part is better engineered for performance & the M chip is optimized for that performance profile (e.g. C# and Java vs Swift/ObjC - Rc is pretty conclusively faster & requires less memory overall not to mention that the M chip has specific optimizations to further reduce the traditional cost of Rc that they have to pay because they don't typically distinguish Rc from Arc as Rust does). The final truth is that Apple throws a lot of money at this problem as well & buy up the latest processing node to stay ahead of the competition. [1] https://www.linkedin.com/pulse/apple-m-cpus-probably-much-faster-than-we-realize-yaron-shani/ https://www.linkedin.com/pulse/apple-m-cpus-probably-much-fa... [2] https://www.anandtech.com/show/16252/mac-mini-apple-m1-tested https://www.anandtech.com/show/16252/mac-mini-apple-m1-teste... [3] https://www.anandtech.com/show/17047/the-intel-12th-gen-core-i912900k-review-hybrid-performance-brings-hybrid-complexity/6 https://www.anandtech.com/show/17047/the-intel-12th-gen-core... [4] https://www.anandtech.com/show/17024/apple-m1-max-performance-review/2 https://www.anandtech.com/show/17024/apple-m1-max-performanc...

neonsunset 2y ago
No. M1/2/3 has data memory-dependent prefetcher[0] which can speculatively chase pointers in contiguous memory ranges, reducing indirection cost provided the data there is homogenous, looks like pointers and prefetch attempts are successful per cache line. It also has extra good indirect load predictor. These have little to do with memory packaging and a lot to do with cache and prefetching architecture. It does have one of the lowest latencies for atomics[1][2] which is something Swift cares about, but the overall core design just as much benefits other, less indirection and synchronization heavy languages (side note: Swift and Java are closer with each other in defaulting to virtual dispatch, where-as C# sits in the middle of the road with static dispatch by default but some code heavily using interfaces and virtual methods which do make such calls virtual to an extent (JIT has DPGO to devirtualize, AOT can uncoinditionally devirtualize in certain scenarios too, similar to what Swift's WMO does)). [0] https://gofetch.fail/files/gofetch.pdf https://gofetch.fail/files/gofetch.pdf [1] https://dougallj.github.io/applecpu/measurements/firestorm/DMB_ISH.html https://dougallj.github.io/applecpu/measurements/firestorm/D... [2] https://dougallj.github.io/applecpu/measurements/firestorm/CASAL_64.html https://dougallj.github.io/applecpu/measurements/firestorm/C...