3 ms·

Some of the things that make Apple's ARM implementation so performant, taken from Anandtech's excellent A14 deep dive. https://www.anandtech.com/show/16226/app

Some of the things that make Apple's ARM implementation so performant, taken from Anandtech's excellent A14 deep dive. https://www.anandtech.com/show/16226/apple-silicon-m1-a14-deep-dive/2 https://www.anandtech.com/show/16226/apple-silicon-m1-a14-de... Extremely wide instruction decode: >Featuring an 8-wide decode block, Apple’s Firestorm is by far the current widest commercialized design in the industry. IBM’s upcoming P10 Core in the POWER10 is the only other official design that’s expected to come to market with such a wide decoder design Other contemporary designs such as AMD’s Zen(1 through 3) and Intel’s µarch’s, x86 CPUs today still only feature a 4-wide decoder designs (Intel is 1+4) A very deep reorder buffer: >A +-630 deep ROB is an immensely huge out-of-order window for Apple’s new core, as it vastly outclasses any other design in the industry. Intel’s Sunny Cove and Willow Cove cores are the second-most “deep” OOO designs out there with a 352 ROB structure, while AMD’s newest Zen3 core makes due with 256 entries, and recent Arm designs such as the Cortex-X1 feature a 224 structure. Exactly how and why Apple is able to achieve such a grossly disproportionate design compared to all other designers in the industry isn’t exactly clear, but it appears to be a key characteristic of Apple’s design philosophy and method to achieve high ILP (Instruction level-parallelism). A huge number of execution units: >On the Integer side, whose in-flight instructions and renaming physical register file capacity we estimate at around 354 entries, we find at least 7 execution ports for actual arithmetic operations. These include 4 simple ALUs capable of ADD instructions, 2 complex units which feature also MUL (multiply) capabilities, and what appears to be a dedicated integer division unit. The core is able to handle 2 branches per cycle, which I think is enabled by also one or two dedicated branch forwarding ports On the floating point and vector execution side of things, the new Firestorm cores are actually more impressive as they a 33% increase in capabilities, enabled by Apple’s addition of a fourth execution pipeline. The FP rename registers here seem to land at 384 entries, which is again comparatively massive. The four 128-bit NEON pipelines thus on paper match the current throughput capabilities of desktop cores from AMD and Intel, albeit with smaller vectors. Massive low latency Level 1 caches: >128KB L1 Data cache for which we can test for, however following Darwin kernel source dumps Apple has confirmed that it’s actually a massive 192KB instruction cache. That’s absolutely enormous and is 3x larger than the competing Arm designs, and 6x larger than current x86 designs The huge caches also appear to be extremely fast – the L1D lands in at a 3-cycle load-use latency. AMD has a 32KB 4-cycle cache, whilst Intel’s latest Sunny Cove saw a regression to 5 cycles when they grew the size to 48KB. Massive L2 Cache: >On the L2 side of things, Apple has been employing an 12MB structure that’s shared between their four big cores. This is an extremely unusual cache hierarchy and contrasts to everybody else’s use of an intermediary sized private L2 combined with a larger slower L3. Apple here disregards the norms, and chooses a large and fast L2. In comparison Zen 3 has 512 KiB L2 per core, and Tiger Lake has 1.25 MiB L2 per core. Another point tweeted by the author of the above piece but not in the above article... Much more accurate branch prediction: >Finally doing perf counters on a dedicated test bench... 9900K having 60% worse branch misprediction than Apple's A12. https://twitter.com/andreif7/status/1307420010177007625 https://twitter.com/andreif7/status/1307420010177007625 There's more going on with the M1 design than "it's ARM/RISC".

cesarb 6y ago
> There's more going on with the M1 design than "it's ARM/RISC". Yeah, but the interesting part is how many of these points are enabled by the 64-bit ARM ISA. > Featuring an 8-wide decode block, [...] x86 CPUs today still only feature a 4-wide decoder designs (Intel is 1+4) This is the obvious one: 64-bit ARM has fixed size instructions, so you don't need to know the size of each instruction to decode the next one, making it much easier to parallelize. But there's a less obvious detail, shown by that "1+" on the Intel design: for microcoded instructions which can generate many µops, only one of the decoders can be used. AFAIK, no instructions in 64-bit ARM can generate that many µops, probably at most two µops per instruction (for the load and store pair instructions), so probably all decoders can be used for all instructions. This is an advantage (and in fact the whole point) of a RISC design. > A +-630 deep ROB is an immensely huge out-of-order window for Apple’s new core, as it vastly outclasses any other design in the industry. [...] we estimate at around 354 entries [...] The FP rename registers here seem to land at 384 entries, which is again comparatively massive. [...] My guess is that these are all consequences of the 8-wide decode: they had to increase all these resources, otherwise there would be a bottleneck. The reason Intel and AMD don't have as much would be because their bottleneck is elsewhere, so it would be wasteful. > 128KB L1 Data cache for which we can test for, [...] a massive 192KB instruction cache [...] The huge caches also appear to be extremely fast – the L1D lands in at a 3-cycle load-use latency. AMD has a 32KB 4-cycle cache, [...] It's quite curious that the L1D is four times the size of the AMD L1D cache, while AFAIK the page size used by iOS and macOS (16KiB) is four times the page size used by Intel and AMD (4KiB). My guess is that the L1D size could be increased without getting slower precisely because of the page size increase; the L1 cache is usually VIPT, which has lower latency because it doesn't have to wait for the TLB, but has the disadvantage that you can get cache aliases unless only the bits which do not change in the virtual to physical translation are used to index the cache. So to increase the L1D cache, Intel and AMD have to increase the number of ways, which is already at a massive 8 ways for a 32KiB cache (and increasing it has a latency cost); increasing the page size allows you to increase the cache size without increasing the number of ways. And 64-bit ARM can go even further; its standard page sizes are 4KiB, 16KiB, and 64KiB, the later being the default for at least RHEL/CentOS, so we might soon be seeing even larger L1D caches in ARM servers (this last page size is also required for huge amounts of physical memory, which is probably why RedHat chose to use it). And for the instruction cache, we again have an advantage of the ARM ISA: for legacy reasons, whenever data is written to memory in the x86 architecture, the instruction cache has to either be written to, or the corresponding instruction cache line has to be implicitly invalidated. ARM doesn't have that legacy: all instruction cache invalidation is explicit. This means that aliases in the instruction cache are much less of an issue on ARM (in the worst case, they are just duplicating information in the cache), which allows the L1 instruction cache to be much bigger without increasing the number of ways. > This is an extremely unusual cache hierarchy and contrasts to everybody else’s use of an intermediary sized private L2 combined with a larger slower L3. I don't have an opinion on why Apple used that L2 cache design; it might be because, with the larger L1 caches, there's less need and/or advantage of having a "closer" (less latency) L2.

cesarb 6y ago

> There's more going on with the M1 design than "it's ARM/RISC". Yeah, but the interesting part is how many of these points are enabled by the 64-bit ARM ISA. > Featuring an 8-wide decode block, [...] x86 CPUs today still only feature a 4-wide decoder designs (Intel is 1+4) This is the obvious one: 64-bit ARM has fixed size instructions, so you don't need to know the size of each instruction to decode the next one, making it much easier to parallelize. But there's a less obvious detail, shown by that "1+" on the Intel design: for microcoded instructions which can generate many µops, only one of the decoders can be used. AFAIK, no instructions in 64-bit ARM can generate that many µops, probably at most two µops per instruction (for the load and store pair instructions), so probably all decoders can be used for all instructions. This is an advantage (and in fact the whole point) of a RISC design. > A +-630 deep ROB is an immensely huge out-of-order window for Apple’s new core, as it vastly outclasses any other design in the industry. [...] we estimate at around 354 entries [...] The FP rename registers here seem to land at 384 entries, which is again comparatively massive. [...] My guess is that these are all consequences of the 8-wide decode: they had to increase all these resources, otherwise there would be a bottleneck. The reason Intel and AMD don't have as much would be because their bottleneck is elsewhere, so it would be wasteful. > 128KB L1 Data cache for which we can test for, [...] a massive 192KB instruction cache [...] The huge caches also appear to be extremely fast – the L1D lands in at a 3-cycle load-use latency. AMD has a 32KB 4-cycle cache, [...] It's quite curious that the L1D is four times the size of the AMD L1D cache, while AFAIK the page size used by iOS and macOS (16KiB) is four times the page size used by Intel and AMD (4KiB). My guess is that the L1D size could be increased without getting slower precisely because of the page size increase; the L1 cache is usually VIPT, which has lower latency because it doesn't have to wait for the TLB, but has the disadvantage that you can get cache aliases unless only the bits which do not change in the virtual to physical translation are used to index the cache. So to increase the L1D cache, Intel and AMD have to increase the number of ways, which is already at a massive 8 ways for a 32KiB cache (and increasing it has a latency cost); increasing the page size allows you to increase the cache size without increasing the number of ways. And 64-bit ARM can go even further; its standard page sizes are 4KiB, 16KiB, and 64KiB, the later being the default for at least RHEL/CentOS, so we might soon be seeing even larger L1D caches in ARM servers (this last page size is also required for huge amounts of physical memory, which is probably why RedHat chose to use it). And for the instruction cache, we again have an advantage of the ARM ISA: for legacy reasons, whenever data is written to memory in the x86 architecture, the instruction cache has to either be written to, or the corresponding instruction cache line has to be implicitly invalidated. ARM doesn't have that legacy: all instruction cache invalidation is explicit. This means that aliases in the instruction cache are much less of an issue on ARM (in the worst case, they are just duplicating information in the cache), which allows the L1 instruction cache to be much bigger without increasing the number of ways. > This is an extremely unusual cache hierarchy and contrasts to everybody else’s use of an intermediary sized private L2 combined with a larger slower L3. I don't have an opinion on why Apple used that L2 cache design; it might be because, with the larger L1 caches, there's less need and/or advantage of having a "closer" (less latency) L2.