4 ms·

Do you happen to know where to find any resources on how Apple managed to make the M1 so good compared to the competition? And why this has not happened before

Do you happen to know where to find any resources on how Apple managed to make the M1 so good compared to the competition? And why this has not happened before with other manufacturers?

dragontamer 6y ago
> Do you happen to know where to find any resources on how Apple managed to make the M1 so good compared to the competition? If you know computer microarchitecture, the specs have been discussed all across the internet by now. Reorder buffers, execution widths, everything. If you don't know how to read those specs... well... that's a bit harder. I don't really know how to help ya there. Maybe read Agner Fog's microarchitecture manual until you understand the subject, and then read the M1 microarchitecture discussions? I do realize this is a non-answer. But... I'm not sure if there's any way to easily understand computer microarchitecture unless you put effort to learn it. https://www.agner.org/optimize/ https://www.agner.org/optimize/ Read Manual #3: Microarchitecture. Understand what all these parts of a modern CPU does. Then, when you look at something like the M1's design, it becomes obvious what all those parts are doing. > And why this has not happened before with other manufacturers? 1. Apple is on TSMC 5nm, and no one else can afford that yet. So they have the most advanced process in the world, and Apple pays top-dollar to TSMC to ensure they're the first on the new node. 2. Apple has made some interesting decisions that runs very much counter to Intel and AMD's approach. Intel is pushing wider vector units, as you might know (AVX512), and despite the poo-pooing of AVX512, it gets the job done. AMD's approach is "more cores", they have a 4-wide execution unit and are splitting up their chips across multiple dies now to give better-and-better multithreaded performance. Apple's decision to make a 8-wide decoder engine is a decision, a compromise, which will make scaling up to more-cores more difficult. Apple's core is simply the biggest core on the market. Whereas AMD decided that 4-wide decode was enough (and then split into new cores), Apple ran the math and came out with the opposite conclusion, pushing for 8-wide decode instead. As such, the M1 will achieve the best single-threaded numbers. --------- Note that Apple has also largely given up on SIMD-execute. ARM 128-bit vectors are supported, but AVX2 from x86 land and AVX512 support 256-bit and 512-bit vectors respectively. As such, the M1's 128-bit wide vectors are its weak point, and it shows. Apple has decided that integer-performance is more important. It seems like Apple is using either its iGPU or Neural Engine for regular math / compute applications however. (The Neural Engine is a VLIW architecture, and iGPUs are of course just a wider SIMD unit in general). So Apple's strategy seems to be to offload the SIMD-compute to other, more specialized computers (still on the same SoC).
- Zigurd 6y ago
  This "and Apple pays top-dollar to TSMC to ensure they're the first on the new node" is Tim Cook's crowning achievement in the way Apple combines supply chain dominance with technology strategy. They do not win every bet they make (e.g. growing their own sapphire) but when they win it is stunning.
  - kergonath 6y ago
    Yes, and it’s been how they operate for quite a while now, ever like when they bought Toshiba’s production of 1.8’’ drives for the iPod (again, a part that defines the device), or how they paid to build NAND factories a couple of years later in exchange of a significant part of their production (including all of Samsung’s output in 2009).
- temac 6y ago
  > Apple's decision to make a 8-wide decoder engine is a decision, a compromise, which will make scaling up to more-cores more difficult. Apple's core is simply the biggest core on the market. > Whereas AMD decided that 4-wide decode was enough (and then split into new cores), Apple ran the math and came out with the opposite conclusion, pushing for 8-wide decode instead. As such, the M1 will achieve the best single-threaded numbers. That's not as simple. x86 is way more difficult to decode than ARM. Also, the insanely large OOO probably helps a lot to keep the wide M1 beast occupied. Does the large L1 helps? I don't know. Maybe a large enough L2 would be OK. And the perf cores do not occupy the largest area of the die. Can you do a very large L1 with not too bad latency impact? I guess a small node helps, plus maybe you keep a reasonable associativity and a traditional L1 lookup thanks to the larger pages. So I'm curious what happens with 4kB pages and it probably has that mode for emulation? Going specialized instead of putting large vector in the CPU also makes complete sense. You want to be slow and wide to optimize for efficiency. Of course it's less possible for mainly scalar and branch rich workloads, so you can't be as wide on a CPU. You still need a middle ground for your low latency compute needs in the middle of your scalar code and 128-bits certainly is one esp if you can imagine to scale to lots of execution units (well I this point I admit you can also support a wider size, but that shows the impact of staying 128 won't necessarily be crazy if structured like that), although one could argue for 256, but 512 starts to not be reasonable and probably has a way worse impact on core size than wide superscalar - or at least even if the impact is similar (I'm not sure) I suspect that wide superscalar is more useful most of the time. It's understandable that a more CPU oriented vendor will be far more interested by large vectors. Apple is not that -- although of course what they will do for their high end will be extremely interesting to watch. Of course you have to solve a wide variety of problems, but the recent AMD approach has shown that the good old method of optimizing for real workloads just continue to be the way to go. Who cares if you have somehow more latency in not so frequent cases, or if int <-> fp is slower, if in the end that let you optimise the structures were you reap most benefits. Now each has its own history obviously and the mobile roots of the M1 also gives a strong influence, plus the vertical integration of Apple helps immensely. I want to add: even if the M1 is impressive, Apple has not a too insane advance in the end result compared to what AMD does on 7nm. But of course they will continue to improve.
fulafel 6y ago
It did happen before, with a) Apple -look at the benchmarks of Apple chips vs other ARM implementations from past years. The M1 essentially the same SoC as the current iPad one with more cores and memory. b) with other manufacturers: there have been "wow" CPUs from time to time. Early MIPS chips, The Alpha victorious period of 21064/21164/21264, Pentium Pro, AMD K7, StrongArm (Apple connection here as wel), etc. Then Intel managed to torpedo the fragmented high-performance RISC competition and convinced their patrons to jump ship to the ill-fated Itanium, which led to a long lull in serious competition.
nialo 6y ago
The vague impression I get is that maybe the answer is "Because Apple's software people and chip design people are in the same company, they did a better job of coordinating to make good tradeoffs in the chip and software design." (I'm getting this from reading between lines on Twitter, so it's not exactly a high confidence guess)
brennanpeterson 6y ago
A good design + a 1 node (15% transistor level benefit) advantage + embedded(ish) HBM. You kind of expect 15-30% benefit in the same design. Whether CPU or bandwidth bound. Some latency bound benchmarks will be at par. It is a perfect hit on all cylinders: a good design, a node advantage, and better memory. This has happened before. We have been at a recent lull in performance, but annual performance increases used to be about 30%.
- floatboth 6y ago
  Where did you find HBM? The mock-up picture they showed at the event clearly shows two traditional DDR style chips (encased in their own plastic with white text on it) on the package. This is absolutely not how HBM looks like, HBM is bare die stacks connected to the processor die with an interposer. Also HBM would've made it significantly more expensive.
  - dragontamer 6y ago
    There is a European site (swedish? Danish?) That mistranslated this information. And then the information came full circle back into english.
  - brennanpeterson 6y ago
    You are right, I misread 'high bandwidth' as HBM. I was surprised at the cost as well.

Do you happen to know where to find any resources on how Apple managed to make the M1 so good compared to the competition? And why this has not happened before with other manufacturers?

dragontamer 6y ago

> Do you happen to know where to find any resources on how Apple managed to make the M1 so good compared to the competition? If you know computer microarchitecture, the specs have been discussed all across the internet by now. Reorder buffers, execution widths, everything. If you don't know how to read those specs... well... that's a bit harder. I don't really know how to help ya there. Maybe read Agner Fog's microarchitecture manual until you understand the subject, and then read the M1 microarchitecture discussions? I do realize this is a non-answer. But... I'm not sure if there's any way to easily understand computer microarchitecture unless you put effort to learn it. https://www.agner.org/optimize/ https://www.agner.org/optimize/ Read Manual #3: Microarchitecture. Understand what all these parts of a modern CPU does. Then, when you look at something like the M1's design, it becomes obvious what all those parts are doing. > And why this has not happened before with other manufacturers? 1. Apple is on TSMC 5nm, and no one else can afford that yet. So they have the most advanced process in the world, and Apple pays top-dollar to TSMC to ensure they're the first on the new node. 2. Apple has made some interesting decisions that runs very much counter to Intel and AMD's approach. Intel is pushing wider vector units, as you might know (AVX512), and despite the poo-pooing of AVX512, it gets the job done. AMD's approach is "more cores", they have a 4-wide execution unit and are splitting up their chips across multiple dies now to give better-and-better multithreaded performance. Apple's decision to make a 8-wide decoder engine is a decision, a compromise, which will make scaling up to more-cores more difficult. Apple's core is simply the biggest core on the market. Whereas AMD decided that 4-wide decode was enough (and then split into new cores), Apple ran the math and came out with the opposite conclusion, pushing for 8-wide decode instead. As such, the M1 will achieve the best single-threaded numbers. --------- Note that Apple has also largely given up on SIMD-execute. ARM 128-bit vectors are supported, but AVX2 from x86 land and AVX512 support 256-bit and 512-bit vectors respectively. As such, the M1's 128-bit wide vectors are its weak point, and it shows. Apple has decided that integer-performance is more important. It seems like Apple is using either its iGPU or Neural Engine for regular math / compute applications however. (The Neural Engine is a VLIW architecture, and iGPUs are of course just a wider SIMD unit in general). So Apple's strategy seems to be to offload the SIMD-compute to other, more specialized computers (still on the same SoC).

Zigurd 6y ago
This "and Apple pays top-dollar to TSMC to ensure they're the first on the new node" is Tim Cook's crowning achievement in the way Apple combines supply chain dominance with technology strategy. They do not win every bet they make (e.g. growing their own sapphire) but when they win it is stunning.
- kergonath 6y ago
  Yes, and it’s been how they operate for quite a while now, ever like when they bought Toshiba’s production of 1.8’’ drives for the iPod (again, a part that defines the device), or how they paid to build NAND factories a couple of years later in exchange of a significant part of their production (including all of Samsung’s output in 2009).
temac 6y ago
> Apple's decision to make a 8-wide decoder engine is a decision, a compromise, which will make scaling up to more-cores more difficult. Apple's core is simply the biggest core on the market. > Whereas AMD decided that 4-wide decode was enough (and then split into new cores), Apple ran the math and came out with the opposite conclusion, pushing for 8-wide decode instead. As such, the M1 will achieve the best single-threaded numbers. That's not as simple. x86 is way more difficult to decode than ARM. Also, the insanely large OOO probably helps a lot to keep the wide M1 beast occupied. Does the large L1 helps? I don't know. Maybe a large enough L2 would be OK. And the perf cores do not occupy the largest area of the die. Can you do a very large L1 with not too bad latency impact? I guess a small node helps, plus maybe you keep a reasonable associativity and a traditional L1 lookup thanks to the larger pages. So I'm curious what happens with 4kB pages and it probably has that mode for emulation? Going specialized instead of putting large vector in the CPU also makes complete sense. You want to be slow and wide to optimize for efficiency. Of course it's less possible for mainly scalar and branch rich workloads, so you can't be as wide on a CPU. You still need a middle ground for your low latency compute needs in the middle of your scalar code and 128-bits certainly is one esp if you can imagine to scale to lots of execution units (well I this point I admit you can also support a wider size, but that shows the impact of staying 128 won't necessarily be crazy if structured like that), although one could argue for 256, but 512 starts to not be reasonable and probably has a way worse impact on core size than wide superscalar - or at least even if the impact is similar (I'm not sure) I suspect that wide superscalar is more useful most of the time. It's understandable that a more CPU oriented vendor will be far more interested by large vectors. Apple is not that -- although of course what they will do for their high end will be extremely interesting to watch. Of course you have to solve a wide variety of problems, but the recent AMD approach has shown that the good old method of optimizing for real workloads just continue to be the way to go. Who cares if you have somehow more latency in not so frequent cases, or if int <-> fp is slower, if in the end that let you optimise the structures were you reap most benefits. Now each has its own history obviously and the mobile roots of the M1 also gives a strong influence, plus the vertical integration of Apple helps immensely. I want to add: even if the M1 is impressive, Apple has not a too insane advance in the end result compared to what AMD does on 7nm. But of course they will continue to improve.

fulafel 6y ago

It did happen before, with a) Apple -look at the benchmarks of Apple chips vs other ARM implementations from past years. The M1 essentially the same SoC as the current iPad one with more cores and memory. b) with other manufacturers: there have been "wow" CPUs from time to time. Early MIPS chips, The Alpha victorious period of 21064/21164/21264, Pentium Pro, AMD K7, StrongArm (Apple connection here as wel), etc. Then Intel managed to torpedo the fragmented high-performance RISC competition and convinced their patrons to jump ship to the ill-fated Itanium, which led to a long lull in serious competition.

nialo 6y ago

The vague impression I get is that maybe the answer is "Because Apple's software people and chip design people are in the same company, they did a better job of coordinating to make good tradeoffs in the chip and software design." (I'm getting this from reading between lines on Twitter, so it's not exactly a high confidence guess)

brennanpeterson 6y ago

A good design + a 1 node (15% transistor level benefit) advantage + embedded(ish) HBM. You kind of expect 15-30% benefit in the same design. Whether CPU or bandwidth bound. Some latency bound benchmarks will be at par. It is a perfect hit on all cylinders: a good design, a node advantage, and better memory. This has happened before. We have been at a recent lull in performance, but annual performance increases used to be about 30%.

floatboth 6y ago
Where did you find HBM? The mock-up picture they showed at the event clearly shows two traditional DDR style chips (encased in their own plastic with white text on it) on the package. This is absolutely not how HBM looks like, HBM is bare die stacks connected to the processor die with an interposer. Also HBM would've made it significantly more expensive.
- dragontamer 6y ago
  There is a European site (swedish? Danish?) That mistranslated this information. And then the information came full circle back into english.
- brennanpeterson 6y ago
  You are right, I misread 'high bandwidth' as HBM. I was surprised at the cost as well.