9 ms·

Amdahal's Law says hello. https://en.wikipedia.org/wiki/Amdahl%27s_law https://en.wikipedia.org/wiki/Amdahl%27s_law If we keep scaling up number of processors

by nickcw 5y ago

Amdahal's Law says hello. https://en.wikipedia.org/wiki/Amdahl%27s_law https://en.wikipedia.org/wiki/Amdahl%27s_law If we keep scaling up number of processors rather than clock speed, what is going to be the maximum number useful cores in a laptop or desktop? 20? 100? 1000? At some point adding more cores is going to make no difference to the user experience, but the way we are going we'll be at 1000 cores in about a decade so we better start thinking about it now. Or to put it another way, what normal workloads will load up all the cores in the new M1 chip? Being a software developer, compiling things is the obvious choice, except when you come to that rather serial linking phase at the end of the compile job. Already my incremental Go compiles are completely dominated by the linking phase. There are a few easy to parallelise tasks, mostly to do with media (as it says in the article). However a lot of stuff isn't like that. Will 20 cores speed up my web browser? How about Excel? Your average user is going to prefer double the clock rate of your processors to doubling the number of processors. Anyway, I don't want to rain on Apple's parade with these musings - the M1 Ultra is an amazing achievement and it certainly isn't for your average user. I wish I had one in a computer of my choice running Linux!

zitterbewegung 5y ago
If I were to make a best case scenario on where multiprocessing , GPU acceleration and the neural engine I would say a iOS App development using a simulator , a container that contains your backend services and the iOS app using ML inference. Training on the M1 Ultra isn’t really worth it.
kayoone 5y ago
Well, chrome could certainly use more threads for all your tabs which might make it even more of a resource hog but probably also run faster. Software Engineering will evolve too, like we have seen in game development over the last decade. While 10 years ago most game engines were hardly using let alone optimized for multicore systems, nowadays the modern engines very much benefit from more cores and I guess the modern OSes also benefit from having more threads to put processes on.
simonh 5y ago
These pro machines are squarely aimed at creative professionals doing video work and image processing. Of course there are used for a lot of other stuff too, but that's the biggest single use case by long way. The studio name for the new systems was well advised.
xmodem 5y ago
Derbauer pointed out that, for Intel's 12 series, that the e-cores dominate the p-cores not just in performance per watt, but also performance-per-die-area. I haven't run the numbers for the M1, but I'd be shocked if it wasn't similar. It seems unavoidable that you can get more total performance with larger numbers of slower cores than smaller numbers of faster cores. The silicon industry has spent the entire multi-core era - the last 15 years - fighting this reality, but it finally seems to have caught up with us, so hopefully in the next few years we will start to see software actually start to adapt.
- hajile 5y ago
  It's pretty well established that M1 E-cores are something like 50% the performance, but 1/10 the power consumption. A55 is probably 1/8 the performance, but something like 1/100 of the power consumption and a miniscule die area. I wouldn't want to have all A55 cores on my phone though. Performance per die area is also relative. For example, Apple clocks their chips around 3GHz. If they redesigned them so they could ramped them up to 5GHz like Intel or AMD, they would stomp those companies, but they would also use several times more power. What is really relevant is something like the ratio of a given core's performance per area per watt to the same value for the fastest known core. The only interesting area for ultimate low-power in general purpose computing is some kind of A55 with a massive SIMD unit going with a larabee-style approach for a system that can both do massive compute AND not have performance plummet if you need branchy code too.
  - paulmd 5y ago
    Apple's e-cores are actually (relatively speaking) much better than Intel's. From memory, Apple gets about the same performance (actually higher in single-threaded, but same MT performance) out of about 2/3rds the transistor count as Intel. Note that this is invariant of node - obviously Blizzard cores are physically much smaller than Gracemont, but they are more smaller than you get out of a node shrink alone, apple is doing more with actually less transistors than Intel. Since the Intel e-cores still have a relatively wide decoder, e-core designs may be the part where the bill comes due for x86 in terms of decoder complexity. Sure it's only 3% of a performance core, but if you cut the performance core in half then now they're 6%. And the decoder doesn't shrink that much, Gracemont has a 3-wide decoder vs 4-wide on Golden Cove, and you still have to have the same amount of instruction cache (instruction cache hit rate depends on the amount of "hot code", and programs don't get smaller just because you run them on e-cores). A lot of the x86 "tricks" to keep the cores fed don't scale down much/any. edit: Intel Golden Cove: 7.04mm^2 with L2, 5.55mm^2 w/o L2 Intel Gracemont: 2.2mm^2 with L2, 1.7 mm^2 w/o L2 Apple Avalanche: 2.55mm^2. (I believe these are both w/o cache) Apple Blizzard: 0.69mm^2 (nice) Note that N7 to N5 has roughly 1.6x logic density scaling - so a Blizzard core would be 1.24mm^2 or roughly 73% of the transistor count of Gracemont for equivalent performance! For the p-cores the number is 82%. This is one of the reasons I feel Apple is so far ahead. It's not about raw performance, or even efficiency, it's the fact that Apple is winning on both those metrics while using 2/3rds the transistors. It's not just "apple throwing transistors at the problem", which of course they are, but just they're starting from a much better baseline such that they can afford to throw those transistors around. The higher transistor count in total is coming from the GPU, the cores themselves Apple is actually much more efficient (perf-per-transistor) than x86 competitors.
minhazm 5y ago
I'm guessing you mean double the single core performance and not clock speed. Even Intel Pentium 4's from 2000 had higher clock speeds than Apple's M1 chips. The clock speed matters far less than what you can actually get done in a single clock cycle. Just looking at Apple's own products and their single core performance from Geekbench you can see they achieved around a 40% improvement in just single core performance [1]. So it's not just adding more cores. Apple has usually been the one to hold out on cores and usually has resisted adding more cores to the iPhone / iPad chips in the past and focused on better single core performance. > Your average user is going to prefer double the clock rate of your processors to doubling the number of processors. I disagree. The reality is that these days people are running multi-threaded workloads even if they don't know it. Running a dozen chrome tabs, Slack, Teams, Zoom, some professional tools like IDE's, Adobe creative suite, etc. adds up very quickly to a lot of processes that can use a lot of cores. [1] https://browser.geekbench.com/mac-benchmarks https://browser.geekbench.com/mac-benchmarks
- rayiner 5y ago
  > Even Intel Pentium 4's from 2000 had higher clock speeds than Apple's M1 chips. It’s wild to see that in print.
  - ac29 5y ago
    Isnt true though - the Pentium 4 was released in 2000, but the fastest version available that year was only 1.5GHz. It wouldnt hit 3.2GHz (~M1) until 2003.
- pantulis 5y ago
  Isn't running multiple single-threaded workloads at the same time also a case for better performance with more cores? So even for basic tasks having more cores is better (although maybe not that cost-efficient)
  - katbyte 5y ago
    only if those workloads are very CPU intensive, and the ones that are should probably be parallelized as much as possible anyway.
bcrosby95 5y ago
It's a bit of a chicken-and-egg problem. In general, devs don't target machines that don't exist. But from some quick searching, excel will split out independent calculations into their own threads. So for that, the answer seems to be: it depends. If you're using 20 cores to calculate a single thing, it seems like the answer is "no". But if you're using 20 cores to calculate 20 different things, it seems like the answer is "yes".
paulmd 5y ago
Apple's advantage is actually that it provides much higher single-thread (really, per-thread) performance than "comparable" x86 processors. A M1 Pro can match a 12900HK in perfectly-threaded high-code-density scenarios like Cinebench, with half the thread count. Real-world IPC is something like 3x (!) that of Intel right now - obviously it also clocks lower but Apple hits very hard in lower-thread-count scenarios. If your code is bottlenecked on a single thread, or if it doesn't scale well to higher thread counts, Apple is actually great right now. The downside is that you can't get higher core counts, but that's where the Pro and Ultra SKUs come in. (The real, real downside is that right now you can't get higher core counts on M1 without being tied to a giant GPU you may not even use. What would be really nice is an M1 Ultra-sized chip with 20C or 30C and the same iGPU size as A14 or M1, or a server chip full of e-cores like Denverton or Sierra Forest, but that's very much not Apple's wheelhouse in terms of products unfortunately.)
- __init 5y ago
  > Real-world IPC is something like 3x (!) that of Intel right now - obviously it also clocks lower That's the problem, though -- if you clock yourself much lower, of course you can get higher IPC; you can pack more into your critical paths. Now, certainly Apple has some interesting and significant innovations over Intel here, but quoting IPC figures like that is highly misleading.
  - paulmd 5y ago
    Of course IPC needs to be contextualized, but it's still a very important metric. And Intel's processors aren't clocked 3x higher than Apple either - that would be 9 GHz, and you can't even sustain 5 GHz all-core at 35W let alone 9 GHz which just isn't physically possible even on LN2. https://images.anandtech.com/graphs/graph17024/117496.png https://images.anandtech.com/graphs/graph17024/117496.png That's an absolutely damning chart for x86, at iso-power the M1 Max scores 2.5x as high as a 5980HS in FP and 1.43x as high in integer workloads, despite having just over half the cores and ~0.8x the transistor budget per core. So it's a lot closer to the ~2.5-3x IPC scores than you'd think just from "but x86 clocks higher!". And these results do hold up across the broad spectrum of workloads: https://images.anandtech.com/graphs/graph17024/117494.png https://images.anandtech.com/graphs/graph17024/117494.png Yes, Alder Lake does better (although people always insist to me that Alder still somehow "scales worse at lower power levels than AMD"? That's not what the chart shows...) but even in the best-case scenario, you have Intel basically matching (slightly underperforming) AMD while using twice the thread count. And that's a single, cherrypicked benchmark that is known for favoring raw computation and disregarding performance of the front-end, if you are concerned about the x86 front-end, this is basically a best-case scenario for it... high code compactness and extremely high threadability. And it still needs twice the threads to do it. https://i.imgur.com/vaYTmDF.png https://i.imgur.com/vaYTmDF.png Like your "but x86 uses higher clock rates", you can also say "but x86 uses SMT", so maybe "performance per thread" is an unfair metric in some sense, but there is practical merit to it. If you have to use twice the threads to achieve equal performance on x86 then that's a downside, where Apple gives you high performance on tasks that don't scale to higher thread counts. And if Apple put out a processor with high P-core count and without the giant GPU, it would easily be the best hardware on the market. I just strongly doubt that "it's all node" like everyone insists. Apple is running fewer transistors per core already, and AMD/Intel are not going to double or triple their performance-per-thread within the next generation regardless of how many transistors they might use to do it (AMD will be on N5P this year, which will be node parity with Apple A15). x86 vendors can put out a product that will be competitive in one of several areas, but they can't win all of them at once like Apple can. And going forward - it's hard to see how x86 fixes that IPC gap. You can't scale the decoder as wide, Golden Cove already has a pretty big decoder in fact. A lot of the "tricks" have already been used. How do you triple IPC in that scenario, without blowing up transistor budgets hugely? Even if you get rid of SMT, you're not going to triple IPC. Or, how do you triple clockrate in an era when things are actually winding backwards? Others are very, very confident this lead will disappear when AMD moves to N5P. I just don't see it imo. The gap is too big.
tomjen3 5y ago
Amdahals law is sometimes brought out as a whammy (not saying that is what you do), but economically we would expect that there will be a usage for this extra compute: we did it server side by hosting SAAS and thereby adding more users than anybody could think of. I suspect that rather than 1000 cores we might start to see more levels of cores, and hardware for more things. Already Apple has video encoding support. AI seems an obvious idea and it scales much better than most classical computing. If I may bring up something that may be more of a wish: I wish that we could give up the idea of shared memory and we could have many more cores that communicated by shared messaging. We are already seeing this spread with webworkers - if it became cheap to create a new thread and computers weren't bottlenecked then maybe more games would use it too.
- samwillis 5y ago
  Message passing without shared memory is slow, having to copy data and maybe even serialise/deserialise it in the process. Message passing where you are just passing a pointer is fast. Web workers are basically the worst case, you have to serialise your data to and from JSON when passing it to and from a worker. It’s not built for performance. There have been many cases where people have tried to improve performance of their apps by offloading work to a web worker but the added cost of serialisation ultimately made it slower than running on the main thread.
gameswithgo 5y ago
It can accelerate compilation times quite a bit at times, which could allow us to use more compile time safety checks and more compile time optimizations, indirectly making the lower core count computers faster.

by nickcw 5y ago

zitterbewegung 5y ago

If I were to make a best case scenario on where multiprocessing , GPU acceleration and the neural engine I would say a iOS App development using a simulator , a container that contains your backend services and the iOS app using ML inference. Training on the M1 Ultra isn’t really worth it.

kayoone 5y ago

Well, chrome could certainly use more threads for all your tabs which might make it even more of a resource hog but probably also run faster. Software Engineering will evolve too, like we have seen in game development over the last decade. While 10 years ago most game engines were hardly using let alone optimized for multicore systems, nowadays the modern engines very much benefit from more cores and I guess the modern OSes also benefit from having more threads to put processes on.

simonh 5y ago

These pro machines are squarely aimed at creative professionals doing video work and image processing. Of course there are used for a lot of other stuff too, but that's the biggest single use case by long way. The studio name for the new systems was well advised.

xmodem 5y ago

Derbauer pointed out that, for Intel's 12 series, that the e-cores dominate the p-cores not just in performance per watt, but also performance-per-die-area. I haven't run the numbers for the M1, but I'd be shocked if it wasn't similar. It seems unavoidable that you can get more total performance with larger numbers of slower cores than smaller numbers of faster cores. The silicon industry has spent the entire multi-core era - the last 15 years - fighting this reality, but it finally seems to have caught up with us, so hopefully in the next few years we will start to see software actually start to adapt.

hajile 5y ago
It's pretty well established that M1 E-cores are something like 50% the performance, but 1/10 the power consumption. A55 is probably 1/8 the performance, but something like 1/100 of the power consumption and a miniscule die area. I wouldn't want to have all A55 cores on my phone though. Performance per die area is also relative. For example, Apple clocks their chips around 3GHz. If they redesigned them so they could ramped them up to 5GHz like Intel or AMD, they would stomp those companies, but they would also use several times more power. What is really relevant is something like the ratio of a given core's performance per area per watt to the same value for the fastest known core. The only interesting area for ultimate low-power in general purpose computing is some kind of A55 with a massive SIMD unit going with a larabee-style approach for a system that can both do massive compute AND not have performance plummet if you need branchy code too.
- paulmd 5y ago
  Apple's e-cores are actually (relatively speaking) much better than Intel's. From memory, Apple gets about the same performance (actually higher in single-threaded, but same MT performance) out of about 2/3rds the transistor count as Intel. Note that this is invariant of node - obviously Blizzard cores are physically much smaller than Gracemont, but they are more smaller than you get out of a node shrink alone, apple is doing more with actually less transistors than Intel. Since the Intel e-cores still have a relatively wide decoder, e-core designs may be the part where the bill comes due for x86 in terms of decoder complexity. Sure it's only 3% of a performance core, but if you cut the performance core in half then now they're 6%. And the decoder doesn't shrink that much, Gracemont has a 3-wide decoder vs 4-wide on Golden Cove, and you still have to have the same amount of instruction cache (instruction cache hit rate depends on the amount of "hot code", and programs don't get smaller just because you run them on e-cores). A lot of the x86 "tricks" to keep the cores fed don't scale down much/any. edit: Intel Golden Cove: 7.04mm^2 with L2, 5.55mm^2 w/o L2 Intel Gracemont: 2.2mm^2 with L2, 1.7 mm^2 w/o L2 Apple Avalanche: 2.55mm^2. (I believe these are both w/o cache) Apple Blizzard: 0.69mm^2 (nice) Note that N7 to N5 has roughly 1.6x logic density scaling - so a Blizzard core would be 1.24mm^2 or roughly 73% of the transistor count of Gracemont for equivalent performance! For the p-cores the number is 82%. This is one of the reasons I feel Apple is so far ahead. It's not about raw performance, or even efficiency, it's the fact that Apple is winning on both those metrics while using 2/3rds the transistors. It's not just "apple throwing transistors at the problem", which of course they are, but just they're starting from a much better baseline such that they can afford to throw those transistors around. The higher transistor count in total is coming from the GPU, the cores themselves Apple is actually much more efficient (perf-per-transistor) than x86 competitors.

minhazm 5y ago

I'm guessing you mean double the single core performance and not clock speed. Even Intel Pentium 4's from 2000 had higher clock speeds than Apple's M1 chips. The clock speed matters far less than what you can actually get done in a single clock cycle. Just looking at Apple's own products and their single core performance from Geekbench you can see they achieved around a 40% improvement in just single core performance [1]. So it's not just adding more cores. Apple has usually been the one to hold out on cores and usually has resisted adding more cores to the iPhone / iPad chips in the past and focused on better single core performance. > Your average user is going to prefer double the clock rate of your processors to doubling the number of processors. I disagree. The reality is that these days people are running multi-threaded workloads even if they don't know it. Running a dozen chrome tabs, Slack, Teams, Zoom, some professional tools like IDE's, Adobe creative suite, etc. adds up very quickly to a lot of processes that can use a lot of cores. [1] https://browser.geekbench.com/mac-benchmarks https://browser.geekbench.com/mac-benchmarks

rayiner 5y ago
> Even Intel Pentium 4's from 2000 had higher clock speeds than Apple's M1 chips. It’s wild to see that in print.
- ac29 5y ago
  Isnt true though - the Pentium 4 was released in 2000, but the fastest version available that year was only 1.5GHz. It wouldnt hit 3.2GHz (~M1) until 2003.
pantulis 5y ago
Isn't running multiple single-threaded workloads at the same time also a case for better performance with more cores? So even for basic tasks having more cores is better (although maybe not that cost-efficient)
- katbyte 5y ago
  only if those workloads are very CPU intensive, and the ones that are should probably be parallelized as much as possible anyway.

bcrosby95 5y ago

It's a bit of a chicken-and-egg problem. In general, devs don't target machines that don't exist. But from some quick searching, excel will split out independent calculations into their own threads. So for that, the answer seems to be: it depends. If you're using 20 cores to calculate a single thing, it seems like the answer is "no". But if you're using 20 cores to calculate 20 different things, it seems like the answer is "yes".

paulmd 5y ago

Apple's advantage is actually that it provides much higher single-thread (really, per-thread) performance than "comparable" x86 processors. A M1 Pro can match a 12900HK in perfectly-threaded high-code-density scenarios like Cinebench, with half the thread count. Real-world IPC is something like 3x (!) that of Intel right now - obviously it also clocks lower but Apple hits very hard in lower-thread-count scenarios. If your code is bottlenecked on a single thread, or if it doesn't scale well to higher thread counts, Apple is actually great right now. The downside is that you can't get higher core counts, but that's where the Pro and Ultra SKUs come in. (The real, real downside is that right now you can't get higher core counts on M1 without being tied to a giant GPU you may not even use. What would be really nice is an M1 Ultra-sized chip with 20C or 30C and the same iGPU size as A14 or M1, or a server chip full of e-cores like Denverton or Sierra Forest, but that's very much not Apple's wheelhouse in terms of products unfortunately.)

__init 5y ago
> Real-world IPC is something like 3x (!) that of Intel right now - obviously it also clocks lower That's the problem, though -- if you clock yourself much lower, of course you can get higher IPC; you can pack more into your critical paths. Now, certainly Apple has some interesting and significant innovations over Intel here, but quoting IPC figures like that is highly misleading.
- paulmd 5y ago
  Of course IPC needs to be contextualized, but it's still a very important metric. And Intel's processors aren't clocked 3x higher than Apple either - that would be 9 GHz, and you can't even sustain 5 GHz all-core at 35W let alone 9 GHz which just isn't physically possible even on LN2. https://images.anandtech.com/graphs/graph17024/117496.png https://images.anandtech.com/graphs/graph17024/117496.png That's an absolutely damning chart for x86, at iso-power the M1 Max scores 2.5x as high as a 5980HS in FP and 1.43x as high in integer workloads, despite having just over half the cores and ~0.8x the transistor budget per core. So it's a lot closer to the ~2.5-3x IPC scores than you'd think just from "but x86 clocks higher!". And these results do hold up across the broad spectrum of workloads: https://images.anandtech.com/graphs/graph17024/117494.png https://images.anandtech.com/graphs/graph17024/117494.png Yes, Alder Lake does better (although people always insist to me that Alder still somehow "scales worse at lower power levels than AMD"? That's not what the chart shows...) but even in the best-case scenario, you have Intel basically matching (slightly underperforming) AMD while using twice the thread count. And that's a single, cherrypicked benchmark that is known for favoring raw computation and disregarding performance of the front-end, if you are concerned about the x86 front-end, this is basically a best-case scenario for it... high code compactness and extremely high threadability. And it still needs twice the threads to do it. https://i.imgur.com/vaYTmDF.png https://i.imgur.com/vaYTmDF.png Like your "but x86 uses higher clock rates", you can also say "but x86 uses SMT", so maybe "performance per thread" is an unfair metric in some sense, but there is practical merit to it. If you have to use twice the threads to achieve equal performance on x86 then that's a downside, where Apple gives you high performance on tasks that don't scale to higher thread counts. And if Apple put out a processor with high P-core count and without the giant GPU, it would easily be the best hardware on the market. I just strongly doubt that "it's all node" like everyone insists. Apple is running fewer transistors per core already, and AMD/Intel are not going to double or triple their performance-per-thread within the next generation regardless of how many transistors they might use to do it (AMD will be on N5P this year, which will be node parity with Apple A15). x86 vendors can put out a product that will be competitive in one of several areas, but they can't win all of them at once like Apple can. And going forward - it's hard to see how x86 fixes that IPC gap. You can't scale the decoder as wide, Golden Cove already has a pretty big decoder in fact. A lot of the "tricks" have already been used. How do you triple IPC in that scenario, without blowing up transistor budgets hugely? Even if you get rid of SMT, you're not going to triple IPC. Or, how do you triple clockrate in an era when things are actually winding backwards? Others are very, very confident this lead will disappear when AMD moves to N5P. I just don't see it imo. The gap is too big.

tomjen3 5y ago

Amdahals law is sometimes brought out as a whammy (not saying that is what you do), but economically we would expect that there will be a usage for this extra compute: we did it server side by hosting SAAS and thereby adding more users than anybody could think of. I suspect that rather than 1000 cores we might start to see more levels of cores, and hardware for more things. Already Apple has video encoding support. AI seems an obvious idea and it scales much better than most classical computing. If I may bring up something that may be more of a wish: I wish that we could give up the idea of shared memory and we could have many more cores that communicated by shared messaging. We are already seeing this spread with webworkers - if it became cheap to create a new thread and computers weren't bottlenecked then maybe more games would use it too.

samwillis 5y ago
Message passing without shared memory is slow, having to copy data and maybe even serialise/deserialise it in the process. Message passing where you are just passing a pointer is fast. Web workers are basically the worst case, you have to serialise your data to and from JSON when passing it to and from a worker. It’s not built for performance. There have been many cases where people have tried to improve performance of their apps by offloading work to a web worker but the added cost of serialisation ultimately made it slower than running on the main thread.

gameswithgo 5y ago

It can accelerate compilation times quite a bit at times, which could allow us to use more compile time safety checks and more compile time optimizations, indirectly making the lower core count computers faster.