4 ms·

In fact the CPU stagnation is worse. In contrast to the shrinking chip size and higher frequencies of past scaling laws, CPU core count does not easily translat

In fact the CPU stagnation is worse. In contrast to the shrinking chip size and higher frequencies of past scaling laws, CPU core count does not easily translate to performance gains. It needs specialized and careful designed software to tap the capability. In fact if CPU core count did translate more easily to performance gains I think already with the existing CPU's we'd have a fairly signficant one-time boost. Maybe somebody has statistical survey of how much of the existing deployed CPU core count is typically used?

rbanffy 3y ago
> It needs specialized and careful designed software to tap the capability. I've been suggesting engineers get more cores of lower speed to gain insight on what will be performant a few years down the road since I saw my first Xeon Phi. It's been a while since clock speeds got higher (IBM has been pushing 5GHz in their highest end for the past couple years now and it doesn't seem likely they'll cross 6 anytime soon), but we get more cores every year. We now have 4-core entry-level machines and 2-core/4-thread ones are the bottom of the barrel, with a decent one being 8-core. Ampère just announced a 192-core server beast. And then we have another thing: performance for most users has been "good enough" for the past couple decades. I haven't gotten a new computer just because it had a faster CPU since the early 2000's - they usually turn to dust well before they become too slow to use. My wife will need to upgrade her Macbook soon-ish for regulatory reasons (when Apple EOLs and stops patching macOS 12) and her laptop is still going strong. Considering that, there is little advantage in making all but the most demanding software more parallel. This leaves the high-end, the stuff that needs a POWER10 or a Telum to run at acceptable speeds, and the cloud vendors, who'd kill to be able to serve 1% more VMs per kilowatt because 1% of their revenue is the GDP of a small country.
- robocat 3y ago
  > and the cloud vendors, who'd kill to be able to serve 1% more VMs per kilowatt If Google's only expense were electricity, and profit margin was 50%, then saving 1% on power bill would increase profits by 1%. I skimmed the Alphabet annual report: saw $260 revenue, $80 profit, $110 operating expenses (electricity and staff). Say $10 on power, then 1% is $0.1 - improve profits a bit over 0.1%. Anyone know how many $/year Google spends on power?
phkahler 3y ago
>> Maybe somebody has statistical survey of how much of the existing deployed CPU core count is typically used? My guess is very few cores are used on average. I did some testing with Solvespace to see which build options contributed most to performance: https://github.com/solvespace/solvespace/issues/972 https://github.com/solvespace/solvespace/issues/972 Obviously using OpenMP for multi-core was the big win. But what's not shown is that in typical usage (not the test I ran) if you're dragging some geometry around it will use all cores (in my case 4 cores / 8 threads) at about 50 percent utilization. That percentage probably drops as more cores are thrown at it due to Amdahl's Law. In other words, throwing double the cores at it will give a good boost to a lot of code that is already taking less than half the time (wall clock time, not CPU time). We added OpenMP to a number of functions for significant performance gains. And in fact, any remining single-thread operation that gets the parallel treatment is likely to have a significant impact on overall performance since that is where most of the time is spent now. At this point we're more focused on features and bugs. Algorithmic improvements are possible and I'd like to do those in the future, but they are much harder to do than sprinkling some #pragmas around critical loops. That will improve the scalability though, where multithreading really did not.
jandrewrogers 3y ago
CPU core count does translate to performance gains but the popular software architecture idioms most people use are incapable of taking advantage of large numbers of cores. The gap between the performance you typically see and what is possible with proper software architecture and performance engineering is orders of magnitude in scale. A lot of performance and scalability is left on the table. We've known how to scale software on large silicon for a long time, but as an industry we mostly can't be bothered (or lack the skills) to do it.
- pjmlp 3y ago
  It will hardly change as long as scripting languages keep being used for full blown applications.
- JamesBarney 3y ago
  The simple reason is devs are more expensive than servers for most applications.
duped 3y ago
> CPU core count does not easily translate to performance gains. In theory, performance scales logistically with the number of parallel processors (Amdahl's law). In practice, the limit is (and has always been) memory and i/o. That's why Apple silicon kicks everyone's ass. If we want faster computers, the biggest gains are not to be found in making processors do more work. It's in designing systems (not just CPUs) that don't let the CPU wait around to do work.
- lostmsu 3y ago
  > That's why Apple silicon kicks everyone's ass. ??? But it doesn't: https://browser.geekbench.com/processor-benchmarks https://browser.geekbench.com/processor-benchmarks https://browser.geekbench.com/mac-benchmarks https://browser.geekbench.com/mac-benchmarks
  - eppsilon 3y ago
    *kicks ass per-watt
    - lostmsu 3y ago
      The parent was talking about unified memory.
  - owenmarshall 3y ago
    Doesn’t Geekbench overwhelmingly measure raw CPU performance, whereas GP was talking about overall system performance? Isn’t this chalk and cheese?
    - lostmsu 3y ago
      Well, what other benchmark results say otherwise?
- ketralnis 3y ago
  > In practice, the limit is (and has always been) memory and i/o Absolutely not. In practise the limit is (1) how many cores are actually _used_ by programs and (2) how much work is put into making anything fast at all, ever. We're using web frontends powered by python backends over a network. The vast majority of programs use nowhere near the resoures available to them.
inconceivable 3y ago
multi-core architectures have been optimized for web servers i.e. tons of clients hitting relatively simple backend processes. also, my day-to-day gui workstation (ancient xeon mac) gets far more core usage than my dev server (threadripper) which is honestly kind of a waste of money, but it's shiny so i bought it anyway. it should last me a decade+ just like my gui workstation has. now that single-core GPU/CPU/TPU/whatever performance is back on the front burner i think we'll see some horsepower and compiler improvements over the next few years. luckily the i/o problem has made great strides in the meantime so network/memory/storage will be there to support it, unlike in the past. ecc ram is also plummeting in cost, so that's good.
sangnoir 3y ago
> It needs specialized and careful designed software to tap the capability. I wouldn't exactly call Golang[1] "specialized", but it does make multiprocessing easier than most languages. 1.Or Erlang or Elixir
- nologic01 3y ago
  I only use go through hugo but its fast enough I never checked how many cores it uses :-)
JonChesterfield 3y ago
Or it needs multiple processes running at the same time. Notably running lots of C++ compilers simultaneously works great and really likes the high core count machines.
nabakin 3y ago
I was looking for this comment and I think you're exactly right. Over the years, Moore's Law became a household term for computer performance doubling every couple of years. Under that definition, Moore's Law died in 2005 with Dennard Scaling so for most intents and purposes, Moore's Law has been dead for a long time. It only held under the more restrictive definition of performance for tasks that were able to be parallelized perfectly, but even that has now been broken. You could also argue that Moore's Law died in 2005 because the term CPU used to refer to what we now know as a CPU 'core' and the term was redefined. Ultimately, what matters is that the performance the end user experiences hasn't been doubling every 2 years since 2005.
crote 3y ago
I'm not convinced that this is the case. A lot of computation-heavy workloads do in fact scale rather well with an increased core count. That's why GPUs are essentially using thousands of cores, after all: nontrivial computation virtually always implies a sizeable amount of data, and a lot of data usually implies that you can split it up into multiple sections to compute in parallel. A lot of desktop code is still single-core, that is true. However desktop CPUs are idle >90% of the time, only waking up to do a small burst of computation every once in a while. You're not going to notice some event loop finishing in 0.8ms instead of 1ms. Even then it'll probably be running multiple tasks from multiple processes in parallel, making good use of the available cores. CPUs have simply gotten fast enough that they aren't a bottleneck for desktop use anymore!
pornel 3y ago
It does require software taking advantage, but this was a chicken-egg problem caused by Intel's stagnation. When most CPUs had only 4 cores, developers had an easy excuse: two of the cores were "fake" HT cores so they didn't count, they'd use one core, and the other core was for the OS or other apps, so there's basically no point complicating software for so little gain. Only recently AMD has opened the floodgates and we got consumer systems with 16 cores, where finally it's really hard to find an excuse to leave 15 cores idle, and there's so much raw power that even suboptimal scaling can give a big performance boost. Also Rust became a thing, and it makes much easier to write reliable multi-core software, so I'm optimistic about software catching up.
- fbdab103 3y ago
  I mean Microsoft did just brag about cutting the Teams launch time to 9 seconds. Progress, baby!
  - zirgs 3y ago
    Imagine if something like a digital camera booted up in 9 seconds.

rbanffy 3y ago

> It needs specialized and careful designed software to tap the capability. I've been suggesting engineers get more cores of lower speed to gain insight on what will be performant a few years down the road since I saw my first Xeon Phi. It's been a while since clock speeds got higher (IBM has been pushing 5GHz in their highest end for the past couple years now and it doesn't seem likely they'll cross 6 anytime soon), but we get more cores every year. We now have 4-core entry-level machines and 2-core/4-thread ones are the bottom of the barrel, with a decent one being 8-core. Ampère just announced a 192-core server beast. And then we have another thing: performance for most users has been "good enough" for the past couple decades. I haven't gotten a new computer just because it had a faster CPU since the early 2000's - they usually turn to dust well before they become too slow to use. My wife will need to upgrade her Macbook soon-ish for regulatory reasons (when Apple EOLs and stops patching macOS 12) and her laptop is still going strong. Considering that, there is little advantage in making all but the most demanding software more parallel. This leaves the high-end, the stuff that needs a POWER10 or a Telum to run at acceptable speeds, and the cloud vendors, who'd kill to be able to serve 1% more VMs per kilowatt because 1% of their revenue is the GDP of a small country.

robocat 3y ago
> and the cloud vendors, who'd kill to be able to serve 1% more VMs per kilowatt If Google's only expense were electricity, and profit margin was 50%, then saving 1% on power bill would increase profits by 1%. I skimmed the Alphabet annual report: saw $260 revenue, $80 profit, $110 operating expenses (electricity and staff). Say $10 on power, then 1% is $0.1 - improve profits a bit over 0.1%. Anyone know how many $/year Google spends on power?

phkahler 3y ago

>> Maybe somebody has statistical survey of how much of the existing deployed CPU core count is typically used? My guess is very few cores are used on average. I did some testing with Solvespace to see which build options contributed most to performance: https://github.com/solvespace/solvespace/issues/972 https://github.com/solvespace/solvespace/issues/972 Obviously using OpenMP for multi-core was the big win. But what's not shown is that in typical usage (not the test I ran) if you're dragging some geometry around it will use all cores (in my case 4 cores / 8 threads) at about 50 percent utilization. That percentage probably drops as more cores are thrown at it due to Amdahl's Law. In other words, throwing double the cores at it will give a good boost to a lot of code that is already taking less than half the time (wall clock time, not CPU time). We added OpenMP to a number of functions for significant performance gains. And in fact, any remining single-thread operation that gets the parallel treatment is likely to have a significant impact on overall performance since that is where most of the time is spent now. At this point we're more focused on features and bugs. Algorithmic improvements are possible and I'd like to do those in the future, but they are much harder to do than sprinkling some #pragmas around critical loops. That will improve the scalability though, where multithreading really did not.

jandrewrogers 3y ago

CPU core count does translate to performance gains but the popular software architecture idioms most people use are incapable of taking advantage of large numbers of cores. The gap between the performance you typically see and what is possible with proper software architecture and performance engineering is orders of magnitude in scale. A lot of performance and scalability is left on the table. We've known how to scale software on large silicon for a long time, but as an industry we mostly can't be bothered (or lack the skills) to do it.

pjmlp 3y ago
It will hardly change as long as scripting languages keep being used for full blown applications.
JamesBarney 3y ago
The simple reason is devs are more expensive than servers for most applications.

duped 3y ago

> CPU core count does not easily translate to performance gains. In theory, performance scales logistically with the number of parallel processors (Amdahl's law). In practice, the limit is (and has always been) memory and i/o. That's why Apple silicon kicks everyone's ass. If we want faster computers, the biggest gains are not to be found in making processors do more work. It's in designing systems (not just CPUs) that don't let the CPU wait around to do work.

lostmsu 3y ago
> That's why Apple silicon kicks everyone's ass. ??? But it doesn't: https://browser.geekbench.com/processor-benchmarks https://browser.geekbench.com/processor-benchmarks https://browser.geekbench.com/mac-benchmarks https://browser.geekbench.com/mac-benchmarks
- eppsilon 3y ago
  *kicks ass per-watt
  - lostmsu 3y ago
    The parent was talking about unified memory.
- owenmarshall 3y ago
  Doesn’t Geekbench overwhelmingly measure raw CPU performance, whereas GP was talking about overall system performance? Isn’t this chalk and cheese?
  - lostmsu 3y ago
    Well, what other benchmark results say otherwise?
ketralnis 3y ago
> In practice, the limit is (and has always been) memory and i/o Absolutely not. In practise the limit is (1) how many cores are actually _used_ by programs and (2) how much work is put into making anything fast at all, ever. We're using web frontends powered by python backends over a network. The vast majority of programs use nowhere near the resoures available to them.

inconceivable 3y ago

multi-core architectures have been optimized for web servers i.e. tons of clients hitting relatively simple backend processes. also, my day-to-day gui workstation (ancient xeon mac) gets far more core usage than my dev server (threadripper) which is honestly kind of a waste of money, but it's shiny so i bought it anyway. it should last me a decade+ just like my gui workstation has. now that single-core GPU/CPU/TPU/whatever performance is back on the front burner i think we'll see some horsepower and compiler improvements over the next few years. luckily the i/o problem has made great strides in the meantime so network/memory/storage will be there to support it, unlike in the past. ecc ram is also plummeting in cost, so that's good.

sangnoir 3y ago

> It needs specialized and careful designed software to tap the capability. I wouldn't exactly call Golang[1] "specialized", but it does make multiprocessing easier than most languages. 1.Or Erlang or Elixir

nologic01 3y ago
I only use go through hugo but its fast enough I never checked how many cores it uses :-)

JonChesterfield 3y ago

Or it needs multiple processes running at the same time. Notably running lots of C++ compilers simultaneously works great and really likes the high core count machines.

nabakin 3y ago

I was looking for this comment and I think you're exactly right. Over the years, Moore's Law became a household term for computer performance doubling every couple of years. Under that definition, Moore's Law died in 2005 with Dennard Scaling so for most intents and purposes, Moore's Law has been dead for a long time. It only held under the more restrictive definition of performance for tasks that were able to be parallelized perfectly, but even that has now been broken. You could also argue that Moore's Law died in 2005 because the term CPU used to refer to what we now know as a CPU 'core' and the term was redefined. Ultimately, what matters is that the performance the end user experiences hasn't been doubling every 2 years since 2005.

crote 3y ago

I'm not convinced that this is the case. A lot of computation-heavy workloads do in fact scale rather well with an increased core count. That's why GPUs are essentially using thousands of cores, after all: nontrivial computation virtually always implies a sizeable amount of data, and a lot of data usually implies that you can split it up into multiple sections to compute in parallel. A lot of desktop code is still single-core, that is true. However desktop CPUs are idle >90% of the time, only waking up to do a small burst of computation every once in a while. You're not going to notice some event loop finishing in 0.8ms instead of 1ms. Even then it'll probably be running multiple tasks from multiple processes in parallel, making good use of the available cores. CPUs have simply gotten fast enough that they aren't a bottleneck for desktop use anymore!

pornel 3y ago

It does require software taking advantage, but this was a chicken-egg problem caused by Intel's stagnation. When most CPUs had only 4 cores, developers had an easy excuse: two of the cores were "fake" HT cores so they didn't count, they'd use one core, and the other core was for the OS or other apps, so there's basically no point complicating software for so little gain. Only recently AMD has opened the floodgates and we got consumer systems with 16 cores, where finally it's really hard to find an excuse to leave 15 cores idle, and there's so much raw power that even suboptimal scaling can give a big performance boost. Also Rust became a thing, and it makes much easier to write reliable multi-core software, so I'm optimistic about software catching up.

fbdab103 3y ago
I mean Microsoft did just brag about cutting the Teams launch time to 9 seconds. Progress, baby!
- zirgs 3y ago
  Imagine if something like a digital camera booted up in 9 seconds.