4 ms·

"The downside is that for the past couple of decades computers haven't gotten much faster, except in ways that require recoding (like GPUs and multicore)." Thi

"The downside is that for the past couple of decades computers haven't gotten much faster, except in ways that require recoding (like GPUs and multicore)." This is false? Computers have gotten a lot faster, even if the clock speed is not that much faster. A single modern CPU core turboing at ~5Ghz is going to be significantly faster than a 20 year old cpu overclocked to ~4.5Ghz.

jeffbee 2y ago
Yeah, that detail sinks the rest of it. Even if we assume datacenter CPUs were the market preference has been for more cores operating at the same ~2400MHz speed for a long time, what you get for 1 CPU-second these days is ridiculous compared to what you could have gotten 20 years ago. We're talking about NetBurst Xeons as a baseline.
gopalv 2y ago
> Computers have gotten a lot faster, even if the clock speed is not that much faster We're not stagnating but the same code I thought was too slow in 1998 was good enough in 2008, which is probably not true for code I would've thrown away in 2015. The only place where that has happened in the last decade is for IOPS - old IOPS heavy code which would have been rewritten with group-commit tricks is probably slower than a naive implementation that fsync'd all the time. A 2015 first cut of IO code probably beats the spinning disk optimized version from the same year on modern hardware. The clock-speed comment is totally on the money though - a lot of the clocks were spent waiting for memory latencies and those have improved significantly across the years particularly if you use an Apple Silicon style memory which is physically closer in a light cone from the DIMMs of the past.
- Legend2440 2y ago
  A lot of clocks are still spent waiting for memory. GPUs in particular are limited by memory bandwidth despite a memory bus that runs at terabytes per second. Back when I started programming, it was reasonable to precompute lookup tables for multiplications and trig functions. Now you'd never do that - it's far cheaper to recompute it than to look it up from memory.
  - rbanffy 2y ago
    Indeed. When I was in college I designed a stack-based CPU. It was a perfectly sane design decision to keep the stack in main memory. It made the design VERY simple (but I kept a couple registers in the architecture, just in case). These days it'd be an unbelievably stupid decision.
paulsutter 2y ago
Could you share some numbers on this? Lots of folks would be interested I'm sure
- deleted 2y ago
  [deleted]
PaulKeeble 2y ago
An intel 12900k (Gen 12) compared to a 2600k (Gen 2, launched 2011) is about 120% faster or a bit over 2 times in single threaded applications, those +5-15% uplifts every generation add up over time but its nothing like the earlier years when they might double in performance in a single generation. It really depends if that application uses AES 256 bit and other modern instructions. The 12900k has 16 cores vs 4 of the 2600k, although 8 of those extra cores are E-cores. This performance increase doesn't necessarily come from free given the application may need to be adjusted to utilise those extra cores especially when half of them are slower to ensure the workload is distributed properly. Even within a vertical scaling by getting a new processor for just single threaded applications its interesting that much of the big benefits come from targeting the new instructions and then the new cores. Both of which may require source updates to get significant performance uplift from. https://www.cpu-monkey.com/en/compare_cpu-intel_core_i7_12700k-vs-intel_core_i7_2600k https://www.cpu-monkey.com/en/compare_cpu-intel_core_i7_1270...
- einpoklum 2y ago
  > is about 120% faster or a bit over 2 times in single threaded applications 1. Doesn't that also account for speedups in memory and I/O? 2. Even if the app is single-threaded, the OS isn't, so unless it's very very inactive other than the foreground application (which is possible), there might still be an effect of the higher core count.
  - jaggederest 2y ago
    Unless you're multitasking, the OS on a separate thread gets you about 5-10% speedup. It's not really noteworthy. Unless you lived through the 1990s I don't think you understand how fast things were improving. Routine doubling of scores every 18 months is an insane thing. In 1990 the state of the art was 8mhz chips. By 2002, the state of the art was a 5ghz chip. So almost a thousand times faster in a decade. Are chips now a thousand times faster than they were in 2015? No they are not.
    - sidewndr46 2y ago
bee_rider 2y ago
I think it is often the case that people want to describe the problem as “single core performance has stagnated for decades” because it makes it look like their solution is necessary to make any progress at all. Actually, single core performance has been improving. Not as fast as it was in the 90’s maybe, but it is improving. However, we can speed things up even more by using multiple computers. And it is a really interesting problem where you get to worry about all sorts of fun things, like hiding MPI communication between compute. Nobody wants to say “I have found that if I can make an already fast process even faster by putting in a lot of effort, which I will do because my job is actually really fun.” Technical jobs are supposed to be stressful and serious. The world is doomed and science will stop… unless I come up with a magic trick!
- Legend2440 2y ago
  Single-core performance looks pretty stagnant on this graph, especially in the last ten years: https://imgur.com/DrOvPZt https://imgur.com/DrOvPZt Transistor count has continued to increase exponentially, but single-threaded performance has improved slowly and appears to be leveling off. We may never get another 100x or even 10x improvement in single-threaded performance. It is going to be necessary to parallelize to see gains in the future.
  - achierius 2y ago
    But it's not flat? 10% growth a year is still growth.
    - rbanffy 2y ago
      Moore's law got my generation spoiled. We had a computer that was twice as fast if we waited for a year.
  - bee_rider 2y ago
    I mean, it definitely isn’t the 90’s anymore, for sure. But I don’t think that chart tells the whole story. First off, it only shows half of the last ten years. Things feel like they’ve been picking up a little bit, with Apple and AMD executing fairly well. Second, average is being dragged down a bit as thin-and-light laptops have become very popular. A lot of the chips in that chart could probably be run faster on a higher energy budget, right?
Ygg2 2y ago
> This is false? Computers have gotten a lot faster Depends, what you mean by much. Single threaded performance is no longer 2x fast after a year. I mean, even in the GPU section, you get graphics that looks slightly better for 2-4x the cost (see street prices of 2080 vs 3080 vs 4080). Computing has hit the point of diminishing returns, exponetial growth for linear prices is no longer possible.
foota 2y ago
I think this is meant to be read as, "over the past decade, you haven't been able to wait a year and buy a new CPU to solve your vertical scalability issues.", not necessarily to claim that there hasn't been significant growth when compared over the entire window.
- hinkley 2y ago
  m4, m5, and m6 had a bit of oomph per dollar that was nice to have. Then Amazon got greedy and priced the 7's a little more dearly, and it turns out depending on your workload the math is wrong, particularly for the c6i and c7i versus the m-series, where I think their ratios got a bit wonky. Their benchmarks were not reflective of the real world applications I had to deploy the way they had been in previous generations. I feel like this is a case of them believing their own PR to everyone's detriment. They're going to have to lower their prices on both the Intel and AMD versions by at least 5% before it even became price competitive to upgrade. Even if you ignored the rounding errors of needing for instance X.3 machines to keep the same response curve, which played out for us and greatly compounded my indifference for the new machines. There's a good chance that team will skip straight from m6 to m8 when they are available since as far as I'm aware they rarely change their prices.
lmm 2y ago
> A single modern CPU core turboing at ~5Ghz is going to be significantly faster than a 20 year old cpu overclocked to ~4.5Ghz. Is it actually going to be faster than a P4 running at the speeds that they can clock up to, especially if we're talking about sustained performance rather than a brief turbo? Obviously "non-vectorisable integer computation that never stalls a 70 cycle pipeline" is not the only thing that matters for software performance, but it's worth comparing.

jeffbee 2y ago

Yeah, that detail sinks the rest of it. Even if we assume datacenter CPUs were the market preference has been for more cores operating at the same ~2400MHz speed for a long time, what you get for 1 CPU-second these days is ridiculous compared to what you could have gotten 20 years ago. We're talking about NetBurst Xeons as a baseline.

gopalv 2y ago

> Computers have gotten a lot faster, even if the clock speed is not that much faster We're not stagnating but the same code I thought was too slow in 1998 was good enough in 2008, which is probably not true for code I would've thrown away in 2015. The only place where that has happened in the last decade is for IOPS - old IOPS heavy code which would have been rewritten with group-commit tricks is probably slower than a naive implementation that fsync'd all the time. A 2015 first cut of IO code probably beats the spinning disk optimized version from the same year on modern hardware. The clock-speed comment is totally on the money though - a lot of the clocks were spent waiting for memory latencies and those have improved significantly across the years particularly if you use an Apple Silicon style memory which is physically closer in a light cone from the DIMMs of the past.

Legend2440 2y ago
A lot of clocks are still spent waiting for memory. GPUs in particular are limited by memory bandwidth despite a memory bus that runs at terabytes per second. Back when I started programming, it was reasonable to precompute lookup tables for multiplications and trig functions. Now you'd never do that - it's far cheaper to recompute it than to look it up from memory.
- rbanffy 2y ago
  Indeed. When I was in college I designed a stack-based CPU. It was a perfectly sane design decision to keep the stack in main memory. It made the design VERY simple (but I kept a couple registers in the architecture, just in case). These days it'd be an unbelievably stupid decision.

paulsutter 2y ago

Could you share some numbers on this? Lots of folks would be interested I'm sure

deleted 2y ago
[deleted]

PaulKeeble 2y ago

An intel 12900k (Gen 12) compared to a 2600k (Gen 2, launched 2011) is about 120% faster or a bit over 2 times in single threaded applications, those +5-15% uplifts every generation add up over time but its nothing like the earlier years when they might double in performance in a single generation. It really depends if that application uses AES 256 bit and other modern instructions. The 12900k has 16 cores vs 4 of the 2600k, although 8 of those extra cores are E-cores. This performance increase doesn't necessarily come from free given the application may need to be adjusted to utilise those extra cores especially when half of them are slower to ensure the workload is distributed properly. Even within a vertical scaling by getting a new processor for just single threaded applications its interesting that much of the big benefits come from targeting the new instructions and then the new cores. Both of which may require source updates to get significant performance uplift from. https://www.cpu-monkey.com/en/compare_cpu-intel_core_i7_12700k-vs-intel_core_i7_2600k https://www.cpu-monkey.com/en/compare_cpu-intel_core_i7_1270...

einpoklum 2y ago
> is about 120% faster or a bit over 2 times in single threaded applications 1. Doesn't that also account for speedups in memory and I/O? 2. Even if the app is single-threaded, the OS isn't, so unless it's very very inactive other than the foreground application (which is possible), there might still be an effect of the higher core count.
- jaggederest 2y ago
  Unless you're multitasking, the OS on a separate thread gets you about 5-10% speedup. It's not really noteworthy. Unless you lived through the 1990s I don't think you understand how fast things were improving. Routine doubling of scores every 18 months is an insane thing. In 1990 the state of the art was 8mhz chips. By 2002, the state of the art was a 5ghz chip. So almost a thousand times faster in a decade. Are chips now a thousand times faster than they were in 2015? No they are not.
  - sidewndr46 2y ago

bee_rider 2y ago

I think it is often the case that people want to describe the problem as “single core performance has stagnated for decades” because it makes it look like their solution is necessary to make any progress at all. Actually, single core performance has been improving. Not as fast as it was in the 90’s maybe, but it is improving. However, we can speed things up even more by using multiple computers. And it is a really interesting problem where you get to worry about all sorts of fun things, like hiding MPI communication between compute. Nobody wants to say “I have found that if I can make an already fast process even faster by putting in a lot of effort, which I will do because my job is actually really fun.” Technical jobs are supposed to be stressful and serious. The world is doomed and science will stop… unless I come up with a magic trick!

Legend2440 2y ago
Single-core performance looks pretty stagnant on this graph, especially in the last ten years: https://imgur.com/DrOvPZt https://imgur.com/DrOvPZt Transistor count has continued to increase exponentially, but single-threaded performance has improved slowly and appears to be leveling off. We may never get another 100x or even 10x improvement in single-threaded performance. It is going to be necessary to parallelize to see gains in the future.
- achierius 2y ago
  But it's not flat? 10% growth a year is still growth.
  - rbanffy 2y ago
    Moore's law got my generation spoiled. We had a computer that was twice as fast if we waited for a year.
- bee_rider 2y ago
  I mean, it definitely isn’t the 90’s anymore, for sure. But I don’t think that chart tells the whole story. First off, it only shows half of the last ten years. Things feel like they’ve been picking up a little bit, with Apple and AMD executing fairly well. Second, average is being dragged down a bit as thin-and-light laptops have become very popular. A lot of the chips in that chart could probably be run faster on a higher energy budget, right?

Ygg2 2y ago

> This is false? Computers have gotten a lot faster Depends, what you mean by much. Single threaded performance is no longer 2x fast after a year. I mean, even in the GPU section, you get graphics that looks slightly better for 2-4x the cost (see street prices of 2080 vs 3080 vs 4080). Computing has hit the point of diminishing returns, exponetial growth for linear prices is no longer possible.

foota 2y ago

I think this is meant to be read as, "over the past decade, you haven't been able to wait a year and buy a new CPU to solve your vertical scalability issues.", not necessarily to claim that there hasn't been significant growth when compared over the entire window.

hinkley 2y ago
m4, m5, and m6 had a bit of oomph per dollar that was nice to have. Then Amazon got greedy and priced the 7's a little more dearly, and it turns out depending on your workload the math is wrong, particularly for the c6i and c7i versus the m-series, where I think their ratios got a bit wonky. Their benchmarks were not reflective of the real world applications I had to deploy the way they had been in previous generations. I feel like this is a case of them believing their own PR to everyone's detriment. They're going to have to lower their prices on both the Intel and AMD versions by at least 5% before it even became price competitive to upgrade. Even if you ignored the rounding errors of needing for instance X.3 machines to keep the same response curve, which played out for us and greatly compounded my indifference for the new machines. There's a good chance that team will skip straight from m6 to m8 when they are available since as far as I'm aware they rarely change their prices.

lmm 2y ago

> A single modern CPU core turboing at ~5Ghz is going to be significantly faster than a 20 year old cpu overclocked to ~4.5Ghz. Is it actually going to be faster than a P4 running at the speeds that they can clock up to, especially if we're talking about sustained performance rather than a brief turbo? Obviously "non-vectorisable integer computation that never stalls a 70 cycle pipeline" is not the only thing that matters for software performance, but it's worth comparing.