6 ms·

noob question, but how is the performance better? I thought clocks weren't increasing (due to heat), so is it that smaller chips mean more per wafer, therefore

noob question, but how is the performance better? I thought clocks weren't increasing (due to heat), so is it that smaller chips mean more per wafer, therefore cheaper and you can buy more? I recall a Sophie Wilson talk about how things will never get faster, past 29nm.

ben-schaaf 5y ago
Clocks aren't increasing much, but IPC still is. Density is also still increasing meaning lower latencies, lower power and therefore higher performance per area.
- Causality1 5y ago
  IPC is going up but slowly compared to previous generations. If AMD can sustain the kind of generation on generation increases it achieved between Zen 2 and 3 I will be tremendously impressed, but as it stands my brand new 5600X has barely 50% more single-core performance than the seven year old 4670K I replaced.
  - flumpcakes 5y ago
    I have a hunch that it would be much greater than 50% if the software was recompiled to take advantage of the newer instruction sets. Then again, you won't be comparing exact binary copies of software, which some people might say invalidates it as a benchmark/comparison. I remember many, many years ago my PC couldn't run the latest adobe software as I was missing SSE2 instructions on my then current CPU. I believe there was some hidden compatability mode, but the software ran extremely poorly vs. the version of software that was out before that didn't require SSE2.
    - aseipp 5y ago
      There have been almost no new broad "performance-oriented" instruction sets introduced since the Haswell era, all the way to the Zen 3 era (e.g. SIMD/vector) for compilers to target. At least not any instructions that the vast majority of software is going to see magical huge improvements from and are wide spread enough to justify it (though specific software may benefit greatly from some specific tuning; pext/pdep and AVX-512 for instance.) The microarchitectures have improved significantly though, which does matter. For instance, Haswell-era AVX2 implementations were significantly poorer than the modern ones in, say, Tiger Lake or Zen 3. The newer ones have completely different power usage and per-core performance characteristics for AVX code; even if you could run AVX2 on older processors, it might not have actually been a good idea if the cumulative slowdowns they cause impacts the whole system (because the chips had to downclock the whole system so they wouldn't brownout). So it's not just a matter of instruction sets, but their individual performance characteristics. And also, it is not just CPUs that have improved. If anything, the biggest improvements have been in storage devices across the stack, which now have significantly better performance and parallelism, and the bandwidth has improved too (many more PCIe lanes). I can read gigabytes a second from a single NVMe drive; millions of IOPS a second, which is vastly better than you could 7 years ago on a consumer-level budget. Modern machines do not just crunch scalar code in isolation, and neither did older ones; we could just arbitrage CPU cycles more often than we can now in an era where a lot of the performance "cliffs" have been dealt with. Isolating things to just look at how fast the CPU can retire instructions is a good metric for CPU designers, but it's a very incomplete view when viewing the system as a whole as an application developer.
      - AtlasBarfed 5y ago
        I'm not deep on the weeds of high-performance compilers, but just because ISA evolution hasn't happened doesn't mean compilers can't evolve to use the silicon better. There always was an "Intel advantage" to compilers for decades (admittedly Intel invested in compilers more than AMD, but they also were sneaky about trying to nerf AMD in compilers), but with AMD being such a clear leader for so many years, I would hope at least GCC has started supporting AMD flavors of compilation better. Anyone know if this has happened with GCC and AMD silicon? Or at least is there a better body of knowledge of what GCC flags help AMD more?
        scns 5y ago
        march=native
        aseipp 5y ago
        Yes, I consider this to fall under the umbrella of general microarchitectural improvements I mentioned. GCC and LLVM are regularly updated with microarchitectural scheduling models to better emit code that matches the underlying architecture, and have featured these for at least 5-7 years; there can be a big difference between say, Skylake and Zen 2, for instance, so targeting things appropriately is a good idea. You can use the `-march` flag for your compiler to target specific architectures, for instance -march=tigerlake or -march=znver3 But in general I think it's a bit of a red herring for the thrust of my original post; first off you always have to target the benchmark to test a hypothesis, you don't run them in isolation for zero reason. My hypothesis when I ran my own for instance was "General execution of bog standard scalar code is only up by about 50-60%" and using the exact same binary instructions was the baseline criteria for that; it was not "Does targeting a specific microarchitecture scheduling model yield specific gains." If you want to test the second one, you need to run another benchmark. There are too many factors for any particular machine for any such post to be comprehensive, as I'm sure you're aware. I'm just speaking in loose generalities.
        jmgao 5y ago
        > You can use the `-march` flag for your compiler to target specific architectures, for instance -march=tigerlake or -march=znver3 Note that -march will use instructions that might be unavailable on other CPUs of the target. -mtune (which is implied by -march) is the flag that sets the cost tables used by instruction selection, cache line sizes, etc.
  - jbluepolarbear 5y ago
    I doubt that very much. I have a i7 4790 and I have Ryzen 7 3700X. In my testing single core speed is nearly 2.5 times in favor of my Ryzen. What where you using as bench marks? My test was a single threaded software rasterizer with real-time vertex lighting. That I compiled in vs c++ 2008 around 2010.
    - aseipp 5y ago
      Basic single-core scalar-only workloads of my own corroborate the grandparent, as well as most of the other benchmarks I've seen. My own 5600X is "only" about 50-60% better than my old Haswell i5-4950 (from Q2 '14) on this note. But the scalar speed isn't everything, because you're often not bounded solely by retirement in isolation (the system, in aggregate, is an open one not a closed one.) Fatter caches and extraordinarily improved storage devices with lots of parallelism (even on a single core you can fire off a ton of asynchronous I/O at the device) make a huge difference here even for single-core workloads, because you can actually keep the core fed well enough to do work. So the cumulative improvement is even better in practice.
      - jbluepolarbear 5y ago
        Now I’m curious, is this testing against some requirements for an application? Are they any applications that would benefit solely from scalar performance vs scalar, cache size, and memory speed.
    - speeder 5y ago
      Need to point out in particular that that best Haswell i5 was FASTER than the best i7 for single-core workloads, so this might be a factor in OP post. And also that is the reason why I use such processor on my own computer, it was already "outdated" when I bought it, but since one of the things I like to do is play some simulation style games that rely heavily on single-core performance, I choose the fastest single-core CPU I could find without bankrupting myself. (the i5 4690k that with some squeezing can even be pushed past 4ghz, it is a beastly CPU this one)
      - 5y ago
  - epmaybe 5y ago
    Haha I’m in almost the exact same boat, I replaced a haswell i5 with a 5800x. I definitely went overkill with my system and still haven’t gotten a GPU upgrade yet due to cost/laziness.
    - Causality1 5y ago
      Indeed. I feel like I could've gotten most of the speedup by wiping windows and reinstalling but I had to make a clean break from Windows 7 or I was never going to stop using it. Hopefully GPU prices come down soon.
  - deviledeggs 5y ago
    Even crazier, my i5 750 (from 2009) was overclocked to 3.5ghz and per core Ryzen 3 isn't even twice as fast. We're near the end of IPC scaling per core. And it was never that good in the first place. Pentium 3 IPC is only 3-4x worse than the fastest Ryzen. Most of our speed increases came from frequency. IMO we need to get off silicon substrate so we can frequency scale again. I wonder if the end of scaling will push everyone into faster languages like Rust. You can't sit around 2 years for your code performance to double anymore. Will this eventually kill slow languages? I think so.
    - michaelmrose 5y ago
      Hardware increases of speed began to slow down 20 years ago and people are still using software that is 50x slower than the fastest possible technology. If this alone was going to kill slow languages one would suppose it would have already done so. The arguable shift that is more significant is not about hardware its hopefully about newer languages like Rust making this performance cost less in terms of safety and development time which is a more recent development.
      - deviledeggs 5y ago
        I agree partially. But excuses for using dog slow languages, at least where I've worked, revolved around "it will be twice as fast in two years anyways" It's clear that's no longer true, which gives less excuses for using say Ruby over Java.
gameswithgo 5y ago
instructions per clock have been going up, l3 cache sizes going up, core count going up, new wider instructions becoming available (avx 2 and 512 ), branch predictors improving. overall speed increases are still nothing like the old days and spectre style mitigations have been eating away at the improvements
- bahmboo 5y ago
  Yes. Without a L1 cache a cpu is just a hyper active kid in a box. L2 important too. If one cares about performance learn about caches.
- deleted 5y ago
  [deleted]
techrat 5y ago
IPCs. Instructions per clock. We haven't had a strict clock speed = performance ratio for over a decade, it's just one component of it now. https://en.wikipedia.org/wiki/Cycles_per_instruction https://en.wikipedia.org/wiki/Cycles_per_instruction https://en.wikipedia.org/wiki/Instructions_per_second https://en.wikipedia.org/wiki/Instructions_per_second Rather than just bumping clock speed, they've made improvements on what can be done within the clock cycle. I recently did a desktop rebuild. Went from a Ryzen 7 2700X to a Ryzen 5 5600X. On paper, this looks like a downgrade. After all, the 7 is higher than a 5, right? I have two more cores with a 2700X... However, the 5600x has about a 28% IPC gain over the 2700x in single core performance despite running at the same base clock speed. Literally 20%+ faster in the same tasks even when taking the turbo boost out of the equation. The 2700x was at 12/14nm process while the 5600x is at 7nm which also helps with the power consumption as the 5600x has a 40 watt lower TDP. Since what I need is better single core performance over more cores, the 5600X is quite an upgrade despite only being two years newer. (A LOT has happened in 2 years with AMD) With two less cores but significantly higher single core performance, the 5600X outperforms the 2700X on both single core and multicore performance. Unfortunately for Intel, they did a lot of little tweaks and cheats to gain performance at the expense of security and now the mitigation patches pulls their chips back to the Bulldozer era in terms of current performance. They were also stuck at the same node for almost a decade now (Broadwell, 2014), so they had no gains from a shrink (speed of light propagation gain from a shrink, less heat and higher clocks also). My i7-4790k is full of jank and microstuttering now, it has become unusable. Imagine how the servers might be doing if (and they should be) they are being kept up to date on security and microcode patches. Then there also comes with spec upgrades within the processor generations. Ryzen 2700X to 5600X also means PCI Express went from 3.0 to 4.0... Not hugely significant among desktop users, but substantial for servers that need that amount of link bandwidth for compute cards and storage. TLDR: Magic.
elif 5y ago
Less heat = more cores
slver 5y ago
Clockrate is not a bottleneck on how fast your computer is. It's just a synchronization primitive. Think about it like the tempo of a song. The entire orchestra needs to play in sync with the tempo, but how many notes you play relative to the tempo is still up to each player. You can play multiple notes per "tempo tick".
- amelius 5y ago
  You will have to use a faster internal clock to play those faster notes, though.
  - can_count 5y ago
    The point is that you don't. A clock tick is not the smallest unit of operation. It's the smallest unit of synchronization. A lot of work can be done in-between synchronization points.
ksec 5y ago
A lot of comment mentioned about IPC, but something obvious no one has mentioned ( yet ). For Server is it also about core count. For the same TDP AMD offer 64 Core option. On a Dual Socket System that is 128 Core. Zen 3 Milan is also socket compatible with Zen 2 Rome. Basically for Server Intel has been stuck on 14nm for far too long. The first 14nm Broadwell Xeon was released in 2015, and as of mid 2021 Intel barely started rolling out 10nm Xeon Part based on IceLake. That is half a decade of stagnation. But we are now finally getting 7nm server part, 5nm with Zen 4 and and SRAM Die Stacking. I am only hoping EUV + DDR5 will bring Server ECC DRAM price down as well. In a few years time we will have affordable ( relatively speaking ) Dual Socket 256 Core Server with Terabyte of RAM. Get a few of those and be done with scaling for 90% of us. ( I mean the whole StackOverflow is Served with 9 Web Server [1] on some not very powerful hardware [2] ) [1] https://stackexchange.com/performance https://stackexchange.com/performance [2] https://nickcraver.com/blog/2016/03/29/stack-overflow-the-hardware-2016-edition/ https://nickcraver.com/blog/2016/03/29/stack-overflow-the-ha...
mbfg 5y ago
i think for the data center the key number is performance per watt. With smaller the power requirements go down. As the power goes down, heat goes down, so you can push the processors harder. There are of course architectural improvements that improve performance with space to add gates etc, but when you have 1000s of chips cost of energy is the big thing.

ben-schaaf 5y ago

Clocks aren't increasing much, but IPC still is. Density is also still increasing meaning lower latencies, lower power and therefore higher performance per area.

Causality1 5y ago
IPC is going up but slowly compared to previous generations. If AMD can sustain the kind of generation on generation increases it achieved between Zen 2 and 3 I will be tremendously impressed, but as it stands my brand new 5600X has barely 50% more single-core performance than the seven year old 4670K I replaced.
- flumpcakes 5y ago
  I have a hunch that it would be much greater than 50% if the software was recompiled to take advantage of the newer instruction sets. Then again, you won't be comparing exact binary copies of software, which some people might say invalidates it as a benchmark/comparison. I remember many, many years ago my PC couldn't run the latest adobe software as I was missing SSE2 instructions on my then current CPU. I believe there was some hidden compatability mode, but the software ran extremely poorly vs. the version of software that was out before that didn't require SSE2.
  - aseipp 5y ago
    There have been almost no new broad "performance-oriented" instruction sets introduced since the Haswell era, all the way to the Zen 3 era (e.g. SIMD/vector) for compilers to target. At least not any instructions that the vast majority of software is going to see magical huge improvements from and are wide spread enough to justify it (though specific software may benefit greatly from some specific tuning; pext/pdep and AVX-512 for instance.) The microarchitectures have improved significantly though, which does matter. For instance, Haswell-era AVX2 implementations were significantly poorer than the modern ones in, say, Tiger Lake or Zen 3. The newer ones have completely different power usage and per-core performance characteristics for AVX code; even if you could run AVX2 on older processors, it might not have actually been a good idea if the cumulative slowdowns they cause impacts the whole system (because the chips had to downclock the whole system so they wouldn't brownout). So it's not just a matter of instruction sets, but their individual performance characteristics. And also, it is not just CPUs that have improved. If anything, the biggest improvements have been in storage devices across the stack, which now have significantly better performance and parallelism, and the bandwidth has improved too (many more PCIe lanes). I can read gigabytes a second from a single NVMe drive; millions of IOPS a second, which is vastly better than you could 7 years ago on a consumer-level budget. Modern machines do not just crunch scalar code in isolation, and neither did older ones; we could just arbitrage CPU cycles more often than we can now in an era where a lot of the performance "cliffs" have been dealt with. Isolating things to just look at how fast the CPU can retire instructions is a good metric for CPU designers, but it's a very incomplete view when viewing the system as a whole as an application developer.
    - AtlasBarfed 5y ago
      I'm not deep on the weeds of high-performance compilers, but just because ISA evolution hasn't happened doesn't mean compilers can't evolve to use the silicon better. There always was an "Intel advantage" to compilers for decades (admittedly Intel invested in compilers more than AMD, but they also were sneaky about trying to nerf AMD in compilers), but with AMD being such a clear leader for so many years, I would hope at least GCC has started supporting AMD flavors of compilation better. Anyone know if this has happened with GCC and AMD silicon? Or at least is there a better body of knowledge of what GCC flags help AMD more?
      - scns 5y ago
        march=native
      - aseipp 5y ago
        Yes, I consider this to fall under the umbrella of general microarchitectural improvements I mentioned. GCC and LLVM are regularly updated with microarchitectural scheduling models to better emit code that matches the underlying architecture, and have featured these for at least 5-7 years; there can be a big difference between say, Skylake and Zen 2, for instance, so targeting things appropriately is a good idea. You can use the `-march` flag for your compiler to target specific architectures, for instance -march=tigerlake or -march=znver3 But in general I think it's a bit of a red herring for the thrust of my original post; first off you always have to target the benchmark to test a hypothesis, you don't run them in isolation for zero reason. My hypothesis when I ran my own for instance was "General execution of bog standard scalar code is only up by about 50-60%" and using the exact same binary instructions was the baseline criteria for that; it was not "Does targeting a specific microarchitecture scheduling model yield specific gains." If you want to test the second one, you need to run another benchmark. There are too many factors for any particular machine for any such post to be comprehensive, as I'm sure you're aware. I'm just speaking in loose generalities.
        jmgao 5y ago
        > You can use the `-march` flag for your compiler to target specific architectures, for instance -march=tigerlake or -march=znver3 Note that -march will use instructions that might be unavailable on other CPUs of the target. -mtune (which is implied by -march) is the flag that sets the cost tables used by instruction selection, cache line sizes, etc.
- jbluepolarbear 5y ago
  I doubt that very much. I have a i7 4790 and I have Ryzen 7 3700X. In my testing single core speed is nearly 2.5 times in favor of my Ryzen. What where you using as bench marks? My test was a single threaded software rasterizer with real-time vertex lighting. That I compiled in vs c++ 2008 around 2010.
  - aseipp 5y ago
    Basic single-core scalar-only workloads of my own corroborate the grandparent, as well as most of the other benchmarks I've seen. My own 5600X is "only" about 50-60% better than my old Haswell i5-4950 (from Q2 '14) on this note. But the scalar speed isn't everything, because you're often not bounded solely by retirement in isolation (the system, in aggregate, is an open one not a closed one.) Fatter caches and extraordinarily improved storage devices with lots of parallelism (even on a single core you can fire off a ton of asynchronous I/O at the device) make a huge difference here even for single-core workloads, because you can actually keep the core fed well enough to do work. So the cumulative improvement is even better in practice.
    - jbluepolarbear 5y ago
      Now I’m curious, is this testing against some requirements for an application? Are they any applications that would benefit solely from scalar performance vs scalar, cache size, and memory speed.
  - speeder 5y ago
    Need to point out in particular that that best Haswell i5 was FASTER than the best i7 for single-core workloads, so this might be a factor in OP post. And also that is the reason why I use such processor on my own computer, it was already "outdated" when I bought it, but since one of the things I like to do is play some simulation style games that rely heavily on single-core performance, I choose the fastest single-core CPU I could find without bankrupting myself. (the i5 4690k that with some squeezing can even be pushed past 4ghz, it is a beastly CPU this one)
    - 5y ago
- epmaybe 5y ago
  Haha I’m in almost the exact same boat, I replaced a haswell i5 with a 5800x. I definitely went overkill with my system and still haven’t gotten a GPU upgrade yet due to cost/laziness.
  - Causality1 5y ago
    Indeed. I feel like I could've gotten most of the speedup by wiping windows and reinstalling but I had to make a clean break from Windows 7 or I was never going to stop using it. Hopefully GPU prices come down soon.
- deviledeggs 5y ago
  Even crazier, my i5 750 (from 2009) was overclocked to 3.5ghz and per core Ryzen 3 isn't even twice as fast. We're near the end of IPC scaling per core. And it was never that good in the first place. Pentium 3 IPC is only 3-4x worse than the fastest Ryzen. Most of our speed increases came from frequency. IMO we need to get off silicon substrate so we can frequency scale again. I wonder if the end of scaling will push everyone into faster languages like Rust. You can't sit around 2 years for your code performance to double anymore. Will this eventually kill slow languages? I think so.
  - michaelmrose 5y ago
    Hardware increases of speed began to slow down 20 years ago and people are still using software that is 50x slower than the fastest possible technology. If this alone was going to kill slow languages one would suppose it would have already done so. The arguable shift that is more significant is not about hardware its hopefully about newer languages like Rust making this performance cost less in terms of safety and development time which is a more recent development.
    - deviledeggs 5y ago
      I agree partially. But excuses for using dog slow languages, at least where I've worked, revolved around "it will be twice as fast in two years anyways" It's clear that's no longer true, which gives less excuses for using say Ruby over Java.

gameswithgo 5y ago

instructions per clock have been going up, l3 cache sizes going up, core count going up, new wider instructions becoming available (avx 2 and 512 ), branch predictors improving. overall speed increases are still nothing like the old days and spectre style mitigations have been eating away at the improvements

bahmboo 5y ago
Yes. Without a L1 cache a cpu is just a hyper active kid in a box. L2 important too. If one cares about performance learn about caches.
deleted 5y ago
[deleted]

techrat 5y ago

IPCs. Instructions per clock. We haven't had a strict clock speed = performance ratio for over a decade, it's just one component of it now. https://en.wikipedia.org/wiki/Cycles_per_instruction https://en.wikipedia.org/wiki/Cycles_per_instruction https://en.wikipedia.org/wiki/Instructions_per_second https://en.wikipedia.org/wiki/Instructions_per_second Rather than just bumping clock speed, they've made improvements on what can be done within the clock cycle. I recently did a desktop rebuild. Went from a Ryzen 7 2700X to a Ryzen 5 5600X. On paper, this looks like a downgrade. After all, the 7 is higher than a 5, right? I have two more cores with a 2700X... However, the 5600x has about a 28% IPC gain over the 2700x in single core performance despite running at the same base clock speed. Literally 20%+ faster in the same tasks even when taking the turbo boost out of the equation. The 2700x was at 12/14nm process while the 5600x is at 7nm which also helps with the power consumption as the 5600x has a 40 watt lower TDP. Since what I need is better single core performance over more cores, the 5600X is quite an upgrade despite only being two years newer. (A LOT has happened in 2 years with AMD) With two less cores but significantly higher single core performance, the 5600X outperforms the 2700X on both single core and multicore performance. Unfortunately for Intel, they did a lot of little tweaks and cheats to gain performance at the expense of security and now the mitigation patches pulls their chips back to the Bulldozer era in terms of current performance. They were also stuck at the same node for almost a decade now (Broadwell, 2014), so they had no gains from a shrink (speed of light propagation gain from a shrink, less heat and higher clocks also). My i7-4790k is full of jank and microstuttering now, it has become unusable. Imagine how the servers might be doing if (and they should be) they are being kept up to date on security and microcode patches. Then there also comes with spec upgrades within the processor generations. Ryzen 2700X to 5600X also means PCI Express went from 3.0 to 4.0... Not hugely significant among desktop users, but substantial for servers that need that amount of link bandwidth for compute cards and storage. TLDR: Magic.

elif 5y ago

Less heat = more cores

slver 5y ago

Clockrate is not a bottleneck on how fast your computer is. It's just a synchronization primitive. Think about it like the tempo of a song. The entire orchestra needs to play in sync with the tempo, but how many notes you play relative to the tempo is still up to each player. You can play multiple notes per "tempo tick".

amelius 5y ago
You will have to use a faster internal clock to play those faster notes, though.
- can_count 5y ago
  The point is that you don't. A clock tick is not the smallest unit of operation. It's the smallest unit of synchronization. A lot of work can be done in-between synchronization points.

ksec 5y ago

A lot of comment mentioned about IPC, but something obvious no one has mentioned ( yet ). For Server is it also about core count. For the same TDP AMD offer 64 Core option. On a Dual Socket System that is 128 Core. Zen 3 Milan is also socket compatible with Zen 2 Rome. Basically for Server Intel has been stuck on 14nm for far too long. The first 14nm Broadwell Xeon was released in 2015, and as of mid 2021 Intel barely started rolling out 10nm Xeon Part based on IceLake. That is half a decade of stagnation. But we are now finally getting 7nm server part, 5nm with Zen 4 and and SRAM Die Stacking. I am only hoping EUV + DDR5 will bring Server ECC DRAM price down as well. In a few years time we will have affordable ( relatively speaking ) Dual Socket 256 Core Server with Terabyte of RAM. Get a few of those and be done with scaling for 90% of us. ( I mean the whole StackOverflow is Served with 9 Web Server [1] on some not very powerful hardware [2] ) [1] https://stackexchange.com/performance https://stackexchange.com/performance [2] https://nickcraver.com/blog/2016/03/29/stack-overflow-the-hardware-2016-edition/ https://nickcraver.com/blog/2016/03/29/stack-overflow-the-ha...

mbfg 5y ago

i think for the data center the key number is performance per watt. With smaller the power requirements go down. As the power goes down, heat goes down, so you can push the processors harder. There are of course architectural improvements that improve performance with space to add gates etc, but when you have 1000s of chips cost of energy is the big thing.