5 ms·

Does MHz Still Matter?

66 points by furkansahin 1y ago

magicalhippo 1y ago
Well from what I can see the faster one also has 3D V-cache, so not apples to apples. That said, at such low core count the primary Epyc advantage is PCIe lanes no?
- furkansahin 1y ago
  Yes, you're right but we tried to keep the workloads less cache dependent. Also, EPYC's PCIe advantage doesn't hold for the Hetzner provided server setup unfortunately because the configurator allows the same number of devices to be attached to both servers.
adamcharnock 1y ago
This is certainly interesting, and is something I often wonder about. In our case we mostly run Kubernetes clusters on EX130 servers (Xeon, 24c, 256GB). In this situation there are a lot of processes running, for which increased cores-count and memory availability seems worth it. Particularly when we have the cost of 10-25g private networking for each server, lower node counts seems to come out being more economical. But, but with fewer processes I can totally believe this works out to be the better option. Thank you for the write-up!
- furkansahin 1y ago
  Thanks for the comment! Yeah, if you are using the servers dedicated to yourself and considering the larger server packs more, it definitely makes sense. In our case, though, if we provide 1/48 of 10Gbit network, it really doesn't work for our end customers. So, we're trying to provide the VMs from smaller but more up-to-date lineup.
- justsomehnguy 1y ago
  > In this situation there are a lot of processes running, for which increased cores-count and memory availability seems worth it. It's always the workload type. For the mixed environments (some with a heavy constant load while the other have only some occasional spikes) the increase of RAM per node is the most important part what allowed us to actually decrease the node count. The whole racks with multiple switches was replaced by a single rack with a modest amount of servers and a single stacked switch.
bob1029 1y ago
> Once network IO became the main bottleneck, the faster CPU mattered less. It is surprisingly hard to keep a modern CPU core properly saturated unless you are baking global illumination, searching primes or mining crypto currency. I/O and latency will almost always dominate at scale. Moving information is way more expensive than processing it.
- furkansahin 1y ago
  Certainly! That was the main reason why we decided to try postgres benchmarking tbh
- cma 1y ago
  Some major game engines still have large single thread bottlenecks. They aren't fundamental to the problem space though, more just from legacy engine design decisions.
  - SkiFire13 1y ago
    Being single thread bottlenecked doesn't mean they are actually saturating a CPU core, it may likely be waiting for data from RAM for a lot if not most of the time.
    - NoMoreNicksLeft 1y ago
      The solution is simple. A bog-standard RISC instruction set, with 64 billion 256-bit registers. The operating system will come on the die, etched into the silicon. There will be no ram and no storage.
    - cma 1y ago
      If they had it multithreaded, the execution ports could be used on the other threads while one thread is affected by RAM latency. And if there is a pointer chasey task that isn't utilizing all of the core and is waiting on memory and data dependencies, often that just makes threading a bigger win, 16 threads doing pointer chases instead of one if you aren't memory bandwidth bound.
- 1y ago
Aurornis 1y ago
AMD’s fastest consumer CPUs are a great value for small servers. If you’re doing just one task (like in this article) the clock speed is a huge benefit. The larger server grade parts start to shine when the server is doing a lot of different things. The extra memory bandwidth helps keep the CPU fed and the higher core count reduces the need for context switching because your workloads aren’t competing as much. The best part about the AMD consumer CPUs is that you can even use ECC RAM if you get the right motherboard.
- AnimalMuppet 1y ago
  Wait, what? ECC RAM for a consumer CPU? Does anyone sell motherboards like that?
  - anonymars 1y ago
    For example https://www.asus.com/support/faq/1051605/ https://www.asus.com/support/faq/1051605/
  - dcm360 1y ago
    Asrock Rack and Supermico sell AM4/AM5 motherboards with support for ECC UDIMMs. Other vendors might state official support on workstation-class motherboards, and in general it might work even on motherboards without official support.
  - kllrnohj 1y ago
    Literally first Asrock motherboard I happened to click on has it listed as a feature: https://www.asrock.com/mb/AMD/X870%20Taichi%20Creator/index.asp#Specification https://www.asrock.com/mb/AMD/X870%20Taichi%20Creator/index.... Asus has options as well such as https://www.asus.com/motherboards-components/motherboards/proart/proart-x870e-creator-wifi/techspec/ https://www.asus.com/motherboards-components/motherboards/pr... I think it was more rare when AM5 first came out, there were a bunch of ECC supported consumer boards for AM4 and threadripper.
  - 0x457 1y ago
    AMDs I/O die supports ECC RAM, but up to motherboard vendor to run the required traces. Some do, some don't.
chicagojoe 1y ago
Consumer grade CPUs aren't meant to be pushed with heavy load 24/7, meaning, durability becomes another variable which, in my experience, will quickly outweigh the brief burst of speed.
- wmf 1y ago
  If your software can handle machine failures, 20% extra performance is absolutely worth some extra failures.
  - bob1029 1y ago
    I think this is the best path if your problem can support it. I use a 5950X for running genetic programming and neuroevolution experiments and about once every 100 hours the machine will just not like the state/load it is experiencing and will restart. My approach is to checkpoint as often as possible. I restart the program the next morning and it deserializes the last snapshot from disk. Worst case, I lose 5 minutes of work. This also helps with Windows updates, power outages, and EM/cosmic radiation.
- toast0 1y ago
  AMD uses the same chiplets for Epyc and Ryzen. The packaging is different, and the I/O dies are different, but whatever. If you really care, you can buy an Epyc branded AM4 or AM5 cpu which has remarkably similar specifications and MSRP to Ryzen offerings.
mrandish 1y ago
As a consumer who nursed an overclocked 1080ti along for 2.5 gens longer than I would've liked thanks to crypto and then AI, I was reading this fearing a positive conclusion - thinking "Oh great, just when it's time to upgrade my 5600x CPU data centers will start driving up already over-priced consumer CPUs too." Although said somewhat tongue in cheek, it has been a rough several years for tech hobbyist consumers. At least the end of Moore's law scaling and the bite of Dennard scaling combined to nerf generational improvements enough that getting by on existing hardware wasn't as nearly as bad as it would've been 20 yrs ago. Now that maybe the AI bubble is just starting to burst, we've got tariffs to ensure tech consumers still won't see undistorted prices. The silver lining in all this is that it got me into retro gaming and computing which, frankly, is really great.
bgnn 1y ago
If you are running engineering jobs (HPC) like electrical simulation for chip design the only two thingd you care about are the CPU clock speed and memory RW speed. It's unfortunate that we can only have 16 core CPUs running at 5+ GHz. I would have loved to have a 32 or 64 core Ryzen 9. The software we use charge per core used, so 30% less performance is that much extra cost, which is easily an order of magnitude higher than a flagship server CPU. These licenses cost millions per year for couple 16 core seats. So, at the end, CPU speed is determining how fast abd economically chips are developed.
- CBLT 1y ago
  I'm in disbelief that the software you run is completely insensitive to IPC and instruction latency. Without those number, clock speed is meaningless.
  - bgnn 1y ago
    I don't have those numbers, but here is AMD's on results on Cadence Spectre X with different gen EPYC CPUs, maybe this helps for you to understand the issue: https://www.amd.com/content/dam/amd/en/documents/epyc-technical-docs/performance-briefs/amd-epyc-9004x-pb-cadence-spectre-x.pdf https://www.amd.com/content/dam/amd/en/documents/epyc-techni... About the technical problem: it's ODE solution for a sparse matrix of 10s of millions elements. Sparsity comes from the locality of interactions within a chip: not every transistor is connected to thd others. So the modern simulators make use of this sparsity to divide the circuit to independent chunks and spread over to multiple independent threads. Scale of the compute time is not objective, but typically anywhere from couple hours to couple of months. Most top level chip integration verification jobs take weeks. Because of this we spend months for verification after the design is pretty much finish before the tapeout. This applies for every single reasonably complex chip.
- ytreem 1y ago
  >These licenses cost millions per year for couple 16 core seats The ROI on hiring a professional overclocker to build, tune and test a workstation is probably at least break even. As long as the right checksums are in place, extreme OC is just a business writeoff.
theyinwhy 1y ago
Unfortunately, the page's numbers are represented in a sloppy way. A benchmark number with a dollar sign. Different job counts. Lacking documentation. I wouldn't trust this data too much.
- ddalex 1y ago
  > A benchmark number with a dollar sign. Time is money. Or the inverse of money. Ufff, my head hurts.
ianpcook 1y ago
*Do
joennlae 1y ago
How can I make sure that each github runner uses exactly one cpu core?
ksec 1y ago
From a cloud perspective I guess that would make sense. But if you are actually owning the hardware you would be looking at Performance per Watt over single and multiple core and its balance across I/O both in Sequel and Random. Because at the end of the day you are still limited by power budget. And single core boost are not sustainable over a long period of time especially in a multiple core CPU scenario. On that note I cant wait to see 256 Core Zen6c later this year. We will soon be able to buy a server with 512 Core, 1024 vCPU / Thread. 2TB of Memory, x TB of SSD all inside 1U.

66 points by furkansahin 1y ago

Well from what I can see the faster one also has 3D V-cache, so not apples to apples. That said, at such low core count the primary Epyc advantage is PCIe lanes no?

furkansahin 1y ago
Yes, you're right but we tried to keep the workloads less cache dependent. Also, EPYC's PCIe advantage doesn't hold for the Hetzner provided server setup unfortunately because the configurator allows the same number of devices to be attached to both servers.

adamcharnock 1y ago

This is certainly interesting, and is something I often wonder about. In our case we mostly run Kubernetes clusters on EX130 servers (Xeon, 24c, 256GB). In this situation there are a lot of processes running, for which increased cores-count and memory availability seems worth it. Particularly when we have the cost of 10-25g private networking for each server, lower node counts seems to come out being more economical. But, but with fewer processes I can totally believe this works out to be the better option. Thank you for the write-up!

furkansahin 1y ago
Thanks for the comment! Yeah, if you are using the servers dedicated to yourself and considering the larger server packs more, it definitely makes sense. In our case, though, if we provide 1/48 of 10Gbit network, it really doesn't work for our end customers. So, we're trying to provide the VMs from smaller but more up-to-date lineup.
justsomehnguy 1y ago
> In this situation there are a lot of processes running, for which increased cores-count and memory availability seems worth it. It's always the workload type. For the mixed environments (some with a heavy constant load while the other have only some occasional spikes) the increase of RAM per node is the most important part what allowed us to actually decrease the node count. The whole racks with multiple switches was replaced by a single rack with a modest amount of servers and a single stacked switch.

bob1029 1y ago

> Once network IO became the main bottleneck, the faster CPU mattered less. It is surprisingly hard to keep a modern CPU core properly saturated unless you are baking global illumination, searching primes or mining crypto currency. I/O and latency will almost always dominate at scale. Moving information is way more expensive than processing it.

furkansahin 1y ago
Certainly! That was the main reason why we decided to try postgres benchmarking tbh
cma 1y ago
Some major game engines still have large single thread bottlenecks. They aren't fundamental to the problem space though, more just from legacy engine design decisions.
- SkiFire13 1y ago
  Being single thread bottlenecked doesn't mean they are actually saturating a CPU core, it may likely be waiting for data from RAM for a lot if not most of the time.
  - NoMoreNicksLeft 1y ago
    The solution is simple. A bog-standard RISC instruction set, with 64 billion 256-bit registers. The operating system will come on the die, etched into the silicon. There will be no ram and no storage.
  - cma 1y ago
    If they had it multithreaded, the execution ports could be used on the other threads while one thread is affected by RAM latency. And if there is a pointer chasey task that isn't utilizing all of the core and is waiting on memory and data dependencies, often that just makes threading a bigger win, 16 threads doing pointer chases instead of one if you aren't memory bandwidth bound.
1y ago

Aurornis 1y ago

AMD’s fastest consumer CPUs are a great value for small servers. If you’re doing just one task (like in this article) the clock speed is a huge benefit. The larger server grade parts start to shine when the server is doing a lot of different things. The extra memory bandwidth helps keep the CPU fed and the higher core count reduces the need for context switching because your workloads aren’t competing as much. The best part about the AMD consumer CPUs is that you can even use ECC RAM if you get the right motherboard.

AnimalMuppet 1y ago
Wait, what? ECC RAM for a consumer CPU? Does anyone sell motherboards like that?
- anonymars 1y ago
  For example https://www.asus.com/support/faq/1051605/ https://www.asus.com/support/faq/1051605/
- dcm360 1y ago
  Asrock Rack and Supermico sell AM4/AM5 motherboards with support for ECC UDIMMs. Other vendors might state official support on workstation-class motherboards, and in general it might work even on motherboards without official support.
- kllrnohj 1y ago
  Literally first Asrock motherboard I happened to click on has it listed as a feature: https://www.asrock.com/mb/AMD/X870%20Taichi%20Creator/index.asp#Specification https://www.asrock.com/mb/AMD/X870%20Taichi%20Creator/index.... Asus has options as well such as https://www.asus.com/motherboards-components/motherboards/proart/proart-x870e-creator-wifi/techspec/ https://www.asus.com/motherboards-components/motherboards/pr... I think it was more rare when AM5 first came out, there were a bunch of ECC supported consumer boards for AM4 and threadripper.
- 0x457 1y ago
  AMDs I/O die supports ECC RAM, but up to motherboard vendor to run the required traces. Some do, some don't.

chicagojoe 1y ago

Consumer grade CPUs aren't meant to be pushed with heavy load 24/7, meaning, durability becomes another variable which, in my experience, will quickly outweigh the brief burst of speed.

wmf 1y ago
If your software can handle machine failures, 20% extra performance is absolutely worth some extra failures.
- bob1029 1y ago
  I think this is the best path if your problem can support it. I use a 5950X for running genetic programming and neuroevolution experiments and about once every 100 hours the machine will just not like the state/load it is experiencing and will restart. My approach is to checkpoint as often as possible. I restart the program the next morning and it deserializes the last snapshot from disk. Worst case, I lose 5 minutes of work. This also helps with Windows updates, power outages, and EM/cosmic radiation.
toast0 1y ago
AMD uses the same chiplets for Epyc and Ryzen. The packaging is different, and the I/O dies are different, but whatever. If you really care, you can buy an Epyc branded AM4 or AM5 cpu which has remarkably similar specifications and MSRP to Ryzen offerings.

mrandish 1y ago

As a consumer who nursed an overclocked 1080ti along for 2.5 gens longer than I would've liked thanks to crypto and then AI, I was reading this fearing a positive conclusion - thinking "Oh great, just when it's time to upgrade my 5600x CPU data centers will start driving up already over-priced consumer CPUs too." Although said somewhat tongue in cheek, it has been a rough several years for tech hobbyist consumers. At least the end of Moore's law scaling and the bite of Dennard scaling combined to nerf generational improvements enough that getting by on existing hardware wasn't as nearly as bad as it would've been 20 yrs ago. Now that maybe the AI bubble is just starting to burst, we've got tariffs to ensure tech consumers still won't see undistorted prices. The silver lining in all this is that it got me into retro gaming and computing which, frankly, is really great.

bgnn 1y ago

If you are running engineering jobs (HPC) like electrical simulation for chip design the only two thingd you care about are the CPU clock speed and memory RW speed. It's unfortunate that we can only have 16 core CPUs running at 5+ GHz. I would have loved to have a 32 or 64 core Ryzen 9. The software we use charge per core used, so 30% less performance is that much extra cost, which is easily an order of magnitude higher than a flagship server CPU. These licenses cost millions per year for couple 16 core seats. So, at the end, CPU speed is determining how fast abd economically chips are developed.

CBLT 1y ago
I'm in disbelief that the software you run is completely insensitive to IPC and instruction latency. Without those number, clock speed is meaningless.
- bgnn 1y ago
  I don't have those numbers, but here is AMD's on results on Cadence Spectre X with different gen EPYC CPUs, maybe this helps for you to understand the issue: https://www.amd.com/content/dam/amd/en/documents/epyc-technical-docs/performance-briefs/amd-epyc-9004x-pb-cadence-spectre-x.pdf https://www.amd.com/content/dam/amd/en/documents/epyc-techni... About the technical problem: it's ODE solution for a sparse matrix of 10s of millions elements. Sparsity comes from the locality of interactions within a chip: not every transistor is connected to thd others. So the modern simulators make use of this sparsity to divide the circuit to independent chunks and spread over to multiple independent threads. Scale of the compute time is not objective, but typically anywhere from couple hours to couple of months. Most top level chip integration verification jobs take weeks. Because of this we spend months for verification after the design is pretty much finish before the tapeout. This applies for every single reasonably complex chip.
ytreem 1y ago
>These licenses cost millions per year for couple 16 core seats The ROI on hiring a professional overclocker to build, tune and test a workstation is probably at least break even. As long as the right checksums are in place, extreme OC is just a business writeoff.

theyinwhy 1y ago

Unfortunately, the page's numbers are represented in a sloppy way. A benchmark number with a dollar sign. Different job counts. Lacking documentation. I wouldn't trust this data too much.

ddalex 1y ago
> A benchmark number with a dollar sign. Time is money. Or the inverse of money. Ufff, my head hurts.

ianpcook 1y ago

*Do

joennlae 1y ago

How can I make sure that each github runner uses exactly one cpu core?

ksec 1y ago

From a cloud perspective I guess that would make sense. But if you are actually owning the hardware you would be looking at Performance per Watt over single and multiple core and its balance across I/O both in Sequel and Random. Because at the end of the day you are still limited by power budget. And single core boost are not sustainable over a long period of time especially in a multiple core CPU scenario. On that note I cant wait to see 256 Core Zen6c later this year. We will soon be able to buy a server with 512 Core, 1024 vCPU / Thread. 2TB of Memory, x TB of SSD all inside 1U.