3 ms·

Sounds like there is something else going on there, since the Xeon CPU idle power consumption is nowhere near 200W [0] [0] https://www.anandtech.com/show/16594

Sounds like there is something else going on there, since the Xeon CPU idle power consumption is nowhere near 200W [0] [0] https://www.anandtech.com/show/16594/intel-3rd-gen-xeon-scalable-review/5 https://www.anandtech.com/show/16594/intel-3rd-gen-xeon-scal...

helf 4y ago
Right? That is why I want to know more details instead of this vague garbage.
helf 4y ago
Not to mention the fact that "pushing up to 700W underload" is.. absolutely nothing since that means, ignoring everything else and just assuming CPU drawing that, the cores are only using 7w PER CORE. And this AMD system is using apparently a peak of 3.75w per core? really?
- esperent 4y ago
  This comment made me realize I've never seen power per core specified. That seems like a great metric. Would calculating this be as simple as dividing peak usage by the number of cores? (Ignoring that modern CPUs have multiple core types)
  - toast0 4y ago
    Power per core is pretty variable though. If you've got a lightly threaded workload, you shouldn't be getting 40-52 cores per socket, but many of the cores could be at near 0w and some of the cores will use more power than they would on an all cores workload.
  - helf 4y ago
    It wouldn’t really. I only used that because the parent poster was making it sound like total power draws with the cores all pegged out. And if they are actually doing work and not just running full bore through a whole lot of NOPs then that would be a sorta average of power on each core. But as the other commenter said, the actual per core usage will vary /wildly/ even under high load. You would half to run something fairly specialized such as prime95, which can run entirely from cache, to get more accurate per core wattage values under same load maximum load. As it stands, my i9-13900K seems to have an idle “package” draw of around 10-12w and can peak at 360w under prime95 load. And that’s stock. I can make it ramp way past 360W with very little effort. But under “normal” desktop usage it happily sits well under 100w. So… shrugs. I would like to know more on this parent’s setup instead of “My server with literally 40% of the core count uses 1/6 of the power!” But we may not be so lucky.
    - aetherspawn 4y ago
      The load case is not 1:1 and the experience is only anecdotal, but I put a few more details about the load in another comment below. My commentary is only really about what we believed about AMD before making our purchase (internet forums like this one warn about energy use), how that wasn't really the case when we measured it on a real load, and why we 'feel' we would prefer to go the smaller and cheaper AMD server route next time.
      - deleted 4y ago
        [deleted]
paulmd 4y ago
You have to remember that it's not just the CPU but also the IO (memory and pcie) and the chipset... collectively it's often called "platform power". Even if you don't have anything plugged into it, a big powerful chipset and lots of on-chip PCIe expansion still eats a decent amount of power. Also everything you plug into it will eat power, some 10gbe stuff and SAS controllers/etc are not lightweights in idle because it's not expected they'll be idle. Every HDD generally pulls about 5W when idle+spinning, even SSDs are a couple watts apiece. That said, yeah, that sounds way high for idle. ServeTheHome is a good place for stuff like this: > motherboard: X11SPM-F > memory: 6x 8GB RDIMM 2666Mt > kill-a-watt power on the wall at idle: 30..35Watts. https://forums.servethehome.com/index.php?threads/xeon-scalable-vs-epyc-idle-power-consumption.39037/ https://forums.servethehome.com/index.php?threads/xeon-scala... Single-socket though, but, another CPU probably doesn't add more than another 30 watts or so. Epyc actually tends to be worse on idle power because it has to run all its infinity fabric links at all times, if it wants to access that memory, and data movement is comparatively expensive. There were some early Milan/Zen3 epyc numbers that need to be disregarded due to problems with AMD's reference implementation motherboard, but, you can see from that thread it's not uncommon for Epyc to idle at 100W for a single-socket board. You can see from that thread that the consensus is Skylake-SP generally idles better. My own personal anecdote: I am tinkering with a dual-socket 2011-3 server right now (X10DRC-T4+) with a pair of 2697v3 and 2x4x32GB RDIMMs and quad 10gbase-t NICs, at idle I am around 75W at the wall... obviously Skylake-SP has a larger platform (it's the socket after mine) but it's not 350W bigger, guy has something wrong with his setup. It is definitely fair though that under load the Epycs are a lot more energy-efficient. Skylake-SP is still 14nm and Intel lingered there for far too long, 7nm is more than a full node better and Milan has higher IPC, higher clocks, and more cores. Ice Lake-SP was better than Skylake, but nobody bought it because it was years late to market and AMD was already better. So you are effectively comparing 2016-ish Intel stuff against AMD stuff that is 4+ years newer... but on the other hand that's the Intel stuff that is deployed right now, and people generally want to minimize the amount of generations in their clusters to make it easier to operate VMWare vMotion etc. /shruggy

helf 4y ago

Right? That is why I want to know more details instead of this vague garbage.

helf 4y ago

Not to mention the fact that "pushing up to 700W underload" is.. absolutely nothing since that means, ignoring everything else and just assuming CPU drawing that, the cores are only using 7w PER CORE. And this AMD system is using apparently a peak of 3.75w per core? really?

esperent 4y ago
This comment made me realize I've never seen power per core specified. That seems like a great metric. Would calculating this be as simple as dividing peak usage by the number of cores? (Ignoring that modern CPUs have multiple core types)
- toast0 4y ago
  Power per core is pretty variable though. If you've got a lightly threaded workload, you shouldn't be getting 40-52 cores per socket, but many of the cores could be at near 0w and some of the cores will use more power than they would on an all cores workload.
- helf 4y ago
  It wouldn’t really. I only used that because the parent poster was making it sound like total power draws with the cores all pegged out. And if they are actually doing work and not just running full bore through a whole lot of NOPs then that would be a sorta average of power on each core. But as the other commenter said, the actual per core usage will vary /wildly/ even under high load. You would half to run something fairly specialized such as prime95, which can run entirely from cache, to get more accurate per core wattage values under same load maximum load. As it stands, my i9-13900K seems to have an idle “package” draw of around 10-12w and can peak at 360w under prime95 load. And that’s stock. I can make it ramp way past 360W with very little effort. But under “normal” desktop usage it happily sits well under 100w. So… shrugs. I would like to know more on this parent’s setup instead of “My server with literally 40% of the core count uses 1/6 of the power!” But we may not be so lucky.
  - aetherspawn 4y ago
    The load case is not 1:1 and the experience is only anecdotal, but I put a few more details about the load in another comment below. My commentary is only really about what we believed about AMD before making our purchase (internet forums like this one warn about energy use), how that wasn't really the case when we measured it on a real load, and why we 'feel' we would prefer to go the smaller and cheaper AMD server route next time.
    - deleted 4y ago
      [deleted]

paulmd 4y ago

You have to remember that it's not just the CPU but also the IO (memory and pcie) and the chipset... collectively it's often called "platform power". Even if you don't have anything plugged into it, a big powerful chipset and lots of on-chip PCIe expansion still eats a decent amount of power. Also everything you plug into it will eat power, some 10gbe stuff and SAS controllers/etc are not lightweights in idle because it's not expected they'll be idle. Every HDD generally pulls about 5W when idle+spinning, even SSDs are a couple watts apiece. That said, yeah, that sounds way high for idle. ServeTheHome is a good place for stuff like this: > motherboard: X11SPM-F > memory: 6x 8GB RDIMM 2666Mt > kill-a-watt power on the wall at idle: 30..35Watts. https://forums.servethehome.com/index.php?threads/xeon-scalable-vs-epyc-idle-power-consumption.39037/ https://forums.servethehome.com/index.php?threads/xeon-scala... Single-socket though, but, another CPU probably doesn't add more than another 30 watts or so. Epyc actually tends to be worse on idle power because it has to run all its infinity fabric links at all times, if it wants to access that memory, and data movement is comparatively expensive. There were some early Milan/Zen3 epyc numbers that need to be disregarded due to problems with AMD's reference implementation motherboard, but, you can see from that thread it's not uncommon for Epyc to idle at 100W for a single-socket board. You can see from that thread that the consensus is Skylake-SP generally idles better. My own personal anecdote: I am tinkering with a dual-socket 2011-3 server right now (X10DRC-T4+) with a pair of 2697v3 and 2x4x32GB RDIMMs and quad 10gbase-t NICs, at idle I am around 75W at the wall... obviously Skylake-SP has a larger platform (it's the socket after mine) but it's not 350W bigger, guy has something wrong with his setup. It is definitely fair though that under load the Epycs are a lot more energy-efficient. Skylake-SP is still 14nm and Intel lingered there for far too long, 7nm is more than a full node better and Milan has higher IPC, higher clocks, and more cores. Ice Lake-SP was better than Skylake, but nobody bought it because it was years late to market and AMD was already better. So you are effectively comparing 2016-ish Intel stuff against AMD stuff that is 4+ years newer... but on the other hand that's the Intel stuff that is deployed right now, and people generally want to minimize the amount of generations in their clusters to make it easier to operate VMWare vMotion etc. /shruggy