6 ms·

Related and perhaps useful: I’ve seen this in multiple cloud offerings already, where the cpu scaling governor is set to some eco-friendly value, in benefit to

Related and perhaps useful: I’ve seen this in multiple cloud offerings already, where the cpu scaling governor is set to some eco-friendly value, in benefit to the cloud provider and in zero benefit to you and much reduced peak cpu perf. To check, run `cat /sys/devices/system/cpu/cpu/cpufreq/scaling_governor`. It should be `performance`. If it’s not, set it with `echo performance | sudo tee /sys/devices/system/cpu/cpu/cpufreq/scaling_governor`. If your workload is cpu hungry this will help. It will revert on startup, so can make it stick, with some cron/systemd or whichever. Of course if you are the one paying for power or it’s your own hardware, make your own judgement for the scaling governor. But if it’s a rented bare metal server, you do want `performance`.

chpatrick 2y ago
Is there any downside to ondemand? If your servers aren't running at 100% then there's no point wasting watts, even if you aren't paying for them, right?
- anarazel 2y ago
  It performs terrible if you have an intermittent workload. Like e.g. a request response workload where request processing is cheap (so that the time to increase the frequency matters). I've seen cases it's a more than 2x request latency increase. It can be pretty annoying, because it means that systems can perform better under higher load and that you get drastically different latency depending on whether a request is scheduled on a core that just processed another request (already at high freq) or one that was idle. And because the frequency control isn't fun enough, this behavior also exists with cpu idle states. Even at high frequency Linux can enter idle states... I've debugged several cases where this set of issues has caused unintuitive behavior. E.g. a) switching to a more powerful servers drastically increased latency b) optimized code resulting in higher latency / lower throughout because that provided enough idle cycles for a deeper idle time between requests c) slightly increased IO latency leading to significantly worse overall performance, due to the IO getting long though to clock down
  - lucb1e 2y ago
    Intermittent workloads on the order of 2 milliseconds, you mean? Frequency scaling is much faster than blinking, as well as ubiquitous in all recent consumer hardware that I'm aware of due to the huge amount of power it saves. Turning it off would, to me, only make sense if you want a server to handle thousands of fast requests per second, but those requests don't come in for periods of, say, 50 ms at a time and so the CPU scales back. Actually, thinking this through, even then it doesn't make much sense to me: if you have that many short requests coming in, the CPU would simply never scale back if it's reasonably constant. It would first need to see some gap, and why not scale the CPU back in that gap (at the cost of having the 1st request of the next batch be a few milliseconds slower)? From there on, every subsequent request is fast again until there's another lull. Keeping the CPU always on high frequency should only be needed if you have a very tight deadline on that surprise request (high-frequency trading perhaps?), or if your requests are coincidentally always spaced by the same amount of time as CPU scaling measures across. I'm sure these things exist but "intermittent workload" is 90% of all workloads and most workloads definitely aren't meaningfully impacted by cpu scaling
    - anarazel 2y ago
      > Intermittent workloads on the order of 2 milliseconds, you mean? Yea. Most of the cases I was looking at were with postgres, with fully cached simple queries. Each taking << 10ms. The problem is more extreme if the client takes some time to actually process the result or there is network latency, but even without it's rather noticeable. > Turning it off would, to me, only make sense if you want a server to handle thousands of fast requests per second, but those requests don't come in for periods of, say, 50 ms at a time and so the CPU scales back. I see regressions at periods well below 50ms, but yea, that's the shape of it. E.g. a postgres client running 1000 QPS over a single TCP connection from a different server, connected via switched 10Gbit Ethernet (ping RTT 0.030ms), has the following client side visible per-query latencies: powersave, idle enabled: 0.392 ms performance, idle enabled: 0.295 ms performance, idle disabled: 0.163 ms If I make that same 1 client go full tilt, instead of limiting it to 1000 QPS: powersave, idle enabled: 0.141 ms performance, idle enabled: 0.107 ms performance, idle disabled: 0.090 ms I'd call that a significant performance change. > if you have that many short requests coming in, the CPU would simply never scale back if it's reasonably constant. Indeed, that's what makes the whole issue so pernicious. One of the ways I saw this was when folks moved postgres to more powerful servers and got worse performance due to frequency/idle handling. The reason being that it made it more likely that cores were idle long enough to clock down. On the same setup as above, if I instead have 800 client connections going full tilt, there's no meaningful difference between powersave/performance and idle enabled/disabled.
Tijdreiziger 2y ago
However, eco-friendly power modes can reduce electricity usage, so they can be friendlier for our climate. https://www.rvo.nl/onderwerpen/energie-besparen-de-industrie/datacenters https://www.rvo.nl/onderwerpen/energie-besparen-de-industrie...
- rat9988 2y ago
  I'm not sure why you are downvoted. Is it wrong?
  - wmf 2y ago
    Probably blowback from "environmentalism at any cost" thinking.
    - Tijdreiziger 2y ago
      I’m not downvoted at the moment, but in any case, it’s not really ‘at any cost’. Salient points from the linked report [1]: * Data centers in the Netherlands use approx. 2% of nationwide electricity production (4% in the US [2]) * Data center electricity usage is nearly constant, while access patterns aren’t * Even heavily used servers spend 1/3 of power usage on idle cycles, 99% for the most lightly used servers * Power-saving modes save approx. 10% of electricity without affecting application performance * Many respondents do not use power-saving modes because of a lack of knowledge, because they fear the consequences, or because they have been instructed not to by their sysadmin/vendor * Nonetheless, latency-sensitive applications (e.g. HPC or HFT) are not well-suited to power-saving modes Given these results, it seems sensible to use power-saving modes by default, unless your workload is extremely latency-sensitive. In any case, I disagree that potential 10% electricity savings across the worldwide data center industry, without affecting application performance, are ‘environmentalism at any cost’. [1, Dutch] Harryvan, D. et al. (2020). Analyse LEAP Track 1 “Powermanagement.” Rijksdienst voor Ondernemend Nederland. URL: https://www.rvo.nl/sites/default/files/2021/01/Rapport%20LEAP_NL_TG.pdf https://www.rvo.nl/sites/default/files/2021/01/Rapport%20LEA... [1, English] Harryvan, D. et al. (2021). Analysis LEAP Track 1 “Powermanagement.” Netherlands Enterprise Agency. URL: https://www.rvo.nl/sites/default/files/2021/01/Rapport%20LEAP_EN_TG.pdf https://www.rvo.nl/sites/default/files/2021/01/Rapport%20LEA... [2] Shehabi, A. et al. (2024) United States Data Center Energy Usage Report (page 5). Berkeley Lab. URL: https://eta-publications.lbl.gov/sites/default/files/2024-12/lbnl-2024-united-states-data-center-energy-usage-report.pdf https://eta-publications.lbl.gov/sites/default/files/2024-12...
PinkSheep 2y ago
You can tune the ondemand (or any other) governor first to ramp up faster and clock down slower. "performance" should be seen as the nuclear option.
- andai 2y ago
  How? I've been wondering the same thing for my laptop. My laptop doesn't even seem to support the ondemand option. It just switches from performance to powersave when I unplug the charger, and becomes painfully slow.

chpatrick 2y ago

Is there any downside to ondemand? If your servers aren't running at 100% then there's no point wasting watts, even if you aren't paying for them, right?

anarazel 2y ago
It performs terrible if you have an intermittent workload. Like e.g. a request response workload where request processing is cheap (so that the time to increase the frequency matters). I've seen cases it's a more than 2x request latency increase. It can be pretty annoying, because it means that systems can perform better under higher load and that you get drastically different latency depending on whether a request is scheduled on a core that just processed another request (already at high freq) or one that was idle. And because the frequency control isn't fun enough, this behavior also exists with cpu idle states. Even at high frequency Linux can enter idle states... I've debugged several cases where this set of issues has caused unintuitive behavior. E.g. a) switching to a more powerful servers drastically increased latency b) optimized code resulting in higher latency / lower throughout because that provided enough idle cycles for a deeper idle time between requests c) slightly increased IO latency leading to significantly worse overall performance, due to the IO getting long though to clock down
- lucb1e 2y ago
  Intermittent workloads on the order of 2 milliseconds, you mean? Frequency scaling is much faster than blinking, as well as ubiquitous in all recent consumer hardware that I'm aware of due to the huge amount of power it saves. Turning it off would, to me, only make sense if you want a server to handle thousands of fast requests per second, but those requests don't come in for periods of, say, 50 ms at a time and so the CPU scales back. Actually, thinking this through, even then it doesn't make much sense to me: if you have that many short requests coming in, the CPU would simply never scale back if it's reasonably constant. It would first need to see some gap, and why not scale the CPU back in that gap (at the cost of having the 1st request of the next batch be a few milliseconds slower)? From there on, every subsequent request is fast again until there's another lull. Keeping the CPU always on high frequency should only be needed if you have a very tight deadline on that surprise request (high-frequency trading perhaps?), or if your requests are coincidentally always spaced by the same amount of time as CPU scaling measures across. I'm sure these things exist but "intermittent workload" is 90% of all workloads and most workloads definitely aren't meaningfully impacted by cpu scaling
  - anarazel 2y ago
    > Intermittent workloads on the order of 2 milliseconds, you mean? Yea. Most of the cases I was looking at were with postgres, with fully cached simple queries. Each taking << 10ms. The problem is more extreme if the client takes some time to actually process the result or there is network latency, but even without it's rather noticeable. > Turning it off would, to me, only make sense if you want a server to handle thousands of fast requests per second, but those requests don't come in for periods of, say, 50 ms at a time and so the CPU scales back. I see regressions at periods well below 50ms, but yea, that's the shape of it. E.g. a postgres client running 1000 QPS over a single TCP connection from a different server, connected via switched 10Gbit Ethernet (ping RTT 0.030ms), has the following client side visible per-query latencies: powersave, idle enabled: 0.392 ms performance, idle enabled: 0.295 ms performance, idle disabled: 0.163 ms If I make that same 1 client go full tilt, instead of limiting it to 1000 QPS: powersave, idle enabled: 0.141 ms performance, idle enabled: 0.107 ms performance, idle disabled: 0.090 ms I'd call that a significant performance change. > if you have that many short requests coming in, the CPU would simply never scale back if it's reasonably constant. Indeed, that's what makes the whole issue so pernicious. One of the ways I saw this was when folks moved postgres to more powerful servers and got worse performance due to frequency/idle handling. The reason being that it made it more likely that cores were idle long enough to clock down. On the same setup as above, if I instead have 800 client connections going full tilt, there's no meaningful difference between powersave/performance and idle enabled/disabled.

Tijdreiziger 2y ago

However, eco-friendly power modes can reduce electricity usage, so they can be friendlier for our climate. https://www.rvo.nl/onderwerpen/energie-besparen-de-industrie/datacenters https://www.rvo.nl/onderwerpen/energie-besparen-de-industrie...

rat9988 2y ago
I'm not sure why you are downvoted. Is it wrong?
- wmf 2y ago
  Probably blowback from "environmentalism at any cost" thinking.
  - Tijdreiziger 2y ago
    I’m not downvoted at the moment, but in any case, it’s not really ‘at any cost’. Salient points from the linked report [1]: * Data centers in the Netherlands use approx. 2% of nationwide electricity production (4% in the US [2]) * Data center electricity usage is nearly constant, while access patterns aren’t * Even heavily used servers spend 1/3 of power usage on idle cycles, 99% for the most lightly used servers * Power-saving modes save approx. 10% of electricity without affecting application performance * Many respondents do not use power-saving modes because of a lack of knowledge, because they fear the consequences, or because they have been instructed not to by their sysadmin/vendor * Nonetheless, latency-sensitive applications (e.g. HPC or HFT) are not well-suited to power-saving modes Given these results, it seems sensible to use power-saving modes by default, unless your workload is extremely latency-sensitive. In any case, I disagree that potential 10% electricity savings across the worldwide data center industry, without affecting application performance, are ‘environmentalism at any cost’. [1, Dutch] Harryvan, D. et al. (2020). Analyse LEAP Track 1 “Powermanagement.” Rijksdienst voor Ondernemend Nederland. URL: https://www.rvo.nl/sites/default/files/2021/01/Rapport%20LEAP_NL_TG.pdf https://www.rvo.nl/sites/default/files/2021/01/Rapport%20LEA... [1, English] Harryvan, D. et al. (2021). Analysis LEAP Track 1 “Powermanagement.” Netherlands Enterprise Agency. URL: https://www.rvo.nl/sites/default/files/2021/01/Rapport%20LEAP_EN_TG.pdf https://www.rvo.nl/sites/default/files/2021/01/Rapport%20LEA... [2] Shehabi, A. et al. (2024) United States Data Center Energy Usage Report (page 5). Berkeley Lab. URL: https://eta-publications.lbl.gov/sites/default/files/2024-12/lbnl-2024-united-states-data-center-energy-usage-report.pdf https://eta-publications.lbl.gov/sites/default/files/2024-12...

PinkSheep 2y ago

You can tune the ondemand (or any other) governor first to ramp up faster and clock down slower. "performance" should be seen as the nuclear option.

andai 2y ago
How? I've been wondering the same thing for my laptop. My laptop doesn't even seem to support the ondemand option. It just switches from performance to powersave when I unplug the charger, and becomes painfully slow.