7 ms·

How Temporal Workload Shifting Can Reduce Carbon Emissions in the Cloud

48 points by mpweiher 5y ago

m0zg 5y ago
Don't stop there. This is what I'd like to know by zipcode as well. What's the best time to charge my wife's Tesla, run large appliances, or GPU workloads in my garage. I bet there are a lot of consumer workloads that could easily be "shifted", and in some states that'd probably have non-trivial CO2 savings.
- ZeroGravitas 5y ago
  There's APIs that exist for this, including both current and 24-48h prediction. Some EV chargers have had this built in for years, though it mostly works by time-of-use tariff. If you don't have one of those (or you want to be extra green) currently there's few end-to-end consumer solutions. The framework is there is most places though, probably just waiting for mass adoption). https://www.watttime.org/api-documentation/ https://www.watttime.org/api-documentation/ https://carbonintensity.org.uk/ https://carbonintensity.org.uk/
c0l0 5y ago
Imagine how much CO2 we could NOT emit into the atmosphere if we just yanked all the superfluous, useless and demonstrably harmful workloads from all the clouds and computers on this planet (instead, or in addition). No training your neural net for thousands of accumulated [CG]PU-years to make racial profiling, droning could-be-"combatants", or targeted advertising more effective. No high speed trading any more for ad real estate on the web, stocks, or natural resources. I so, so wish that in many cases, people would perceive the actual value of simply NOT doing certain things. But we're spiraling into a lethal wet-bulb future on the back of so, so many things we not only could perfectly well do without, but that actually could never, in no conceivable scenario, have a net-positive effect on society in any way. What a darned pity.
- melling 5y ago
  We could just stop burning coal. Most everything else discussed is a rounding error. Bitcoin, for example, was generated from a lot of coal power 40% of the global grid is from coal. We’ve been doing this for decades
  - m0zg 5y ago
    Not really doable, unless you're prepared to take it up with the largest consumer of coal, China, which consumes 51% of it. The US is distant third, after India. In fact as of now the US derives more energy from renewables than from coal (mostly hydro and wind - share of solar is abysmally low). That's the trouble with Western environmentalism. It's glorified NIMBY. Just because we destroyed on-shore manufacturing doesn't mean coal won't be burned elsewhere to make stuff people buy. If anything, more of it will be burned, since the environmental regulations aren't as strict there.
    - simion314 5y ago
      Typical american response, ignores population and history to point the finger at China. I am not sure if all parties in US run this propaganda on all TV channels? or how do this people know to copy paste same bad statistics in the hope to delay doing something. Even if your country is number 21 you should still do something proportional with your population and on how much damage you cause. And for the historic part, if we could calculate how much country put CO2 and methane int he atmosphere they should proportionally put money in a fund for investments in poor countries to help them skip over the coal stage.
      - worldsayshi 5y ago
        Yeah, and a lot of the transition is about pushing down prices of renewable energy solutions so that they become more competitive. It happens when those technologies gets adoption. So even a small country should be able to make a big impact if they adopt green technology early. Prices of photovoltaics for example has gone down to a large degree thanks to early adopters and stimulus. Now it's quite competitive without such help. But it's not the only piece of the puzzle. Other technologies deserves to be pushed towards competitiveness as well.
- HPsquared 5y ago
  I sometimes wonder how much energy Windows Update consumes on a global scale. I have some old machines and updating can mean they are run at 100% for hours, multiply this by every Windows PC and it's a lot of energy.
  - kwhitefoot 5y ago
    I finally dropped Windows 10 from my HP Omen laptop a couple of months ago and installed Linux (had to have it to run Visual Studio until recently). The machine now runs cold when in the past it was distinctly warm all the time even when I had not been using it for hours. I suspect that dropping Windows altogether has cut the consumption of his laptop by more than half.
    - fomine3 5y ago
      You just disabled Nvidia dGPU?
      - kwhitefoot 5y ago
        I never do any graphics intensive stuff so I have no idea. But perhaps that is what happened. But that really just makes it even plainer that Windows is very inefficient. Why would enabling the GPU cause high power usage when the machine has been sitting idle for hours with the screen off?
- agumonkey 5y ago
  Apparently, even all of computing is a small factor in energy use. (maybe bitcoin mining aside) I too cringed hard when imgur and mp4/gifs started to pop... amounts of GB streamed just for short lived memes. It felt insane but all in all the value seems to be low.[0] We got to focus on industries, globalization and heating apparently. [0] I'm all for frugal minimal computing. I want to make a milliwat dumb terminal to be used as administration application client (low lag but low refresh TUI style). I'm sure 80% of tasks can be achieved through this. For the occasional 2D/3D user interaction one can still pop his 2020s smartphone
- tda 5y ago
  This is because we don't price energy and raw materials as much as we value them. Putting a higher price on extracting natural resources will swiftly lead to less waste and less frivolous uses as they then become uneconomical. But as of now, compute is cheap and available, power is cheap and available; what else than wasteful use would you expect? Hopefully not that corporation would somehow feel morally obliged to abide by some arbitrary standard and make less money along the way. But that is effectively what you are suggesting.
  - chartpath 5y ago
    As an interesting aside to carbon taxes and such (which appear to be working to a degree): Was shopping for ETFs a while back and came across the WealthSimple SRI portfolio which is based on their funds. They use a normal ESG index but it also de-lists any companies in the top 25% of emissions for their category. This would seem to help avoid some collusion and complacency in industries like software that are just at a natural advantage in absolute terms compared with, say, coal plants. Of course, it's a retail investor and consumer driven thing which leaves too much on the table for some, but I think there's something to the idea of getting companies to compete on the rate of improvement relative to each industry, in addition to internalizing some of the costs through taxes.
    - sitkack 5y ago
      Interesting take on how to value something, even if it is a top performer but wasteful in how it uses resources, the lack of efficiency make it rank lower. Thing is, most top of category performers in anything are wasteful in some resource. I think kinda by definition. Lets go way out into left field and say even top performing artists and athletes are wasteful, because the slope of the gain to expensive curve flattens the higher up you get. I think for large orgs to be high efficiency it has to be baked into their culture. If you don't have high efficiency, the worst thing you can do is also get large. I think the gamedev world could be akin to digital coal, toxic work env, low pay. Your entire labor force could walk/pivot to something else if better any dimension you compensate them on changes.
reacharavindh 5y ago
If only we stopped up energy by making new GPUs and CPUs solving puzzles for random cryptocoin that is not a real currency used for trade and simply a gambling game of speculation. I wonder what would net energy saving be if crypto currency was deemed illegal - as in making it worthless by criminalising the act of converting *coin to any real currency.
jstsch 5y ago
I'd love for providers like DigitalOcean to show me the CO2e of any VM's + networking infrastructure I'm running. Optimally it'd include the entire LCA (so including manufacture).
- redis_mlc 5y ago
  What you can do is run powertop periodically to see what's consuming the most power, or running frequently thus waking up your cpu from sleep modes.
- solarpunk 5y ago
  GCP and Cloudflare both have this fyi.
  - toomuchtodo 5y ago
    Can’t speak to Cloudflare, but Google partners with https://www.electricitymap.org https://www.electricitymap.org for this functionality. https://api.electricitymap.org/ https://api.electricitymap.org/ (no affiliation, just a fan of their work)
dj_mc_merlin 5y ago
CMV: claiming that computers produce CO2 makes no sense. Computers cannot produce CO2, they can only consume energy made by CO2-generating processes. Isn't the real way to stop producing CO2 to migrate to methods of producing energy that are better at mitigating this? Furthermore, are computers even that power hungry? I cannot imagine they account for more than a tenth of our energy use.
- MargotNorman 5y ago
  As the article is analyzing exactly this, I recommend a read.
jl6 5y ago
What are we to make of, e.g. Google’s claims that their data centers run on 100% renewable energy? How can temporal workload shifting help reduce carbon emissions in the cloud if the energy source is already renewable?
- redis_mlc 5y ago
  Sorry to break it to you, but no data center runs on renewable energy.
- birnbaum 5y ago
  Co-author of the paper here. Google currently only buys enough renewable energy to match their annual electricity use - their data centers do not run on 100% renewables all the time. You can read up on this here: https://www.gstatic.com/gumdrop/sustainability/24-7-explainer.pdf https://www.gstatic.com/gumdrop/sustainability/24-7-explaine... However, they are pushing towards operating on carbon-free energy 27/7 by 2030. For this to work, they will need to (and apparently already do) implement approaches exactly like the one mentioned in the paper.
aaron695 5y ago
We should do a transatlantic power cable, but ok lets look at doing it the abstract way, that is interesting. Infrastructure is a massive cost. You are not going to add infrastructure to do this, that would be going backwards for carbon/environment. All carbon neutral energy is already used up or shipped to other locations, what is really 'happening' here is using wasted fossil fuel energy and wasted infrastructure like air conditioned computers not running at maximum. This is just a fancy way of talking about peak and off peak computing. Lots of papers out there about this. There is a scam if you shift it to a wind power location and say your power is from the wind and the locals are using the fossil fuels bit I guess. That's just cooking the books.
hotz 5y ago
At this rate let's just switch everything off. Get rid of everything and become hunter/gatherers again. This climate change fear mongering is getting really tiring. All these green changes ever do is make normal people's lives miserable.
pflanze 5y ago
I have only read the abstract, so maybe I'm missing something, but how would moving workloads to different times really lower CO2 in the long term unless it's using surplus green power? They talk about "less carbon-intensive" supply, but if it's still 1% coal power at a particular time then using more power at that time will require burning coal to satisfy demands? It would seem to only help to place workloads at times where it could use up otherwise unused solar or wind power. Also, leaving hardware idle is both having a CO2 cost, too (production of hardware that's not used), and, other people who are not CO2 conscious will probably move their workloads to those times where the hardware is under utilized and hence cheaper. So, the whole thing is a zero sum game, making one company's workloads appear greener at the cost of making other companies' workloads worse? (Edit: removed negative tone.)
- birnbaum 5y ago
  Co-author here :) Regarding your first point: You are talking about so-called marginal carbon intensity - and you are right, this would indeed be a better metric for incentivizing scheduling. Unfortunately it is extremely hard to compute (or better: to estimate), we discuss this in Section 3.4 of the paper. Regarding the second point: This paper is more written from a data center operator or service provider perspective. They could, for example, (financially?) incentivize their users to give them some slack in when to schedule workloads. Simply put: Computing should be expensive when energy is dirty. Governments could incentivize this via carbon pricing mechanisms (that are actually currently discussed at COP26)
  - pflanze 5y ago
    In section 3.4 you end with "we consider marginal carbon intensity to be no practical signal for demand management at this point due to high uncertainties". But for the direct actual ecological value that would be the only number that counts, according to my understanding, so basically you're agreeing that right now temporal shifts have no ecological benefit? It appears that you hope that in the future, when marginal carbon intensity data might become available, and temporal shifting actually has value, this is a useful approach, and that as of today, preparing for that future is worthwhile. I can see that point. I think this should be pointed out--this changes the proposal from something that looks like pseudo actionism (that I'm sure some companies' PR departments will still love) to an actual valid plan, even if it doesn't have immediate benefits. PS. also, the marginal carbon intensity benefit still has to outweight the CO2 cost for buying more hardware (to afford running the jobs at those time points, and consequentially leaving the hardware unused at the corresponding counter times). Have you looked into this?

48 points by mpweiher 5y ago

m0zg 5y ago

Don't stop there. This is what I'd like to know by zipcode as well. What's the best time to charge my wife's Tesla, run large appliances, or GPU workloads in my garage. I bet there are a lot of consumer workloads that could easily be "shifted", and in some states that'd probably have non-trivial CO2 savings.

ZeroGravitas 5y ago
There's APIs that exist for this, including both current and 24-48h prediction. Some EV chargers have had this built in for years, though it mostly works by time-of-use tariff. If you don't have one of those (or you want to be extra green) currently there's few end-to-end consumer solutions. The framework is there is most places though, probably just waiting for mass adoption). https://www.watttime.org/api-documentation/ https://www.watttime.org/api-documentation/ https://carbonintensity.org.uk/ https://carbonintensity.org.uk/

c0l0 5y ago

Imagine how much CO2 we could NOT emit into the atmosphere if we just yanked all the superfluous, useless and demonstrably harmful workloads from all the clouds and computers on this planet (instead, or in addition). No training your neural net for thousands of accumulated [CG]PU-years to make racial profiling, droning could-be-"combatants", or targeted advertising more effective. No high speed trading any more for ad real estate on the web, stocks, or natural resources. I so, so wish that in many cases, people would perceive the actual value of simply NOT doing certain things. But we're spiraling into a lethal wet-bulb future on the back of so, so many things we not only could perfectly well do without, but that actually could never, in no conceivable scenario, have a net-positive effect on society in any way. What a darned pity.

melling 5y ago
We could just stop burning coal. Most everything else discussed is a rounding error. Bitcoin, for example, was generated from a lot of coal power 40% of the global grid is from coal. We’ve been doing this for decades
- m0zg 5y ago
  Not really doable, unless you're prepared to take it up with the largest consumer of coal, China, which consumes 51% of it. The US is distant third, after India. In fact as of now the US derives more energy from renewables than from coal (mostly hydro and wind - share of solar is abysmally low). That's the trouble with Western environmentalism. It's glorified NIMBY. Just because we destroyed on-shore manufacturing doesn't mean coal won't be burned elsewhere to make stuff people buy. If anything, more of it will be burned, since the environmental regulations aren't as strict there.
  - simion314 5y ago
    Typical american response, ignores population and history to point the finger at China. I am not sure if all parties in US run this propaganda on all TV channels? or how do this people know to copy paste same bad statistics in the hope to delay doing something. Even if your country is number 21 you should still do something proportional with your population and on how much damage you cause. And for the historic part, if we could calculate how much country put CO2 and methane int he atmosphere they should proportionally put money in a fund for investments in poor countries to help them skip over the coal stage.
    - worldsayshi 5y ago
      Yeah, and a lot of the transition is about pushing down prices of renewable energy solutions so that they become more competitive. It happens when those technologies gets adoption. So even a small country should be able to make a big impact if they adopt green technology early. Prices of photovoltaics for example has gone down to a large degree thanks to early adopters and stimulus. Now it's quite competitive without such help. But it's not the only piece of the puzzle. Other technologies deserves to be pushed towards competitiveness as well.
HPsquared 5y ago
I sometimes wonder how much energy Windows Update consumes on a global scale. I have some old machines and updating can mean they are run at 100% for hours, multiply this by every Windows PC and it's a lot of energy.
- kwhitefoot 5y ago
  I finally dropped Windows 10 from my HP Omen laptop a couple of months ago and installed Linux (had to have it to run Visual Studio until recently). The machine now runs cold when in the past it was distinctly warm all the time even when I had not been using it for hours. I suspect that dropping Windows altogether has cut the consumption of his laptop by more than half.
  - fomine3 5y ago
    You just disabled Nvidia dGPU?
    - kwhitefoot 5y ago
      I never do any graphics intensive stuff so I have no idea. But perhaps that is what happened. But that really just makes it even plainer that Windows is very inefficient. Why would enabling the GPU cause high power usage when the machine has been sitting idle for hours with the screen off?
agumonkey 5y ago
Apparently, even all of computing is a small factor in energy use. (maybe bitcoin mining aside) I too cringed hard when imgur and mp4/gifs started to pop... amounts of GB streamed just for short lived memes. It felt insane but all in all the value seems to be low.[0] We got to focus on industries, globalization and heating apparently. [0] I'm all for frugal minimal computing. I want to make a milliwat dumb terminal to be used as administration application client (low lag but low refresh TUI style). I'm sure 80% of tasks can be achieved through this. For the occasional 2D/3D user interaction one can still pop his 2020s smartphone
tda 5y ago
This is because we don't price energy and raw materials as much as we value them. Putting a higher price on extracting natural resources will swiftly lead to less waste and less frivolous uses as they then become uneconomical. But as of now, compute is cheap and available, power is cheap and available; what else than wasteful use would you expect? Hopefully not that corporation would somehow feel morally obliged to abide by some arbitrary standard and make less money along the way. But that is effectively what you are suggesting.
- chartpath 5y ago
  As an interesting aside to carbon taxes and such (which appear to be working to a degree): Was shopping for ETFs a while back and came across the WealthSimple SRI portfolio which is based on their funds. They use a normal ESG index but it also de-lists any companies in the top 25% of emissions for their category. This would seem to help avoid some collusion and complacency in industries like software that are just at a natural advantage in absolute terms compared with, say, coal plants. Of course, it's a retail investor and consumer driven thing which leaves too much on the table for some, but I think there's something to the idea of getting companies to compete on the rate of improvement relative to each industry, in addition to internalizing some of the costs through taxes.
  - sitkack 5y ago
    Interesting take on how to value something, even if it is a top performer but wasteful in how it uses resources, the lack of efficiency make it rank lower. Thing is, most top of category performers in anything are wasteful in some resource. I think kinda by definition. Lets go way out into left field and say even top performing artists and athletes are wasteful, because the slope of the gain to expensive curve flattens the higher up you get. I think for large orgs to be high efficiency it has to be baked into their culture. If you don't have high efficiency, the worst thing you can do is also get large. I think the gamedev world could be akin to digital coal, toxic work env, low pay. Your entire labor force could walk/pivot to something else if better any dimension you compensate them on changes.

reacharavindh 5y ago

If only we stopped up energy by making new GPUs and CPUs solving puzzles for random cryptocoin that is not a real currency used for trade and simply a gambling game of speculation. I wonder what would net energy saving be if crypto currency was deemed illegal - as in making it worthless by criminalising the act of converting *coin to any real currency.

jstsch 5y ago

I'd love for providers like DigitalOcean to show me the CO2e of any VM's + networking infrastructure I'm running. Optimally it'd include the entire LCA (so including manufacture).

redis_mlc 5y ago
What you can do is run powertop periodically to see what's consuming the most power, or running frequently thus waking up your cpu from sleep modes.
solarpunk 5y ago
GCP and Cloudflare both have this fyi.
- toomuchtodo 5y ago
  Can’t speak to Cloudflare, but Google partners with https://www.electricitymap.org https://www.electricitymap.org for this functionality. https://api.electricitymap.org/ https://api.electricitymap.org/ (no affiliation, just a fan of their work)

dj_mc_merlin 5y ago

CMV: claiming that computers produce CO2 makes no sense. Computers cannot produce CO2, they can only consume energy made by CO2-generating processes. Isn't the real way to stop producing CO2 to migrate to methods of producing energy that are better at mitigating this? Furthermore, are computers even that power hungry? I cannot imagine they account for more than a tenth of our energy use.

MargotNorman 5y ago
As the article is analyzing exactly this, I recommend a read.

jl6 5y ago

What are we to make of, e.g. Google’s claims that their data centers run on 100% renewable energy? How can temporal workload shifting help reduce carbon emissions in the cloud if the energy source is already renewable?

redis_mlc 5y ago
Sorry to break it to you, but no data center runs on renewable energy.
birnbaum 5y ago
Co-author of the paper here. Google currently only buys enough renewable energy to match their annual electricity use - their data centers do not run on 100% renewables all the time. You can read up on this here: https://www.gstatic.com/gumdrop/sustainability/24-7-explainer.pdf https://www.gstatic.com/gumdrop/sustainability/24-7-explaine... However, they are pushing towards operating on carbon-free energy 27/7 by 2030. For this to work, they will need to (and apparently already do) implement approaches exactly like the one mentioned in the paper.

aaron695 5y ago

We should do a transatlantic power cable, but ok lets look at doing it the abstract way, that is interesting. Infrastructure is a massive cost. You are not going to add infrastructure to do this, that would be going backwards for carbon/environment. All carbon neutral energy is already used up or shipped to other locations, what is really 'happening' here is using wasted fossil fuel energy and wasted infrastructure like air conditioned computers not running at maximum. This is just a fancy way of talking about peak and off peak computing. Lots of papers out there about this. There is a scam if you shift it to a wind power location and say your power is from the wind and the locals are using the fossil fuels bit I guess. That's just cooking the books.

hotz 5y ago

At this rate let's just switch everything off. Get rid of everything and become hunter/gatherers again. This climate change fear mongering is getting really tiring. All these green changes ever do is make normal people's lives miserable.

pflanze 5y ago

I have only read the abstract, so maybe I'm missing something, but how would moving workloads to different times really lower CO2 in the long term unless it's using surplus green power? They talk about "less carbon-intensive" supply, but if it's still 1% coal power at a particular time then using more power at that time will require burning coal to satisfy demands? It would seem to only help to place workloads at times where it could use up otherwise unused solar or wind power. Also, leaving hardware idle is both having a CO2 cost, too (production of hardware that's not used), and, other people who are not CO2 conscious will probably move their workloads to those times where the hardware is under utilized and hence cheaper. So, the whole thing is a zero sum game, making one company's workloads appear greener at the cost of making other companies' workloads worse? (Edit: removed negative tone.)

birnbaum 5y ago
Co-author here :) Regarding your first point: You are talking about so-called marginal carbon intensity - and you are right, this would indeed be a better metric for incentivizing scheduling. Unfortunately it is extremely hard to compute (or better: to estimate), we discuss this in Section 3.4 of the paper. Regarding the second point: This paper is more written from a data center operator or service provider perspective. They could, for example, (financially?) incentivize their users to give them some slack in when to schedule workloads. Simply put: Computing should be expensive when energy is dirty. Governments could incentivize this via carbon pricing mechanisms (that are actually currently discussed at COP26)
- pflanze 5y ago
  In section 3.4 you end with "we consider marginal carbon intensity to be no practical signal for demand management at this point due to high uncertainties". But for the direct actual ecological value that would be the only number that counts, according to my understanding, so basically you're agreeing that right now temporal shifts have no ecological benefit? It appears that you hope that in the future, when marginal carbon intensity data might become available, and temporal shifting actually has value, this is a useful approach, and that as of today, preparing for that future is worthwhile. I can see that point. I think this should be pointed out--this changes the proposal from something that looks like pseudo actionism (that I'm sure some companies' PR departments will still love) to an actual valid plan, even if it doesn't have immediate benefits. PS. also, the marginal carbon intensity benefit still has to outweight the CO2 cost for buying more hardware (to afford running the jobs at those time points, and consequentially leaving the hardware unused at the corresponding counter times). Have you looked into this?