5 ms·

Google-led paper pushes back against claims of AI inefficiency

33 points by reckel 5y ago

the-rc 5y ago
As I suspected at the time (https://news.ycombinator.com/item?id=25314818 https://news.ycombinator.com/item?id=25314818), the Google numbers are lower than anything in the Strubell paper, because you have to start from a much lower PUE and factor in additional efficiencies. (It did mention that power consumption of TPUs was not public knowledge.) It's not like Google hasn't been bragging about a PUE of 1.2 or better for more than a decade. For its employees to reference a paper with much worse assumptions was odd to say the least. It doesn't look like anyone with production experience was involved. And here we have this paper now, with many more details than what Google discloses usually...
- jeffbee 5y ago
  The power consumption of tpu v2 and v3 are both given in the paper, so if it was unknown before, it no longer is.
psychlops 5y ago
Suddenly bitcoin looks a lot cheaper burning 121 Terrawatt hours per year versus the 200 Terrawatt hours per data center.
- yyyk 5y ago
  Datacenters provide value to society. Bitcoin provides value to ransomware.
- kens 5y ago
  > the 200 Terrawatt hours per data center. That number is completely fictional, off by orders of magnitude.
causality0 5y ago
Frankly, Google publishing a paper about AI is like Exxon publishing a paper about the environment. No company on earth has such a vested interest, or as strong a fetish for using fewer and fewer humans to provide and support its services.
- SpicyLemonZest 5y ago
  At the same time, though, they have a uniquely informed perspective on what it’s like to run AI services at scale. Resolving to disregard AI research from Google seems like resolving to disregard transportation research from UPS.
  - pessimizer 5y ago
    Their uniquely informed perspective makes it very difficult for the less informed to detect deception. It's better that they supply data to independent researchers who aren't going to later end up at Google, and allow them to provide the interpretation and context.
- Isinlor 5y ago
  Take it from Greenpeace: https://www.greenpeace.org/usa/news/greenpeace-response-to-google-carbon-free-by-2030-commitment/ https://www.greenpeace.org/usa/news/greenpeace-response-to-g... > Greenpeace response to Google Carbon-Free by 2030 commitment, September 14, 2020 > By becoming the first major tech company to commit to power its data centers with carbon-free energy around the clock, Google is setting a new high-bar for the sector: a break-up with fossil fuels altogether. I didn't think these people could be happy about anything, but what a surprise. Seems like they like Google approach. Now Google needs to deliver, but I have no doubt that they will. It's really just good business for corporations like Facebook, Google and Microsoft to increase efficiency and use the cheapest sources of power i.e. renewable. It just has to take some time to do the switch. There is a lot of infrastructure involved.
  - craftinator 5y ago
    > Now Google needs to deliver, but I have no doubt that they will. Really? I'm thinking there's about a 90% chance that it will be a stillbirth. And if not, how long will they actually keep it up? Not a great track record for maintaining "commitments" past 5 years or so.
taeric 5y ago
White I confess that it sounds incredible that training some models uses more carbon than several households, I'm not clear on what the point is. Nor on if that is a meaningful comparison. Stated differently, one of the largest cloud companies uses a ton of computer resources. Not exactly surprising. I do think there are good ROI questions there, but I didn't see many suggestions on how to actually address them. Just feels like a playground fight between peers that think they can win with an appeal to fairness. All while none of them are actually getting things improved. :(
c4wrd 5y ago
Although I do see the waste as some may say it is, I don’t think comparing a models emissions to that of the lifetime emissions of five average cars is fair and ultimately is hyperbolic, but please let me explain. Let’s say the average car lasts 20 years. Actually let’s say it lasts 50 years. Back of the hand math means shows that that’s equivalent to about 50*5=250 U.S. car emissions per year. Yes, that is a large number, but if you look at the value provided to society by things like voice translation, picture recognition, etc., that the set of all Google users enjoy, you have to consider that those models provide value to tens of millions of people, and the cost is equivalent to only 250 of those people driving cars for a year, i.e. the efficiency is pretty great in my opinion. These news articles are hyperbolic and not addressing the true waste we have in society. Once again I’m not arguing that there isn’t inefficiency and waste, but we’re ignoring the fact that every one of the those Google users (or at least a substantial amount) has a higher carbon footprint in totality that FAR surpasses that of a single model. I do support further research in this area, but we must consider the harm us engineers cause when we write papers that can be manipulated so well by media to create click bait articles that only enable the mindset of “big tech is the actual problem” while individuals shift responsibility of their own carbon emissions onto others.
- goatcode 5y ago
  "It uses too much electricity" has become the new curmudgeon punt. I've begun ignoring it outright. Thank you for spelling out the bigger picture for those who are pondering the silly abuse of environmental concern signaling bandwagon. Maybe someday we can combat actual threats to our environment, probably using AI.
  - hobs 5y ago
    If complaining about waste, compare it to what you could achieve with the same resources. Ignoring something outright is not a good approach, you basically have created an opposite tribe with the same illogical basis. If we can do the same work in 1/10000th the effort, then we should decry the waste.
    - 5y ago
deleted 5y ago
[deleted]
xbar 5y ago
Google is probably very optimistic in its numbers. I do not care at all. Tuning low order terms because of narratives in media is not a solution for carbon reduction. Target beef production. Your "43 cars' worth of carbon for an entire year" number is a hundredth of the margin of error on the estimated carbon output for US beef production. Laughable to fret about.
andrewmutz 5y ago
We need to take climate change more seriously, but focusing on the carbon emissions of large language models does not seem like a good way to fight climate change. The total emissions just aren't that high. The largest model listed, GPT-3, led to 552 metric tonnes of carbon being emitted. For comparison, every year Google employees emit 644,000 metric tons of Carbon just commuting to work. (4.6 tonnes a year per car times 140,000 employees).
blt 5y ago
It's absurd to make up formulae about energy consumption for something so complicated. Let's see the electricity bill for the data center. Yes, this will require some effort to kick out other tasks and make sure the machines are fully saturated. edit: the paper does not actually try to compute footprint as a function of "choice of algorithm" like the article implies - it's closer to "the electricity bill for the data center". See my comment below.
- jeffbee 5y ago
  That doesn’t make a lot of sense. Removing the other workloads reduces the efficiency of the datacenter. The whole point of cloud computing, really.
  - blt 5y ago
    I think this workload is big enough to saturate a lot of machines. But I should have read the original paper instead of the article. The article says: > This latest Google-led research... defines the footprint of a model as a function of several variables. They include the choice of algorithm, the program that implements it, the number of processors that run the program, the speed and power of those processors, a datacenter’s efficiency in delivering power and cooling the processors, and the energy supply mix... This, in my opinion, would be absurd. There is no way to directly map the choice of algorithm to energy consumption. But in fact, this is not the paper's definition of the footprint, it is only a list of things that influence it. The actual definition is Footprint = (electrical_energy_train + queries * electrical_energy_inference) * CO2e/KWh. In other words, the definition only uses total energy consumption. They compute electrical_energy_train by directly instrumenting the power consumption per machine. It's not a formula involving quantities like "choice of algorithm".
skybrian 5y ago
The paper is here: https://arxiv.org/abs/2104.10350 https://arxiv.org/abs/2104.10350 In section 4.8, they have a snarky comparison to Bitcoin: > Stated alternatively, ~70M people have Bitcoin wallets yet Google consumes 1/10th of Bitcoin’s energy to provide services for billions of people, and all of Google’s energy use is offset. I wrote a bit more commentary that I’ll link to instead of copying here: https://tildes.net/~enviro/wir/carbon_emissions_and_large_neural_network_training https://tildes.net/~enviro/wir/carbon_emissions_and_large_ne...
arsome 5y ago
A bit like saying Exxon led paper pushes back against claims of climate change.

33 points by reckel 5y ago

As I suspected at the time (https://news.ycombinator.com/item?id=25314818 https://news.ycombinator.com/item?id=25314818), the Google numbers are lower than anything in the Strubell paper, because you have to start from a much lower PUE and factor in additional efficiencies. (It did mention that power consumption of TPUs was not public knowledge.) It's not like Google hasn't been bragging about a PUE of 1.2 or better for more than a decade. For its employees to reference a paper with much worse assumptions was odd to say the least. It doesn't look like anyone with production experience was involved. And here we have this paper now, with many more details than what Google discloses usually...

jeffbee 5y ago
The power consumption of tpu v2 and v3 are both given in the paper, so if it was unknown before, it no longer is.

psychlops 5y ago

Suddenly bitcoin looks a lot cheaper burning 121 Terrawatt hours per year versus the 200 Terrawatt hours per data center.

yyyk 5y ago
Datacenters provide value to society. Bitcoin provides value to ransomware.
kens 5y ago
> the 200 Terrawatt hours per data center. That number is completely fictional, off by orders of magnitude.

causality0 5y ago

Frankly, Google publishing a paper about AI is like Exxon publishing a paper about the environment. No company on earth has such a vested interest, or as strong a fetish for using fewer and fewer humans to provide and support its services.

SpicyLemonZest 5y ago
At the same time, though, they have a uniquely informed perspective on what it’s like to run AI services at scale. Resolving to disregard AI research from Google seems like resolving to disregard transportation research from UPS.
- pessimizer 5y ago
  Their uniquely informed perspective makes it very difficult for the less informed to detect deception. It's better that they supply data to independent researchers who aren't going to later end up at Google, and allow them to provide the interpretation and context.
Isinlor 5y ago
Take it from Greenpeace: https://www.greenpeace.org/usa/news/greenpeace-response-to-google-carbon-free-by-2030-commitment/ https://www.greenpeace.org/usa/news/greenpeace-response-to-g... > Greenpeace response to Google Carbon-Free by 2030 commitment, September 14, 2020 > By becoming the first major tech company to commit to power its data centers with carbon-free energy around the clock, Google is setting a new high-bar for the sector: a break-up with fossil fuels altogether. I didn't think these people could be happy about anything, but what a surprise. Seems like they like Google approach. Now Google needs to deliver, but I have no doubt that they will. It's really just good business for corporations like Facebook, Google and Microsoft to increase efficiency and use the cheapest sources of power i.e. renewable. It just has to take some time to do the switch. There is a lot of infrastructure involved.
- craftinator 5y ago
  > Now Google needs to deliver, but I have no doubt that they will. Really? I'm thinking there's about a 90% chance that it will be a stillbirth. And if not, how long will they actually keep it up? Not a great track record for maintaining "commitments" past 5 years or so.

taeric 5y ago

White I confess that it sounds incredible that training some models uses more carbon than several households, I'm not clear on what the point is. Nor on if that is a meaningful comparison. Stated differently, one of the largest cloud companies uses a ton of computer resources. Not exactly surprising. I do think there are good ROI questions there, but I didn't see many suggestions on how to actually address them. Just feels like a playground fight between peers that think they can win with an appeal to fairness. All while none of them are actually getting things improved. :(

c4wrd 5y ago

Although I do see the waste as some may say it is, I don’t think comparing a models emissions to that of the lifetime emissions of five average cars is fair and ultimately is hyperbolic, but please let me explain. Let’s say the average car lasts 20 years. Actually let’s say it lasts 50 years. Back of the hand math means shows that that’s equivalent to about 50*5=250 U.S. car emissions per year. Yes, that is a large number, but if you look at the value provided to society by things like voice translation, picture recognition, etc., that the set of all Google users enjoy, you have to consider that those models provide value to tens of millions of people, and the cost is equivalent to only 250 of those people driving cars for a year, i.e. the efficiency is pretty great in my opinion. These news articles are hyperbolic and not addressing the true waste we have in society. Once again I’m not arguing that there isn’t inefficiency and waste, but we’re ignoring the fact that every one of the those Google users (or at least a substantial amount) has a higher carbon footprint in totality that FAR surpasses that of a single model. I do support further research in this area, but we must consider the harm us engineers cause when we write papers that can be manipulated so well by media to create click bait articles that only enable the mindset of “big tech is the actual problem” while individuals shift responsibility of their own carbon emissions onto others.

goatcode 5y ago
"It uses too much electricity" has become the new curmudgeon punt. I've begun ignoring it outright. Thank you for spelling out the bigger picture for those who are pondering the silly abuse of environmental concern signaling bandwagon. Maybe someday we can combat actual threats to our environment, probably using AI.
- hobs 5y ago
  If complaining about waste, compare it to what you could achieve with the same resources. Ignoring something outright is not a good approach, you basically have created an opposite tribe with the same illogical basis. If we can do the same work in 1/10000th the effort, then we should decry the waste.
  - 5y ago

deleted 5y ago

[deleted]

xbar 5y ago

Google is probably very optimistic in its numbers. I do not care at all. Tuning low order terms because of narratives in media is not a solution for carbon reduction. Target beef production. Your "43 cars' worth of carbon for an entire year" number is a hundredth of the margin of error on the estimated carbon output for US beef production. Laughable to fret about.

andrewmutz 5y ago

We need to take climate change more seriously, but focusing on the carbon emissions of large language models does not seem like a good way to fight climate change. The total emissions just aren't that high. The largest model listed, GPT-3, led to 552 metric tonnes of carbon being emitted. For comparison, every year Google employees emit 644,000 metric tons of Carbon just commuting to work. (4.6 tonnes a year per car times 140,000 employees).

blt 5y ago

It's absurd to make up formulae about energy consumption for something so complicated. Let's see the electricity bill for the data center. Yes, this will require some effort to kick out other tasks and make sure the machines are fully saturated. edit: the paper does not actually try to compute footprint as a function of "choice of algorithm" like the article implies - it's closer to "the electricity bill for the data center". See my comment below.

jeffbee 5y ago
That doesn’t make a lot of sense. Removing the other workloads reduces the efficiency of the datacenter. The whole point of cloud computing, really.
- blt 5y ago
  I think this workload is big enough to saturate a lot of machines. But I should have read the original paper instead of the article. The article says: > This latest Google-led research... defines the footprint of a model as a function of several variables. They include the choice of algorithm, the program that implements it, the number of processors that run the program, the speed and power of those processors, a datacenter’s efficiency in delivering power and cooling the processors, and the energy supply mix... This, in my opinion, would be absurd. There is no way to directly map the choice of algorithm to energy consumption. But in fact, this is not the paper's definition of the footprint, it is only a list of things that influence it. The actual definition is Footprint = (electrical_energy_train + queries * electrical_energy_inference) * CO2e/KWh. In other words, the definition only uses total energy consumption. They compute electrical_energy_train by directly instrumenting the power consumption per machine. It's not a formula involving quantities like "choice of algorithm".

skybrian 5y ago

The paper is here: https://arxiv.org/abs/2104.10350 https://arxiv.org/abs/2104.10350 In section 4.8, they have a snarky comparison to Bitcoin: > Stated alternatively, ~70M people have Bitcoin wallets yet Google consumes 1/10th of Bitcoin’s energy to provide services for billions of people, and all of Google’s energy use is offset. I wrote a bit more commentary that I’ll link to instead of copying here: https://tildes.net/~enviro/wir/carbon_emissions_and_large_neural_network_training https://tildes.net/~enviro/wir/carbon_emissions_and_large_ne...

arsome 5y ago

A bit like saying Exxon led paper pushes back against claims of climate change.