5 ms·

As I read this article I felt myself being extremely critical of it. Part of this may be that I'm enamored with AI and so I feel defensive about it, but I can't

As I read this article I felt myself being extremely critical of it. Part of this may be that I'm enamored with AI and so I feel defensive about it, but I can't help the feeling that this article needed more work. For example: > Moreover, the organizations best placed to produce a bill — companies like Meta, Microsoft, and OpenAI — simply aren’t sharing the relevant information. To me this shows both unfamiliarity with large corporate structures and unfamiliarity with AI research. The former because, if you want an exact accounting at a company where there are several teams running dozens or hundreds of models, it becomes someone's job essentially to compile this information because it takes a lot of work. So, you are surprised that a corporation doesn't outlay $200k/year or more getting these figures to decorate your article with? The corporations do know, on a month-to-month basis, what they're spending on this stuff - since they settle the invoices. But doing the work to get these figures into a simple, digestible form is a lot of effort, and I think that quite frankly, that effort should fall on the journalist, since it's the journalist who gets the benefit from having those numbers... As to unfamiliarity with AI research, many papers I have been reading lately are very interested in measuring and minimizing the compute cost of models, and they often compare different methods and are often extremely precise about their training process and equipment. I feel like this article wants to sell the story that these faceless corpos don't care about energy consumption, but the researchers definitely do. Granted certain specific products / models such as ChatGPT 4 do not disclose exactly their process, but I feel like it would not be difficult to come up with a good estimate using similar models (mostly documented right out in the open in scientific papers). Tracking the energy consumption of AI is an important and emerging issue, but this article feels too partisan to be useful.

zx8080 3y ago
Is parent comment AI generated?
smartmic 3y ago
At the very least, the article has brought to the forefront the awareness that, in addition to all the other effects of the hype, the research and use of AI on a large scale also consumes an extreme amount of energy. Since we should all be concerned about protecting our resources and thus the planet - and that means reducing unnecessary energy consumption - this figure should be clearly included in the general costs of AI. And by that I don't mean in the electricity bills of corporations. Of course, it is in the interest of researchers to increase the efficiency of modelling and application, but for other reasons. As long as the big, well-known companies are in charge, the target figure towards which everything is optimised and maximised is commercial profit. To cut a long story short, the only way to gain some ground here is through independent regulation, for example to get more transparency into this issue.
- sdenton4 3y ago
  To be sure, we only care about electricity usage because of externalities, most importantly CO2 emissions. Many of the big players have gone to great lengths to build out renewables capacity to power their data centers, which tends not to be considered in these articles.
- Gormo 3y ago
  > Since we should all be concerned about protecting our resources and thus the planet To my knowledge, most of the researchers and users of AI are using their own resources, not "ours", so I'm not sure there's much to be worried about. > And by that I don't mean in the electricity bills of corporations. What else would it be accounted for in? The users of electricity are purchasing it from the producers, who in turn purchase equipment and services from vendors, etc. It's all a chain of specific transactions among specific parties all the way down. There's no point at which some fuzzy collection of arbitrarily aggregated people is involved as one of the parties. > As long as the big, well-known companies are in charge, the target figure towards which everything is optimised and maximised is commercial profit. And they make that profit by delivering value to their customers -- what's the problem there? > To cut a long story short, the only way to gain some ground here is through independent regulation, for example to get more transparency into this issue. What is "independent regulation"? Who is conducting it, what makes them "independent" and what is their incentive to be involved in the first place?
mepiethree 3y ago
I work in the sustainability industry so this is obviously important to me, but I agree the article needed more work. Comparing the energy required to train GPT3 to watching Netflix is nonsensical. Training is a global task. A better comparison would be the cost to train GPT vs that to upload all Netflix movies. Plus, there are weird things with the figures. It lists the 0.012kwh figure required to charge a smartphone, but the source it cites for that number explicitly says that the correct number is 0.022kwh. Was this article hallucinated by AI? Overall though, frankly I found it comforting that AI would “only” use 0.5% of global electricity, as I expected much higher. Google and several other peers are committed to being 100% renewable anyway. Still, credit to the author for starting discussion about this.
- Fripplebubby 3y ago
  Yes, I ultimately I guess it does not hurt anything to have this article in circulation even if I have some critiques, better to discuss the elephant in the room.
lynndotpy 3y ago
> As to unfamiliarity with AI research, many papers I have been reading lately are very interested in measuring and minimizing the compute cost of models, and they often compare different methods and are often extremely precise about their training process and equipment It's worth knowing this is a very recent phenomena. Before ~2020, it was extremely rare to see any such paper, with most of the focus in publishing being on maximizing metrics like accuracy. The biggest force pushing people toward smaller models were researchers at smaller institutions who lacked access to big GPU clusters. > So, you are surprised that a corporation doesn't outlay $200k/year or more getting these figures to decorate your article with? I think it's incorrect to suggest these figures would only serve to make for a better "The Verge" article. One of the externalized costs here is climate change.
- Fripplebubby 3y ago
  Fair point about recency. Actually my point is that to specifically isolate costs that are from AI is a lot of work at a company of this scale. These companies are absolutely accountable for their overall energy usage and externalities!
loeg 3y ago
Yeah. Meta has internally widely shared the expected size of a new very large cluster they are building in 2024 and from the number of H100s or whatever you can probably calculate a reasonable low estimate on power draw (there will be other machines to support the compute, e.g., storage). I don't know that it's public yet so I won't say the exact size but I expect it to be leaked or spoken about publicly shortly anyway. It's a lot of MW. They are semi-joking talking about building nuclear reactors to power future DCs.

zx8080 3y ago

Is parent comment AI generated?

smartmic 3y ago

At the very least, the article has brought to the forefront the awareness that, in addition to all the other effects of the hype, the research and use of AI on a large scale also consumes an extreme amount of energy. Since we should all be concerned about protecting our resources and thus the planet - and that means reducing unnecessary energy consumption - this figure should be clearly included in the general costs of AI. And by that I don't mean in the electricity bills of corporations. Of course, it is in the interest of researchers to increase the efficiency of modelling and application, but for other reasons. As long as the big, well-known companies are in charge, the target figure towards which everything is optimised and maximised is commercial profit. To cut a long story short, the only way to gain some ground here is through independent regulation, for example to get more transparency into this issue.

sdenton4 3y ago
To be sure, we only care about electricity usage because of externalities, most importantly CO2 emissions. Many of the big players have gone to great lengths to build out renewables capacity to power their data centers, which tends not to be considered in these articles.
Gormo 3y ago
> Since we should all be concerned about protecting our resources and thus the planet To my knowledge, most of the researchers and users of AI are using their own resources, not "ours", so I'm not sure there's much to be worried about. > And by that I don't mean in the electricity bills of corporations. What else would it be accounted for in? The users of electricity are purchasing it from the producers, who in turn purchase equipment and services from vendors, etc. It's all a chain of specific transactions among specific parties all the way down. There's no point at which some fuzzy collection of arbitrarily aggregated people is involved as one of the parties. > As long as the big, well-known companies are in charge, the target figure towards which everything is optimised and maximised is commercial profit. And they make that profit by delivering value to their customers -- what's the problem there? > To cut a long story short, the only way to gain some ground here is through independent regulation, for example to get more transparency into this issue. What is "independent regulation"? Who is conducting it, what makes them "independent" and what is their incentive to be involved in the first place?

mepiethree 3y ago

I work in the sustainability industry so this is obviously important to me, but I agree the article needed more work. Comparing the energy required to train GPT3 to watching Netflix is nonsensical. Training is a global task. A better comparison would be the cost to train GPT vs that to upload all Netflix movies. Plus, there are weird things with the figures. It lists the 0.012kwh figure required to charge a smartphone, but the source it cites for that number explicitly says that the correct number is 0.022kwh. Was this article hallucinated by AI? Overall though, frankly I found it comforting that AI would “only” use 0.5% of global electricity, as I expected much higher. Google and several other peers are committed to being 100% renewable anyway. Still, credit to the author for starting discussion about this.

Fripplebubby 3y ago
Yes, I ultimately I guess it does not hurt anything to have this article in circulation even if I have some critiques, better to discuss the elephant in the room.

lynndotpy 3y ago

> As to unfamiliarity with AI research, many papers I have been reading lately are very interested in measuring and minimizing the compute cost of models, and they often compare different methods and are often extremely precise about their training process and equipment It's worth knowing this is a very recent phenomena. Before ~2020, it was extremely rare to see any such paper, with most of the focus in publishing being on maximizing metrics like accuracy. The biggest force pushing people toward smaller models were researchers at smaller institutions who lacked access to big GPU clusters. > So, you are surprised that a corporation doesn't outlay $200k/year or more getting these figures to decorate your article with? I think it's incorrect to suggest these figures would only serve to make for a better "The Verge" article. One of the externalized costs here is climate change.

Fripplebubby 3y ago
Fair point about recency. Actually my point is that to specifically isolate costs that are from AI is a lot of work at a company of this scale. These companies are absolutely accountable for their overall energy usage and externalities!

loeg 3y ago

Yeah. Meta has internally widely shared the expected size of a new very large cluster they are building in 2024 and from the number of H100s or whatever you can probably calculate a reasonable low estimate on power draw (there will be other machines to support the compute, e.g., storage). I don't know that it's public yet so I won't say the exact size but I expect it to be leaked or spoken about publicly shortly anyway. It's a lot of MW. They are semi-joking talking about building nuclear reactors to power future DCs.