3 ms·

It's one of the tired tropes that gets brought up every time AI/ML is brought up. Everything we do has climate change impact. Power consumption is among the on

It's one of the tired tropes that gets brought up every time AI/ML is brought up. Everything we do has climate change impact. Power consumption is among the ones that's easiest to get "green", and there is significant progress specifically from cloud operators (at least Google, I assume others are similar). https://www.reddit.com/r/MachineLearning/comments/hwfjej/d_the_cost_of_training_gpt3/fyziw88/ https://www.reddit.com/r/MachineLearning/comments/hwfjej/d_t... suggests hundreds of GPU-years for training such a model under optimal assumptions, so let's assume thousands for the new models under real assumptions. https://images.nvidia.com/content/technologies/volta/pdf/tesla-volta-v100-datasheet-letter-fnl-web.pdf https://images.nvidia.com/content/technologies/volta/pdf/tes... says 300 W. 3000 GPU-years at 300 W = 7.9 million kWh. This would assume that they used those GPUs and not more efficient accelerators. https://www.eia.gov/tools/faqs/faq.php?id=74&t=11 https://www.eia.gov/tools/faqs/faq.php?id=74&t=11 says 0.855 pounds of CO2 emissions per kWh. That's 388 g in normal units, and in line with what other countries are reporting (Germany was 420 g/kWh). g/kWh = metric tons per million kWh. This assumes the data centers are not using "greener" than average power. So training such a model produces ~3000 metric tons of CO2 emissions under the assumption of 3000 GPU-years (how convenient). A round trip from SFO to London in Economy is roughly 0.9 metric tons. (https://www.icao.int/environmental-protection/Carbonoffset/Pages/default.aspx0 https://www.icao.int/environmental-protection/Carbonoffset/P... Building a house (of unfortunately unspecified size) is 15 to 100 tons. (https://climate.mit.edu/ask-mit/how-much-co2-emitted-building-new-house https://climate.mit.edu/ask-mit/how-much-co2-emitted-buildin...) So while building one of these models does have an impact, it's on the order of other common activities that benefit far fewer people. Because once trained, those models provide values to millions of users. Going back to the flights example, training one such massive model is likely about as bad as one larger research conference, once you consider the impact of hotels etc. IMO the demands to justify the impact are ridiculous, coming from people who are just looking for any excuse to criticize, on par with demanding that any researcher doing any research justifies the carbon footprint of their commute as part of their research paper. Thus, it's a good thing that they didn't waste time and space in their paper addressing those claims, and we only see an early, commented out section that's equivalent to "TODO: Should we address these claims that keep getting thrown?"

ChatGTP 4y ago
I understand your view point, but I guess your argument really boils down to, it's all justifiable in some way or another. I don't disagree with you either, I live in a foreign country and fly home to see family each year, so I'm far from perfect in this regard. I guess this proves my point though, it's all justifiable to some degree: "I have to go see my family.", "this benefits millions." and on it goes. We have to realize that it this stage, basically nothing, including the current LLMs will survive climate change if we keep this up.
jwestbury 4y ago
> Because once trained, those models provide values to millions of users. Do they? Those of us in tech often take the positive value of technological progress as a given, but when looking at, e.g., the example in OpenAI's paper of GPT4 tricking a human into solving a CAPTCHA, I think it's quite clear there's possible negative value as well, and claims about "value to millions of users" probably need to be substantiated.

ChatGTP 4y ago

I understand your view point, but I guess your argument really boils down to, it's all justifiable in some way or another. I don't disagree with you either, I live in a foreign country and fly home to see family each year, so I'm far from perfect in this regard. I guess this proves my point though, it's all justifiable to some degree: "I have to go see my family.", "this benefits millions." and on it goes. We have to realize that it this stage, basically nothing, including the current LLMs will survive climate change if we keep this up.

jwestbury 4y ago

> Because once trained, those models provide values to millions of users. Do they? Those of us in tech often take the positive value of technological progress as a given, but when looking at, e.g., the example in OpenAI's paper of GPT4 tricking a human into solving a CAPTCHA, I think it's quite clear there's possible negative value as well, and claims about "value to millions of users" probably need to be substantiated.