4 ms·

They detail the energy used and therefore estimated carbon emissions which is interesting. When I estimate the raw electricity cost using 7-20 cents per kWh for

They detail the energy used and therefore estimated carbon emissions which is interesting. When I estimate the raw electricity cost using 7-20 cents per kWh for US commercial rates, then we are only talking about $16-50k for electricity, that seems pretty small! Is my math wrong? Is there any information on how much the computing costs were for renting the clusters? Is the barrier to entry for a 7B model only a couple $100K? EDIT: https://news.ycombinator.com/item?id=39223467#39224534 https://news.ycombinator.com/item?id=39223467#39224534 Perhaps only $85K total

anonylizard 3y ago
Despite the typical complaints about "X new thing harming the environment!!!", LLMs are as friendly as it gets, it 1. Consumes a minor amount of electricity (Data centers is only 2% of US electricity use, and currently AI is maybe only 5-10% of that). Its trivial compared to say metal smelting. 2. Consume water for cooling. That's it, there is 0 direct pollution generated from AI, and even the water use is very minor compared to say farming, and can be improved via more water efficient cooling techs. The main concern is the scaling speed. As LLMs scale up 10x, 100x, 1000x, those previously very minor electricity costs can quickly become grid impacting in a decade.
- marmaduke 3y ago
  I can't buy this kind of argument anymore. How about the external effect of AI steering the entire semiconductor industry to increase GPU/NPU capacity?
  - vortegne 3y ago
    This kind of argument is actually totally valid. But only if you subscribe to the current meta of widely accepted handwaving. Externalities are never a part of capitalist math. Non-trivial consequences can never hurt if one never looks further than their own nose.
StopTheTechies 3y ago
> we are only talking about $16-50k for electricity, that seems pretty small I suppose this depends greatly on how you view the utility of LLMs. In a capitalist sense, sure—there's great utility here persuading VCs to part with their coins and jobs to be replaced with correspondingly larger profit margins. But the opportunity cost of not solving major problems most of humanity can agree on seems nearly incalculably large. Not that capitalists give a shit.
- wegfawefgawefg 3y ago
  This is a process of exploring new technology. Research is expensive and probably doesnt always yield immediate returns, but when it does you get infinite returns. Imagine how not obvious the first machines must have seemed at the start of the industrial revolution. You only have to feed a man and he can work, but a machine requires iron, oil, water, fuel, engineers, operators. The up front cost for exploring early digging machines must have been absurd. And im sure some people at the time thought: "Wow we could be spending this money on bread for the poor instead." Arent you glad we didnt.
  - deleted 3y ago
    [deleted]
  - antonvs 3y ago
    If we had spent the money on bread for the poor instead, we wouldn't be facing an existential threat created by our lack of understanding of the consequences of our actions, and our collective inability to respond to that effectively.
    - wegfawefgawefg 3y ago
      Consider the world before industry... You really want to have 10 kids and have 50% or more of them die before 10 years old? You want a world before penecillin and antibiotics? No computers? No travel. Women getting marrried off at 15 immediatly pregnant. Most of the world in absolute poverty. Destroyed by a single bad season. Mass famines, plagues, tribal warfare that sweeps over your village. No clean water and soap. malnutrition. These are just non problems for huge portions of the planet now.
passion__desire 3y ago
Facing unbearable heat, Qatar has begun to air-condition the outdoors : https://www.washingtonpost.com/graphics/2019/world/climate-environment/climate-change-qatar-air-conditioning-outdoors/ https://www.washingtonpost.com/graphics/2019/world/climate-e... I feel like we trying to optimize what we measure. No such measurements happen for other industries. How much does Las Vegas use electricity for the extravagant display of lights, water shows and so on.

anonylizard 3y ago

Despite the typical complaints about "X new thing harming the environment!!!", LLMs are as friendly as it gets, it 1. Consumes a minor amount of electricity (Data centers is only 2% of US electricity use, and currently AI is maybe only 5-10% of that). Its trivial compared to say metal smelting. 2. Consume water for cooling. That's it, there is 0 direct pollution generated from AI, and even the water use is very minor compared to say farming, and can be improved via more water efficient cooling techs. The main concern is the scaling speed. As LLMs scale up 10x, 100x, 1000x, those previously very minor electricity costs can quickly become grid impacting in a decade.

marmaduke 3y ago
I can't buy this kind of argument anymore. How about the external effect of AI steering the entire semiconductor industry to increase GPU/NPU capacity?
- vortegne 3y ago
  This kind of argument is actually totally valid. But only if you subscribe to the current meta of widely accepted handwaving. Externalities are never a part of capitalist math. Non-trivial consequences can never hurt if one never looks further than their own nose.

StopTheTechies 3y ago

> we are only talking about $16-50k for electricity, that seems pretty small I suppose this depends greatly on how you view the utility of LLMs. In a capitalist sense, sure—there's great utility here persuading VCs to part with their coins and jobs to be replaced with correspondingly larger profit margins. But the opportunity cost of not solving major problems most of humanity can agree on seems nearly incalculably large. Not that capitalists give a shit.

wegfawefgawefg 3y ago
This is a process of exploring new technology. Research is expensive and probably doesnt always yield immediate returns, but when it does you get infinite returns. Imagine how not obvious the first machines must have seemed at the start of the industrial revolution. You only have to feed a man and he can work, but a machine requires iron, oil, water, fuel, engineers, operators. The up front cost for exploring early digging machines must have been absurd. And im sure some people at the time thought: "Wow we could be spending this money on bread for the poor instead." Arent you glad we didnt.
- deleted 3y ago
  [deleted]
- antonvs 3y ago
  If we had spent the money on bread for the poor instead, we wouldn't be facing an existential threat created by our lack of understanding of the consequences of our actions, and our collective inability to respond to that effectively.
  - wegfawefgawefg 3y ago
    Consider the world before industry... You really want to have 10 kids and have 50% or more of them die before 10 years old? You want a world before penecillin and antibiotics? No computers? No travel. Women getting marrried off at 15 immediatly pregnant. Most of the world in absolute poverty. Destroyed by a single bad season. Mass famines, plagues, tribal warfare that sweeps over your village. No clean water and soap. malnutrition. These are just non problems for huge portions of the planet now.

passion__desire 3y ago

Facing unbearable heat, Qatar has begun to air-condition the outdoors : https://www.washingtonpost.com/graphics/2019/world/climate-environment/climate-change-qatar-air-conditioning-outdoors/ https://www.washingtonpost.com/graphics/2019/world/climate-e... I feel like we trying to optimize what we measure. No such measurements happen for other industries. How much does Las Vegas use electricity for the extravagant display of lights, water shows and so on.