7 ms·

> A version of Google’s language model, BERT, which underpins the company’s search engine, produced 1,438 pounds of CO2 equivalent in Strubell’s estimate So...

• xnx 6y ago
  (Having not read any of this). Isn't Google committed to be net carbon neutral/negative? If so, does the claimed extra electricity usage matter?
  • nullc 6y ago
    Seems so: https://sustainability.google/commitments/ https://sustainability.google/commitments/
  • xiphias2 6y ago
    The paper should have been probably focused on the title only, because the CO2 usage looks like a publicity stunt. She also didn't subtract the amount of CO2 saved by improved productivity by getting better search results.
    • ardit33 6y ago
      Yep, it feels very disingenuous.... It is like someone complaining that google map uses x, amount of energy, yet the old alternative (printing maps, and getting lost) was much worse CO2 wise.
  • lern_too_spel 6y ago
    Does OpenAI? Does the rest of the industry? Everybody is training larger and larger models, with research groups competing on who will be the first to scale up to some number of parameters. It seems obvious that this would be something an ethicist would be interested in researching to figure out what the impact will be and start discussions about how to offset it.
  • CydeWeys 6y ago
    Google isn't the only player in the AI field. The paper is about the entire field in general, not just what Google is doing. And plenty of the other companies in the field have not made similar pledges. The total amount of electricity being used so far for training models does not yet move the needle in a significant way, but the point of the paper is that it's currently growing in an unbounded exponential fashion. And you know how exponential functions work.
• ozborn 6y ago
  There are models much larger than BERT with a much larger footprint, GPT-3 being the most well kniwn example. Models like BERT aren't just trained once either when they are developed, but trained again with different domains, different parameters, different tasks in some cases. There is also fine-tuning (more frequent, less carbon intendive), so these are real environmental problems, and others have pointed them out.
  • melling 6y ago
    Google claims they neutralized their legacy emissions over the entire history of the company. Now about the billion cars on the road and the 40% of the world’s electricity being generated from coal.
    • marcinzm 6y ago
      The existence of other problems does not make a particular problem go away and humanity can in fact focus on more than one thing at a time.
      • melling 6y ago
        Google is carbon neutral. How much more of a problem are a billion cars and 40% of our electricity being generated from coal? We’ve squandered decades ignoring the big problems and now people want to run into the weeds with a thousand little problems, that individually don’t amount to much.
• throwaway74453 6y ago
  I find these CO2 arguments from model training extremely misleading. If this is what you care about, why don't you actually try to make electricity cleaner. Realistic, that's a much more effective way to address this tiny issue, together with the grand total of 50% of humanity's CO2 emissions on top. To me it always seems like virtue signalling, showing that you "care", while not having to actually make any realistic progress. Go build nuclear power plants, or at least prevent them from shutting down, if you want you're electricity to be at 12 g CO2 / kWh rather than 500 from natural gas or 800 from oil (you can also move your servers to France or Ontario). Do not focus on tiny, 10% efficiency gains in NLP. That's a marginal gain within a miniature cause to start with.
  • raverbashing 6y ago
    Of course. Wasn't Google prioritizing green energy for their datacenters?
    • verdverm 6y ago
      yes, https://cloud.google.com/sustainability https://cloud.google.com/sustainability Google is the largest corporate purchaser of renewable energy in the world
• HarryHirsch 6y ago
  1438 pounds of CO2 is 250 liters of gasoline, or thereabouts, that's pumping gas five times. How long does a tank of gas last the average Google employee? Less than a week, considering what East Bay traffic looks like. People get paid to publish that stuff?
  • dragonwriter 6y ago
    The issue isn't how much BERT uses, the issue is the trend over time in how much models use, with BERT being a recent data point. The whole point of having AI ethicists is to identify current indicators of potential future ethical problems so that they can be considered in guiding the direction of development, so that you minimize acute ethical crisis.
    • baskire 6y ago
      You do realize just how small this co2 output is compared to the human co2 output it replaces. You realize the shear scale of co2 output the office the engineers who wrote the model, drove to work, education, etc produced. I’m actually suprised just how small of a co2 output it was.
      • dragonwriter 6y ago
        > You do realize just how small this co2 output is compared to the human co2 output it replaces. Sure, the issue is that the scale of models is increasing by orders of magnitude in a fairly short span of years, as well as the range of applications rapidly expanding. It doesn't take long for that kind of growth to go from a trivial issue to catastrophic one, and it's the exact kind of risk you have ethicists in a field to call out while it still is trivial so that some of the energy of people doing technical development gets directed to mitigate the risk of it ever reaching the catastrophic stage.
        • baskire 6y ago
          Energy has a cost. Using 100$ of energy is much different than 1 million. Capitalism actually mitigates the risk. Companies won’t pay 1 million in R&D to replace 100k of salary.
• machinelabo 6y ago
  This is research in the form of bike shedding. Let’s talk about how the bike shed is going to look for a nuclear power plant. CO2 impact for stuff like this - we probably have bigger fish to fry.