5 ms·

I was a little disappointed in the quality of this research. I can see where Google is coming from. Some of these topics seem regurgitated (e.g. bias in online

I was a little disappointed in the quality of this research. I can see where Google is coming from. Some of these topics seem regurgitated (e.g. bias in online text) and some are just irrelevant (CO2 emissions). Training language models does not contribute in any meaningful way to global CO2 emissions. Overall not a very strong paper in contrast to the other reactions I've been seeing online.

hezag 6y ago
> I was little disappointed in the quality of this research But the research is not even published yet
- fastball 6y ago
  Kinda the point though isn't it? We're only reading a summary of the paper because apparently the authors aren't confident enough in its quality to release it publicly. You can't claim that Google dismissed this paper out-of-hand while simultaneously saying "oh, but it's too much of a draft to release publicly". Uh, if it was too much of a draft for the public why shouldn't it be too drafty for Google? Are we really pretending that Google has lower standards of quality than the general public?
  - viktorcode 6y ago
    That, or Google is censoring the paper because of the conclusion it comes to, which doesn't align with their business interests.
    - fastball 6y ago
      I’m taking the article at face value, which states the authors think it’s too much of a draft.
    - juanbyrge 6y ago
      If a company is paying a researcher a lofty salary (I am guessing ballpark mid six figures), and provides that researcher virtually unlimited tools and resources, shouldn't that company have a say in what material gets published by the researcher? Why shouldn't it have the ability to restrict research of dubious quality that makes borderline incorrect conclusions about the company?
  - deleted 6y ago
    [deleted]
- rundigen12 6y ago
  I naively expected this article to include a link to the paper somewhere in it, so we could decide for ourselves. Oh well.
joe_the_user 6y ago
Training language models does not contribute in any meaningful way to global CO2 emissions. Google consumes a vast amount of energy: "10.6 terawatt hours in 2018, up from 2.86 terawatt hours in 2011."[1] If training and retraining models is a significant and inefficient part of google's energy consumption, the point doesn't seem insignificant(edit: The most advanced AI model involve as much as computing and energy any programs ever created [2], btw). I'm biased by the impression Google's actual search results haven't improved very much but I don't think I'm alone in that impression. [1] https://www.statista.com/statistics/788540/energy-consumption-of-google/ https://www.statista.com/statistics/788540/energy-consumptio... [2] https://openai.com/blog/ai-and-compute/ https://openai.com/blog/ai-and-compute/
- juanbyrge 6y ago
  I obviously don't have the data, but I would argue that a tiny tiny fraction of Google's energy usage is dedicated to training language models. I'm guessing the vast majority of the energy usage is for serving billions of requests for various products, such as search, youtube, maps, gmail, photos, etc.., and the cpu, network, and storage requirements for those requests. Training and retraining ML models is definitely not on the hot path.

hezag 6y ago

> I was little disappointed in the quality of this research But the research is not even published yet

fastball 6y ago
Kinda the point though isn't it? We're only reading a summary of the paper because apparently the authors aren't confident enough in its quality to release it publicly. You can't claim that Google dismissed this paper out-of-hand while simultaneously saying "oh, but it's too much of a draft to release publicly". Uh, if it was too much of a draft for the public why shouldn't it be too drafty for Google? Are we really pretending that Google has lower standards of quality than the general public?
- viktorcode 6y ago
  That, or Google is censoring the paper because of the conclusion it comes to, which doesn't align with their business interests.
  - fastball 6y ago
    I’m taking the article at face value, which states the authors think it’s too much of a draft.
  - juanbyrge 6y ago
    If a company is paying a researcher a lofty salary (I am guessing ballpark mid six figures), and provides that researcher virtually unlimited tools and resources, shouldn't that company have a say in what material gets published by the researcher? Why shouldn't it have the ability to restrict research of dubious quality that makes borderline incorrect conclusions about the company?
- deleted 6y ago
  [deleted]
rundigen12 6y ago
I naively expected this article to include a link to the paper somewhere in it, so we could decide for ourselves. Oh well.

joe_the_user 6y ago

Training language models does not contribute in any meaningful way to global CO2 emissions. Google consumes a vast amount of energy: "10.6 terawatt hours in 2018, up from 2.86 terawatt hours in 2011."[1] If training and retraining models is a significant and inefficient part of google's energy consumption, the point doesn't seem insignificant(edit: The most advanced AI model involve as much as computing and energy any programs ever created [2], btw). I'm biased by the impression Google's actual search results haven't improved very much but I don't think I'm alone in that impression. [1] https://www.statista.com/statistics/788540/energy-consumption-of-google/ https://www.statista.com/statistics/788540/energy-consumptio... [2] https://openai.com/blog/ai-and-compute/ https://openai.com/blog/ai-and-compute/

juanbyrge 6y ago
I obviously don't have the data, but I would argue that a tiny tiny fraction of Google's energy usage is dedicated to training language models. I'm guessing the vast majority of the energy usage is for serving billions of requests for various products, such as search, youtube, maps, gmail, photos, etc.., and the cpu, network, and storage requirements for those requests. Training and retraining ML models is definitely not on the hot path.