4 ms·

Reducing the Computational Cost of Deep Reinforcement Learning Research

101 points by theafh 5y ago

igorkraw 5y ago
I'm a bit surprised this paper: https://arxiv.org/abs/1906.05243 https://arxiv.org/abs/1906.05243 wasn't in the citations, nor did the authors build on it
- creato 5y ago
  One of the authors of that paper is acknowledged here, so at the very least the authors were aware of each other before this was posted.
sxp 5y ago
> By doing so, in addition to reducing the environmental impact of our experiments... How does reducing the cost of individual experiments reduce the total environmental impact? Won't more efficient experiments (measured as experiments/Watt-hour or experiments/$) just trigger https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox as more people take advantage of the lower cost?
- throwawaygh 5y ago
  Strikes me as one of those cases where good research uses a silly motivation to get brownie points. use p100s hosted in data centers powered by renewables, stop flying to conferences, don’t do drl just because it’s sexy
  - monocasa 5y ago
    It reads to me as a response to Gebru's paper that led to her getting fired. A paper on the externalities of training large models, like the environmental impacts due to the large amount of computation required: "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" https://dl.acm.org/doi/10.1145/3442188.3445922 https://dl.acm.org/doi/10.1145/3442188.3445922 That would explain why such a mundane subject is in a blog post, it's focused at tech news to be all "we really do care about this thing" rather than academia.
    - visarga 5y ago
      > Can Language Models Be Too Big? That's interesting, GPT-n with 100T parameters would be bored by just reading the whole internet. Too little information, repetitive and on average, junk. You can try the question in reverse too: Can evolution use too much energy? How much energy has it already consumed?
    - pas 5y ago
      It was not the paper that get her fired. (Probably there isn't just one ultimate cause that led to her firing, but maybe we can say that big G has a tendency to fire those who rock the boat too loudly.)
    - dna_polymerase 5y ago
      Why would they fire someone over this piece of shit paper? Does anyone think Google is threatened by the idea of AI using energy?
- qorrect 5y ago
  > just trigger https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox as more people take advantage of the lower cost? I'm guessing that's exactly what will happen.
  - melling 5y ago
    So, it really sounds like we need to reduce our carbon emissions for electricity generation. The windmills and solar are still on the way. Looks like the new goal in the US is 2035: https://www.scientificamerican.com/article/bidens-infrastructure-plan-would-make-electricity-carbon-free-by-2035/ https://www.scientificamerican.com/article/bidens-infrastruc...
- ackbar03 5y ago
  Well I guess you get more bang for your buck/pollution
DrNuke 5y ago
2016 GTX 1070s 8GB will live another day as the bang-on-the-buck ml/dl/drl intro graphic cards then.
TaylorAlexander 5y ago
Amazing that they fired Timnit Gebru [1][2] after she pushed back against the removal of this very subject from one of her research papers, [3] only to publish their own work on it without mentioning her. [1] https://www.nytimes.com/2017/12/31/technology/google-images-voters.html https://www.nytimes.com/2017/12/31/technology/google-images-... [2] https://www.economist.com/science-and-technology/2017/03/02/a-machine-learning-census-of-americas-cities https://www.economist.com/science-and-technology/2017/03/02/... [3] https://www.theverge.com/22309962/timnit-gebru-google-harassment-campaign-jeff-dean https://www.theverge.com/22309962/timnit-gebru-google-harass...
- aceon48 5y ago
  "Fired" aka she said I'm resigning unless you meet my list of demands, and they accepted her resignation.
  - monocasa 5y ago
    She said that they could discuss a possible resignation when she returned from her preplanned vacation. They said don't bother you don't work here anymore. Threatening to resign is no more resigning than threatening to fire is actually firing.
    - srvmshr 5y ago
      Threatening your employer and sending mutinous emails usually doesn't bode well for employment on a private corp, no matter how righteous the cause.
- FartyMcFarter 5y ago
  Was Timnit's paper related to reinforcement learning at all? Not sure how you're concluding that the papers have much similarity.
  - TaylorAlexander 5y ago
    See this article [1] on the retracted paper. The article says: "Gebru’s draft paper points out that the sheer resources required to build and sustain such large AI models means they tend to benefit wealthy organizations, while climate change hits marginalized communities hardest. “It is past time for researchers to prioritize energy efficiency and cost to reduce negative environmental impact and inequitable access to resources,” they write." Meanwhile TFA concludes: "We are simply urging researchers to consider smaller-scale environments as a valuable tool in their investigations, and reviewers to avoid dismissing empirical work that focuses on smaller-scale environments. By doing so, in addition to reducing the environmental impact of our experiments, we will get both a clearer picture of the research landscape and reduce the barriers for researchers from diverse and often underresourced communities, which can only help make our community and scientific advances stronger." Looking more closely: Gebru: "It is past time for researchers to prioritize energy efficiency and cost to reduce negative environmental impact and inequitable access to resources" TFA: "We are simply urging researchers to consider smaller-scale environments... and ... to avoid dismissing empirical work that focuses on smaller-scale environments. ... in addition to reducing the environmental impact of our experiments, we will ...reduce the barriers for researchers from diverse and often underresourced communities" So yes the similarity is striking. In fact its remarkable that google fired Gebru for questioning the retraction of the paper only to realize that these factors must be considered and then to publish nearly the same message six months later. Having worked at Google, I think they wanted Gebru to sugar coat the message more. That’s what they’ve done in TFA. But in doing so they’ve also taken away a publishing opportunity from a talented black woman (who they just proved right) and I think that’s a shame. [1] https://www.technologyreview.com/2020/12/04/1013294/google-ai-ethics-research-paper-forced-out-timnit-gebru/ https://www.technologyreview.com/2020/12/04/1013294/google-a...

101 points by theafh 5y ago

I'm a bit surprised this paper: https://arxiv.org/abs/1906.05243 https://arxiv.org/abs/1906.05243 wasn't in the citations, nor did the authors build on it

creato 5y ago
One of the authors of that paper is acknowledged here, so at the very least the authors were aware of each other before this was posted.

sxp 5y ago

> By doing so, in addition to reducing the environmental impact of our experiments... How does reducing the cost of individual experiments reduce the total environmental impact? Won't more efficient experiments (measured as experiments/Watt-hour or experiments/$) just trigger https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox as more people take advantage of the lower cost?

throwawaygh 5y ago
Strikes me as one of those cases where good research uses a silly motivation to get brownie points. use p100s hosted in data centers powered by renewables, stop flying to conferences, don’t do drl just because it’s sexy
- monocasa 5y ago
  It reads to me as a response to Gebru's paper that led to her getting fired. A paper on the externalities of training large models, like the environmental impacts due to the large amount of computation required: "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" https://dl.acm.org/doi/10.1145/3442188.3445922 https://dl.acm.org/doi/10.1145/3442188.3445922 That would explain why such a mundane subject is in a blog post, it's focused at tech news to be all "we really do care about this thing" rather than academia.
  - visarga 5y ago
    > Can Language Models Be Too Big? That's interesting, GPT-n with 100T parameters would be bored by just reading the whole internet. Too little information, repetitive and on average, junk. You can try the question in reverse too: Can evolution use too much energy? How much energy has it already consumed?
  - pas 5y ago
    It was not the paper that get her fired. (Probably there isn't just one ultimate cause that led to her firing, but maybe we can say that big G has a tendency to fire those who rock the boat too loudly.)
  - dna_polymerase 5y ago
    Why would they fire someone over this piece of shit paper? Does anyone think Google is threatened by the idea of AI using energy?
qorrect 5y ago
> just trigger https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox as more people take advantage of the lower cost? I'm guessing that's exactly what will happen.
- melling 5y ago
  So, it really sounds like we need to reduce our carbon emissions for electricity generation. The windmills and solar are still on the way. Looks like the new goal in the US is 2035: https://www.scientificamerican.com/article/bidens-infrastructure-plan-would-make-electricity-carbon-free-by-2035/ https://www.scientificamerican.com/article/bidens-infrastruc...
ackbar03 5y ago
Well I guess you get more bang for your buck/pollution

DrNuke 5y ago

2016 GTX 1070s 8GB will live another day as the bang-on-the-buck ml/dl/drl intro graphic cards then.

TaylorAlexander 5y ago

Amazing that they fired Timnit Gebru [1][2] after she pushed back against the removal of this very subject from one of her research papers, [3] only to publish their own work on it without mentioning her. [1] https://www.nytimes.com/2017/12/31/technology/google-images-voters.html https://www.nytimes.com/2017/12/31/technology/google-images-... [2] https://www.economist.com/science-and-technology/2017/03/02/a-machine-learning-census-of-americas-cities https://www.economist.com/science-and-technology/2017/03/02/... [3] https://www.theverge.com/22309962/timnit-gebru-google-harassment-campaign-jeff-dean https://www.theverge.com/22309962/timnit-gebru-google-harass...

aceon48 5y ago
"Fired" aka she said I'm resigning unless you meet my list of demands, and they accepted her resignation.
- monocasa 5y ago
  She said that they could discuss a possible resignation when she returned from her preplanned vacation. They said don't bother you don't work here anymore. Threatening to resign is no more resigning than threatening to fire is actually firing.
  - srvmshr 5y ago
    Threatening your employer and sending mutinous emails usually doesn't bode well for employment on a private corp, no matter how righteous the cause.
FartyMcFarter 5y ago
Was Timnit's paper related to reinforcement learning at all? Not sure how you're concluding that the papers have much similarity.
- TaylorAlexander 5y ago
  See this article [1] on the retracted paper. The article says: "Gebru’s draft paper points out that the sheer resources required to build and sustain such large AI models means they tend to benefit wealthy organizations, while climate change hits marginalized communities hardest. “It is past time for researchers to prioritize energy efficiency and cost to reduce negative environmental impact and inequitable access to resources,” they write." Meanwhile TFA concludes: "We are simply urging researchers to consider smaller-scale environments as a valuable tool in their investigations, and reviewers to avoid dismissing empirical work that focuses on smaller-scale environments. By doing so, in addition to reducing the environmental impact of our experiments, we will get both a clearer picture of the research landscape and reduce the barriers for researchers from diverse and often underresourced communities, which can only help make our community and scientific advances stronger." Looking more closely: Gebru: "It is past time for researchers to prioritize energy efficiency and cost to reduce negative environmental impact and inequitable access to resources" TFA: "We are simply urging researchers to consider smaller-scale environments... and ... to avoid dismissing empirical work that focuses on smaller-scale environments. ... in addition to reducing the environmental impact of our experiments, we will ...reduce the barriers for researchers from diverse and often underresourced communities" So yes the similarity is striking. In fact its remarkable that google fired Gebru for questioning the retraction of the paper only to realize that these factors must be considered and then to publish nearly the same message six months later. Having worked at Google, I think they wanted Gebru to sugar coat the message more. That’s what they’ve done in TFA. But in doing so they’ve also taken away a publishing opportunity from a talented black woman (who they just proved right) and I think that’s a shame. [1] https://www.technologyreview.com/2020/12/04/1013294/google-ai-ethics-research-paper-forced-out-timnit-gebru/ https://www.technologyreview.com/2020/12/04/1013294/google-a...