2 ms·
It reads to me as a response to Gebru's paper that led to her getting fired. A paper on the externalities of training large models, like the environmental impa
by monocasa 5y ago
It reads to me as a response to Gebru's paper that led to her getting fired. A paper on the externalities of training large models, like the environmental impacts due to the large amount of computation required: "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?"
https://dl.acm.org/doi/10.1145/3442188.3445922 https://dl.acm.org/doi/10.1145/3442188.3445922
That would explain why such a mundane subject is in a blog post, it's focused at tech news to be all "we really do care about this thing" rather than academia.
- visarga 5y ago> Can Language Models Be Too Big? That's interesting, GPT-n with 100T parameters would be bored by just reading the whole internet. Too little information, repetitive and on average, junk. You can try the question in reverse too: Can evolution use too much energy? How much energy has it already consumed?
- pas 5y agoIt was not the paper that get her fired. (Probably there isn't just one ultimate cause that led to her firing, but maybe we can say that big G has a tendency to fire those who rock the boat too loudly.)
- dna_polymerase 5y agoWhy would they fire someone over this piece of shit paper? Does anyone think Google is threatened by the idea of AI using energy?