3 ms·
Longish article about the cost of training increasingly big neural nets. Worried about carbon. "Training such a model would cost US $100 billion and would produ
by abecedarius 5y ago
Longish article about the cost of training increasingly big neural nets. Worried about carbon. "Training such a model would cost US $100 billion and would produce as much carbon emissions as New York City does in a month. And if we estimate the computational burden of a 1 percent error rate, the results are considerably worse."
- Veedrac 5y agoThis number is not credible. Are we done with ImageNet?[1] points out that the best ImageNet models that currently exist surpass the accuracy of the original labels they are trained on. Expecting to reduce error by a factor 10 on unseen data points is incoherent. [1] https://arxiv.org/abs/2006.07159 https://arxiv.org/abs/2006.07159
- abecedarius 5y agoI was unimpressed too; when I posted that snippet there were no other comments and I figured it'd help others judge whether it was worth the read.