4 ms·
Jeff is right. Training a machine learning model for any common task is getting radically cheaper and more energy efficient. This is a combination of both bet
by nl 7y ago
Jeff is right.
Training a machine learning model for any common task is getting radically cheaper and more energy efficient.
This is a combination of both better hardware (eg, Google TPUs) and better optimisation of training techniques (which is mostly done outside Google).
Eg, DAWN Bench[1] benchmarks training of ReseNet50 to 93% accuracy. This used to take days.
Now the FastAI group has shown you can do it in 18 minutes on 16 p3.16xlarge on AWS Spot instances. This is a huge energy saving.
Huawei has shown you can do it on 16x8xV100s on their cloud in less than 3 minutes.
[1] https://dawn.cs.stanford.edu/benchmark/ https://dawn.cs.stanford.edu/benchmark/
- paganel 7y ago“Energy saving” does not mean “cool, then we won’t use/consume that saved energy anymore”, it means “cool, now there’s more stuff for us to do/build with the same energy as before”. And more stuff to do/build means more pressure on the environment.
- aguyfromnb 7y agoElectronic computers have become vastly more energy efficient since they became widely used in the 60s. On net, do we spend more or less energy on computers now than we did then? That's the paradox. The cheaper energy becomes, the more uses we find for it.
- choppaface 7y agoCan’t wait for Cerebras. They can probably do an ImageNet in a couple seconds.
- nl 7y agoI'll believe it when I see independent tests.