4 ms·
Isn't it the case that training a ML model consumes vastly more power than using the trained model for inference? If so, the problem may be somewhat overstated
by lovemenot 4y ago
Isn't it the case that training a ML model consumes vastly more power than using the trained model for inference?
If so, the problem may be somewhat overstated. Training is a dispatchable power load. It can be scheduled for periods of surplus power in regions where surplus power is periodically abundant. Training a ML model could literally follow the sun.
- brucethemoose2 4y agoAt Big Tech scale, the same model is used for inference so many times that most of the energy is probably from inference.
- palata 4y agoNote that if you have surplus of power, maybe you want to use it more wisely :).
- lovemenot 4y agoPerhaps, you are right about utility, but I can think of few power loads as well suited to dispatch. Bauxite has to be shipped to Iceland to take advantage of their abundant geothermal and hydro power for Aluminium smelting. Data moves with much less friction.