3 ms·
Celebrating new ways to further burn up the planet rather than discovering more efficient and better ways for training, inference and fine-tuning AI systems wit
by rvz 3y ago
Celebrating new ways to further burn up the planet rather than discovering more efficient and better ways for training, inference and fine-tuning AI systems without needing to scale up more GPUs, TPUs, data centers and water for the same purpose.
The end result of this announcement is another expensive system only available to the same incumbent of tech giants with tens of billions at their disposal.
- renewiltord 3y agoWe found a way: nuclear fission reactors. So that problem is solved.
- anaganisk 3y agoYet, the climate is still changing. Inflation is rising. The world definitely has advanced but never became a better place.
- renewiltord 3y agoThe world has never been better. This is an incredible time to be alive.
- gwoolhurme 3y agoI don't feel that way personally... I have Nth level anxiety, maybe even N+1 level anxiety about losing my job to AI. Maybe in the future whoever comes next can enjoy things, but this literally keeps me up at night. With talks of extinction, job loss, etc. I feel like I wish I wasn't alive at this time.
- rvz 3y ago> We found a way: nuclear fission reactors. Nope. I'm talking about efficient methods in training, inferencing and fine-tuning these AI models that doesn't require lots of data centers, TPUs, GPUs, etc. You're talking about something else. Petrol and diesel cars are already burning the planet, but the main difference is, that there are efficient alternatives available today like electric cars to use instead. AI (Deep learning) however, does not have any viable and efficient methods in training, fine-turning these AI models, at all [0] [1] and wastes a tremendous amount of resources, all to keep up with scalability. So that problem is still NOT solved after a decade of using GPUs, the wastage is getting worse. [0] https://gizmodo.com/chatgpt-ai-water-185000-gallons-training-nuclear-1850324249 https://gizmodo.com/chatgpt-ai-water-185000-gallons-training... [1] https://www.independent.co.uk/tech/chatgpt-data-centre-water-consumption-b2318972.html https://www.independent.co.uk/tech/chatgpt-data-centre-water...
- detrites 3y ago> efficient methods in training, inferencing and fine-tuning these AI models that doesn't require lots of data centers, TPUs, GPUs, etc. Exactly the types of problems future AI models could solve. Dire climate alarms are based on the predictions made using models. As modelling advances as a field, both predictions, and solutions become more and more voluminous and accurate, along with revealing mistakes and failures of prior models. Anyone concerned with climate should rally behind this kind of general progress. Further, it simply is progressing, and fields that don't embrace it, will be left behind. We're in the midst of an unprecedented revolution which touches all.
- qwertox 3y ago> efficient methods in training, inferencing and fine-tuning these AI models Which can also be archived by training more with the same amount of spent energy. Why learn about training ("make training more efficient") on old hardware, which is more energy inefficient?
- rvz 3y agoIt goes more fundamental than that in the algorithms and it should not take tens of billions of dollars with multiple data centers to train, learn, fine-tune and do inference with these AI models. A decade later, there are no viable alternatives to solve that instead of the costly replacement of hardware with more expensive hardware. Add that towards scalability and you will realize that training AI models scales terribly with more data as it is very energy and time inefficient. Even if you replace all the hardware in the data centers it still wouldn't reduce the emissions regardless and replacing them also costs at most billions either way. That is my the entire point. So that does nothing to solve the issue. Only ignores and prolongs it.
- smoldesu 3y ago> A decade later, there are no viable alternatives to solve that instead of the costly replacement of hardware with more expensive hardware. I mean, that's the root of scaling as a principle, right? You could viably start training an AI on your cell phone. It would be completely useless, lack meaningful parameter saturation and take months to reach an inferencing checkpoint, but you could do it. Nvidia is offering a similar system to people, but at a scale that doesn't suck like a cellphone does. Businesses can then choose how much power they need on-site, or rent it from a cloud provider. If a product like this convinces some customers to ditch older and less efficient training silicon, I don't see how it's any more antagonistic than other CPU designers with perennial product updates.
- metaphor 3y ago> The end result of this announcement is another expensive system only available to the same incumbent of tech giants with tens of billions at their disposal. Certainly wasn't the case when a public research university partnership seeded by a generous donation from Nvidia co-founder/UF alumnus Chris Malachowsky was formally announced[1] shortly after DGX A100 launch[2] several years ago, never mind the handful of other academic early adopters mentioned in the press release. Of course, we tend to conveniently forget such exogenous details. [1] https://news.ufl.edu/2020/07/nvidia-partnership/ https://news.ufl.edu/2020/07/nvidia-partnership/ [2] https://nvidianews.nvidia.com/news/nvidias-new-ampere-data-center-gpu-in-full-production https://nvidianews.nvidia.com/news/nvidias-new-ampere-data-c...