3 ms·
> There's a significant environmental cost to training these large models, and it's harder to understand, use, and control these very large models (even as a re
by rvz 3y ago
> There's a significant environmental cost to training these large models, and it's harder to understand, use, and control these very large models (even as a researcher).
Exactly. The worse part is there is NO viable efficient way of training, fine tuning etc and inference with these massive deep learning models for more than a decade since deep neural networks used GPUs for training, and it still requires a substantial amount of compute power and energy that is incinerating the planet and the result is untrustworthy large black-box models that cannot be trusted or transparently explain their decisions, especially with safety-critical or high risk tasks.
Crypto at least has an alternative to the wasteful proof-of-work system, and Ethereum which was formerly PoW has shown it is possible to switch to a greener alternative consensus method.
Deep Learning however, still has not shown such a viable switch and still needs to burn the planet with more data-centers of GPUs, ASICs and FPGAs to create hallucinating models that have shown to break on a single pixel or to confidently regurgitate nonsense as the truth with little understanding in reasoning and also answer with demonstrably false information.
LLMs like this one is still essentially snake-oil BS generators hiding behind regurgitation and sophistry to pretend to show signs of 'intelligence'.
EDIT: It is all true. [0] There is no amount of green-washing to hide the problem of deep learning systems wasting essential resources like thousands of running taps of water. Literally.
[0] https://gizmodo.com/chatgpt-ai-water-185000-gallons-training-nuclear-1850324249 https://gizmodo.com/chatgpt-ai-water-185000-gallons-training...