4 ms·
The quality concerns are absolutely justified. The complaints about energy use sound like unfounded, extremely far-fetched arguments just used by people who do
by tgsovlerkhgsel 3y ago
The quality concerns are absolutely justified. The complaints about energy use sound like unfounded, extremely far-fetched arguments just used by people who don't like LLMs for other reasons.
The inference energy cost is likely on the same order of magnitude as the computer + screen used to read the answer (higher wattage, but much shorter time to generate the response than to formulate the request and read it).
The training energy cost is significant only if we ignore that it is used by many people. For GPT-3, I've seen plausible estimates of ~1 GWh, which would equal to about 400 tons of CO2, about as much as a single long-distance plane (total, not per passenger, fuel consumption only) round trip. Estimates for newer models usually ignore the existence and likely use of more efficient accelerators.
- agentultra 3y agoIt’s not completely unfounded. Water consumption skyrocketed while training up the current generation of models [0]. The energy (and water) usage to service requests is not staggering but it is concerning that is uses as much as it does for the output it generates [1] [0] https://futurism.com/critics-microsoft-water-train-ai-drought https://futurism.com/critics-microsoft-water-train-ai-drough... [1] https://www.forbes.com/sites/cindygordon/2024/03/12/chatgpt-and-generative-ai-innovations-are-creating-sustainability-havoc/?sh=d0472193f6d6#:~:text=ChatGPT%20consumes%20over%20half%20a,using%20about%20twenty%2Dnine%20kilowatts https://www.forbes.com/sites/cindygordon/2024/03/12/chatgpt-....
- pantalaimon 3y agoThe energy use of Gentoo is already way above any other distribution as it requires the user to compile all software themselves.
- TheFreim 3y agoHearing complaints about energy consumption from someone using a source-based distribution is quite rich considering they're often wasting much more energy than they need by compiling themselves instead of using a binary distribution.