3 ms·
It’s good that we can build cheaper LLMs, but the problem companies guzzling energy for LLM training won’t use less energy, they’ll just have better models
by dartos 2y ago
It’s good that we can build cheaper LLMs, but the problem companies guzzling energy for LLM training won’t use less energy, they’ll just have better models
- kaba0 2y agoThat energy is still on the order of a household’s yearly electricity, not that of Argentina as cryptos were, and that’s just for training. Inferring is much cheaper and arguably provides quite a lot of value (even though I also think it is overhyped), for very little energy consumption, probably more is lost due to inefficiency for any physical product.
- dartos 2y agoI got news for you. The 1M GPUs that meta purchased running at full (or less probably) load 24/7 is more than the energy of a single household. The energy cost is in training, not inference.
- kaba0 2y agoWhich is the same order of magnitude? Also, how often do they train from scratch?
- dartos 2y agoI don’t understand your first question, sorry. But for the second: Llama 1, 2, and 3 all have different architectures and needed to be trained from scratch. Llama 1 was released February 2023. Same training story for openAI’s Sora, dalle, and 4o. All of mistral’s models Mamba, Kan, and Each version of rwkv (they’re on 6 now) Not that this list is a result of survivor bias. It’s only looking at their published models too. Not the probably 1000s of individual training experiments that go into producing each model.
- kaba0 2y agoWhich is still absolutely nothing compared to something like youtube’s servers, which is absolutely nothing compared to something like the food industry. Like, if a couple of millions of people can use chatgpt in the manner they do today, would it matter if a house’s yearly energy budget was used up for that? Or 10?
- dartos 2y agoI guess because training AI doesn’t use as much energy as the largest energy consumers on the planet, we just shouldn’t even worry about it.
- deleted 2y ago[deleted]