4 ms·
At the risk of being pedantic, it's not AI that requires massive resources, chatgpt 3.x was trained on a few million dollars. The jump to trillions being table
by preommr 10mo ago
At the risk of being pedantic, it's not AI that requires massive resources, chatgpt 3.x was trained on a few million dollars. The jump to trillions being table stakes happened because everyone started using free services and there was just too much money in the hands of these tech companies. Among other things.
There are so many chickens that are coming home to roost where LLMs was just the catalyst.
- KaiserPro 10mo ago> it's not AI that requires massive resources no it really is. If you took away training costs, OpenAI would be profitable. When I was at meta they were putting in something like 300k GPUs in a massive shared memory cluster just for training. I think they are planning to triple that, if not more.
- YetAnotherNick 10mo agoYeah for some reason AI energy use is so overreported. Using chatgpt for query does not even use two order of magnitude less energy compared to toasting a bread. And you can eat bread untoasted too if you care about energy use. [1]: https://epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use https://epoch.ai/gradient-updates/how-much-energy-does-chatg...
- ori_b 10mo agoHow many slices of toast are you making a day? If you fly a plane a millimeter, you're using less energy than making a slice of toast; would you also say that it's accurate that all global plane travel is more efficient than making toast?
- YetAnotherNick 10mo ago1-2 slice a day and 1-50 chatgpt query per day. For me it would be within same order of magnitude, and I don't really care about both as both of them are dwarfed by my heater or aircon usage.
- KaiserPro 10mo agoFrom my estimation each second of gpt eats about 0.5-1.5 watthours
- YetAnotherNick 10mo agoYou can say it takes 1800-5400 W. Not sure where you are estimating it from.
- KaiserPro 10mo agoDepending on what you're doing its taking up to 8GPUs working in parallel to serve those queries.
- YetAnotherNick 10mo agoYes but then the batch size is in 100s or even 1000s. These GPU doesn't serve just 1 user at a time.