> However, with LLMs coming, I could see data center energy usage jump 10x over the next decade. Just a note: we are probably right now at peak inefficiency fo

> However, with LLMs coming, I could see data center energy usage jump 10x over the next decade. Just a note: we are probably right now at peak inefficiency for LLMs. There are obvious improvements available but they will only be explored by organizations which have different time vs resources trade-off calculations compared to OpenAI and Google. Right now OpenAi and Google are racing to ship products. A working training framework is more important than saving $X (where X is a very large number) because capturing marketshare is seen as vital and they have no shortage of capital. But the massive and rapid progress seen by projects like llama-cpp show how much performance they are leaving on the table. (And yes this also applies at training time).