3 ms·
On the vendor's side, there's incentive to reduce the costs per token, but also an incentive to sell more tokens. On the consumer's side, it's more about what
by ben_w 2y ago
On the vendor's side, there's incentive to reduce the costs per token, but also an incentive to sell more tokens.
On the consumer's side, it's more about what the alternative would be, and the alternative is getting a human to do it.
Right now the general quality of the LLMs is good enough to be a tool, but not good enough to fully replace humans using it — I've seen GPT-4 called an intern, I'd agree having seen student code not too long before ChatGPT came out, and you don't want to let students do a whole project unsupervised either. Even though o1 is better, I think it's still at the "interesting, but not good enough to leave unsupervised" level*.
For tasks where the AI is competent enough to be interesting for whatever task, even if it took 10 kW to run the compute, at $0.1/kWh that's often cheaper than offshoring to a poor country.
As the world's entire electricity supply is 2 TW, which is 250 watts per capita, I think it's not at all implausible that enough people decide that the output of various AI (not just LLMs, also vision, audio, robots etc.) are valuable enough to demand enough of that electricity to make the prices rise.
* and I'm not just saying that because when it gets that good none of us will be employable any more