3 ms·
Inference is not where the majority of this CAPEX is used. And even if, monetization will no doubt discourage developers from dispensing the secret sauce to use
by AYBABTME 2y ago
Inference is not where the majority of this CAPEX is used. And even if, monetization will no doubt discourage developers from dispensing the secret sauce to user controlled devices. So I posit that data centres inference is safe for a good while.
- littlestymaar 2y ago> Inference is not where the majority of this CAPEX is used That's what's baffling with Deepseek's results: they spent very little on training (at least that's what they claim). If true, then it's a complete paradigm shift. And even if it's false, the more wide AI usage is, the bigger the share of inference will be, and inference cost will be the main cost driver at some point anyway.
- m3kw9 2y agoYou are looking at one model and also you do realize it isn’t even multimodal, also it shifts training compute to inference compute. They are shifting the paradigm for this architecture for LLMs, but I don’t think this is really new either.
- littlestymaar 2y ago> it shifts training compute to inference compute No, this is the change introduced by o1, what's different with R1 is that its use of RL is fundamentally different (and cheaper) that what OpenAI did.