3 ms·
> Inference is not where the majority of this CAPEX is used That's what's baffling with Deepseek's results: they spent very little on training (at least that's
by littlestymaar 2y ago
> Inference is not where the majority of this CAPEX is used
That's what's baffling with Deepseek's results: they spent very little on training (at least that's what they claim). If true, then it's a complete paradigm shift.
And even if it's false, the more wide AI usage is, the bigger the share of inference will be, and inference cost will be the main cost driver at some point anyway.
- m3kw9 2y agoYou are looking at one model and also you do realize it isn’t even multimodal, also it shifts training compute to inference compute. They are shifting the paradigm for this architecture for LLMs, but I don’t think this is really new either.
- littlestymaar 2y ago> it shifts training compute to inference compute No, this is the change introduced by o1, what's different with R1 is that its use of RL is fundamentally different (and cheaper) that what OpenAI did.