2 ms·
Perhaps, but this is not guaranteed. For example, demand might shift from datacenter to on-site inference when high-performing models can run locally on consume
by mitthrowaway2 2y ago
Perhaps, but this is not guaranteed. For example, demand might shift from datacenter to on-site inference when high-performing models can run locally on consumer hardware. Kind of like how demand for desktop PCs went down in the 2010s as mobile phones, laptops, and ipads became more capable, even though desktops also became even more capable. People found that running apps on their phone was good enough. Now perhaps everyone will want to run inference on-site for security and privacy, and so demand might shift away from big datacenters into desktops and consumer-grade hardware, and those datacenters will be left bidding each other down looking for workloads.
- AYBABTME 2y agoInference is not where the majority of this CAPEX is used. And even if, monetization will no doubt discourage developers from dispensing the secret sauce to user controlled devices. So I posit that data centres inference is safe for a good while.
- littlestymaar 2y ago> Inference is not where the majority of this CAPEX is used That's what's baffling with Deepseek's results: they spent very little on training (at least that's what they claim). If true, then it's a complete paradigm shift. And even if it's false, the more wide AI usage is, the bigger the share of inference will be, and inference cost will be the main cost driver at some point anyway.
- m3kw9 2y agoYou are looking at one model and also you do realize it isn’t even multimodal, also it shifts training compute to inference compute. They are shifting the paradigm for this architecture for LLMs, but I don’t think this is really new either.
- littlestymaar 2y ago> it shifts training compute to inference compute No, this is the change introduced by o1, what's different with R1 is that its use of RL is fundamentally different (and cheaper) that what OpenAI did.