5 ms·
LLMs are a utility, not a platform, and utility markets exert a downward pressure on pricing. Moreover, it's not obvious that -- once trained models hit the wil
by HillRat 2y ago
LLMs are a utility, not a platform, and utility markets exert a downward pressure on pricing. Moreover, it's not obvious that -- once trained models hit the wild -- any one actor has or can develop significant competitive moats that would allow them to escape that price pressure. Beyond that, the digital marginal cost of services needs to be significantly reduced to keep these companies in business, but more efficient models leads to pushing inference out to end-user compute, which hollows out their business model (I assume that Apple dropping out of the OpenAI investment round was partially due to the wildly optimistic valuations involved, partially because they're betting on being able to optimize runtime costs down to iPhone levels).
Basically, I'd argue that LLMs look less like a Web 2.0 social media opportunity and more like Hashicorp or Docker, except with operational expenses running many orders of magnitude higher with costs scaling linearly to revenue.
- Ekaros 2y agoAnd on other side, how many more paying users will there be? And of users that known about AI or have tried it are happy to use whatever is free at the moment. Or just whatever is on Google or Bing with add next to it? Social media is free, subscription services are for stuff you can't get for free easily. But will there actually be similar need for AI? Be it any generation.
- bhouston 2y ago> LLMs are a utility, not a platform, and utility markets exert a downward pressure on pricing. I think competition exerts a downward pressure on pricing, not being a utility personally. But I guess I agree with the utility analogy in that there are massively initial upfront costs and then the marginal costs are low. > more efficient models leads to pushing inference out to end-user compute, which hollows out their business model Faster CPUs have been coming forever but we keep coming up with ways of keeping them busy. I suspect the same pattern with AI. Thus server-based AI will always be better than local. In the future, I expect to be served by many dozens of persistent agents acting on my behalf (more agents as you go further into the future) and they won't be hosted on my smartphone.
- LtWorf 2y agoI'm running on a decade old computer and it is just fine. 10 years ago a decade old computer vs a current one would have made a huge difference.
- bhouston 2y ago> 10 years ago a decade old computer vs a current one would have made a huge difference. For running Word or Excel or Node.js server apps, I would agree with you. But this is where new applications come in. Modern PCs with either a GPU or an NPU can run circles around your PC when it comes to running Llama or StableDiffusion locally. Same with regards to high end graphics, old PCs can not do real-time raytracing or upscaling with their lesser capabilities. I personally do rendering via Blender or astrophotography via PixInsight and I need all the cores + memory I can get for that. Faster PCs make for more opportunities that were not possible earlier. But if you do not change your workloads as the hardware evolves, then you do not need to upgrade.
- insane_dreamer 2y agoAn M3 vs an Intel Macbook from 10 years ago is a shockingly large leap forward (I say this as someone who has both).