3 ms·
> LLMs are a utility, not a platform, and utility markets exert a downward pressure on pricing. I think competition exerts a downward pressure on pricing, not
by bhouston 2y ago
> LLMs are a utility, not a platform, and utility markets exert a downward pressure on pricing.
I think competition exerts a downward pressure on pricing, not being a utility personally. But I guess I agree with the utility analogy in that there are massively initial upfront costs and then the marginal costs are low.
> more efficient models leads to pushing inference out to end-user compute, which hollows out their business model
Faster CPUs have been coming forever but we keep coming up with ways of keeping them busy. I suspect the same pattern with AI. Thus server-based AI will always be better than local. In the future, I expect to be served by many dozens of persistent agents acting on my behalf (more agents as you go further into the future) and they won't be hosted on my smartphone.
- LtWorf 2y agoI'm running on a decade old computer and it is just fine. 10 years ago a decade old computer vs a current one would have made a huge difference.
- bhouston 2y ago> 10 years ago a decade old computer vs a current one would have made a huge difference. For running Word or Excel or Node.js server apps, I would agree with you. But this is where new applications come in. Modern PCs with either a GPU or an NPU can run circles around your PC when it comes to running Llama or StableDiffusion locally. Same with regards to high end graphics, old PCs can not do real-time raytracing or upscaling with their lesser capabilities. I personally do rendering via Blender or astrophotography via PixInsight and I need all the cores + memory I can get for that. Faster PCs make for more opportunities that were not possible earlier. But if you do not change your workloads as the hardware evolves, then you do not need to upgrade.
- insane_dreamer 2y agoAn M3 vs an Intel Macbook from 10 years ago is a shockingly large leap forward (I say this as someone who has both).