4 ms·
No - but we already have cheap LLMs priced way below frontier models. This is not the housing market. There will always be someone willing to take a lower profi
by HarHarVeryFunny 2mo ago
No - but we already have cheap LLMs priced way below frontier models. This is not the housing market. There will always be someone willing to take a lower profit margin for a slice of the pie, and of course smaller models are cheaper to serve so can afford to be cheaper.
DeepSeek recently said that their super-low pricing let's them recoup the cost of the hardware it runs on in 10 months, so there is evidentially plenty of profit to be had over a projected 3+ year lifespan of a "GPU".
Some in the AI industry, or breathing the same air (Dwarkesh) project that limited GPUs will only be used to serve the most expensive models with the highest profit margins, but it is just not what we are seeing. If the only LLMs available were ones at Opus/Fable price points then the GPU scarcity would disappear since the demand at that price is just not there. It's remarkably like trying to fill all the seats on a plane - you can fill a few at 1st class prices, but most of the plane better be coach if you want to sell all the seats.
For a GPU, "selling all the seats", keeping it busy 24x7, is critical to profitability since the primary cost to serving is the GPU which has a limited lifespan.
- WarmWash 2mo agoAs we have seen, those cheap LLMs are cheap because their volume is so low, not because they aren't greedy or discovered some kind of efficiency hack. There is a clear trend of popularity and price.
- HarHarVeryFunny 2mo agoThere's a huge range of cost-per-task variation across models, and the capability of the smaller cheaper models keeps increasing. For example, here we have Fable 5 at $3.14/task vs Kimi K3 at $0.84/task, with very little difference between them in coding capability (and this isn't even a coding/agentic fine tune of Kimi). https://artificialanalysis.ai/models https://artificialanalysis.ai/models We now have models like Qwen 3.8 27B, small enough to run locally, with coding capability similar to Opus 4.5 based on challenging tasks like the Anthropic Kernel challenge. I think we are rapidly getting to the "good enough" stage of LLMs, just like we did long ago with PCs. A cheap PC/LLM is all you need for 99.9% of normal use cases. Maybe nothing can touch whatever latest greatest models Anthropic and OpenAI have when it comes to solving Erdos problems, but most developers are working on problems more like the Anthropic Kernel challenge in complexity (or in fact typically way simpler ones).