4 ms·
I'm a little skeptical of a full on 2008-style 'burst'. I imagine it'll be closer to a slow deflation as these companies need to turn a profit. Fundamentally,
by ej88 1y ago
I'm a little skeptical of a full on 2008-style 'burst'. I imagine it'll be closer to a slow deflation as these companies need to turn a profit.
Fundamentally, serving a model via API is profitable (re: Dario, OpenAI), and inference costs come down drastically over time.
The main expense comes twofold:
1. The cost of train a new model is extremely expensive. GPUs / yolo runs / data
2. Newer models tend to churn through more tokens and be more expensive to serve in the beginning before optimizations are made.
(not including payrolls)
OpenAI and Anthropic can become money printers once they downgrade the Free tiers, add ads or other attention monetizing methods, and rely on a usage model once people and businesses become more and more integrated with LLMs, which are undoubtedly useful.
- gjsman-1000 1y agohttps://www.wheresyoured.at/how-to-argue-with-an-ai-booster/#ultimate-booster-quip-the-cost-of-inference-is-coming-down-this-proves-that-things-are-getting-cheaper https://www.wheresyoured.at/how-to-argue-with-an-ai-booster/...
- ej88 1y agoNot really sure how this article refutes what I said? He defines it as "everything that happens from when you put a prompt in to generate an output" -> but he seems to conflate inference with a query. Putting in input to generate the next single token is inference. A query or response just means the LLM repeats this until the stop token is emitted. (Happy to be corrected here) The cost of inference per token is going down - the cost per query goes up because models consume more tokens, which was my point. Either way, charging consumers per token pretty much guarantees that serving models is profitable (each of Anthropic's prior models turn a profit). The consumer-friendly flat 20$ subscription is not sustainable in the long run. https://epoch.ai/data-insights/llm-inference-price-trends https://epoch.ai/data-insights/llm-inference-price-trends https://www.snellman.net/blog/archive/2025-06-02-llms-are-cheap/ https://www.snellman.net/blog/archive/2025-06-02-llms-are-ch... https://x.com/eladgil/status/1827521805755806107 https://x.com/eladgil/status/1827521805755806107