3 ms·
> Inference providers are already profitable. That surprises me, do you remember where you learned that?
by oa335 10mo ago
> Inference providers are already profitable.
That surprises me, do you remember where you learned that?
- nl 10mo agoLots of sources, and you can do the math yourself. Here's a few good ones: https://github.com/deepseek-ai/open-infra-index/blob/main/202502OpenSourceWeek/day_6_one_more_thing_deepseekV3R1_inference_system_overview.md https://github.com/deepseek-ai/open-infra-index/blob/main/20... (suggests Deepseek is making 80% raw margin on inference) https://www.snellman.net/blog/archive/2025-06-02-llms-are-cheap/ https://www.snellman.net/blog/archive/2025-06-02-llms-are-ch... https://martinalderson.com/posts/are-openai-and-anthropic-really-losing-money-on-inference/ https://martinalderson.com/posts/are-openai-and-anthropic-re... (there's a HN discussion of this where it was pointed out this overestimates the costs) https://www.tensoreconomics.com/p/llm-inference-economics-from-first https://www.tensoreconomics.com/p/llm-inference-economics-fr... (long, but the TL;DR is that serving Lllama 3.3 70B costs around $0.28/million tokens input, $0.95 output at high utilization. These are close to what we see in the market: https://artificialanalysis.ai/models/llama-3-3-instruct-70b/providers?pricing-relationships=input-and-output-pricing https://artificialanalysis.ai/models/llama-3-3-instruct-70b/... )