3 ms·

I believe that API pricing runs at a healthy margin, at least compared to the server and energy costs used to serve the tokens. Recent conversation here on tha

by simonw 5mo ago

I believe that API pricing runs at a healthy margin, at least compared to the server and energy costs used to serve the tokens. Recent conversation here on that topic: https://news.ycombinator.com/item?id=47062534#47063134 https://news.ycombinator.com/item?id=47062534#47063134

bigbuppo 5mo ago
There isn't a single thing about how the AI companies are operating that looks like a normal business. I know people who were in the room when Scott Sullivan, CFO of Worldcom, assured everyone that the future was bright at Worldcom days before they collapsed. So you'll have to excuse me if I don't believe the words of someone whose sole job is to justify hundreds of billions of dollars being thrown at Anthropic when he says their future is bright.
- simonw 5mo ago
  I agree that the amount of investment thrown at these companies is absurd. But I also think that their API token pricing represents a real margin over the inference costs for serving those tokens. Both things can be true at once.
gpugreg 5mo ago
> There exist a large number of people who are absolutely convinced that LLM providers are all running inference at a loss in order to capture the market and will drive the prices up sky high as soon as everyone is hooked. > I think this is often a mental excuse for continuing to avoid engaging with this tech, in the hope that it will all go away. Thanks for that psychological explanation. I was wondering why people were simply ignoring the math that shows that inference at API pricing can be quite profitable, e.g. published here for DeepSeek V3/R1 with 545% profitability: https://github.com/deepseek-ai/open-infra-index/blob/main/202502OpenSourceWeek/day_6_one_more_thing_deepseekV3R1_inference_system_overview.md#statistics-of-deepseeks-online-service https://github.com/deepseek-ai/open-infra-index/blob/main/20...

by simonw 5mo ago

bigbuppo 5mo ago

There isn't a single thing about how the AI companies are operating that looks like a normal business. I know people who were in the room when Scott Sullivan, CFO of Worldcom, assured everyone that the future was bright at Worldcom days before they collapsed. So you'll have to excuse me if I don't believe the words of someone whose sole job is to justify hundreds of billions of dollars being thrown at Anthropic when he says their future is bright.

simonw 5mo ago
I agree that the amount of investment thrown at these companies is absurd. But I also think that their API token pricing represents a real margin over the inference costs for serving those tokens. Both things can be true at once.

gpugreg 5mo ago

> There exist a large number of people who are absolutely convinced that LLM providers are all running inference at a loss in order to capture the market and will drive the prices up sky high as soon as everyone is hooked. > I think this is often a mental excuse for continuing to avoid engaging with this tech, in the hope that it will all go away. Thanks for that psychological explanation. I was wondering why people were simply ignoring the math that shows that inference at API pricing can be quite profitable, e.g. published here for DeepSeek V3/R1 with 545% profitability: https://github.com/deepseek-ai/open-infra-index/blob/main/202502OpenSourceWeek/day_6_one_more_thing_deepseekV3R1_inference_system_overview.md#statistics-of-deepseeks-online-service https://github.com/deepseek-ai/open-infra-index/blob/main/20...