3 ms·

Remember that OpenAI is subsidized from here to the highway. A better way to model this, since you seem interested is the following: How much would it cost yo

by emil-lp 4mo ago

Remember that OpenAI is subsidized from here to the highway. A better way to model this, since you seem interested is the following: How much would it cost you to start such a service for, say, 10k users? Any other internet service has had virtually Zero cost, $0. Google, Facebook, youtube, Wikipedia, you name it. They all went into the dumpster to pick up a thrown away desktop computer, and they could serve up towards 100k if not a million users. How much would it cost you to serve, say, 10k simultaneous users with a SOTA model? And if you wanted to go cash positive after a year, how much would each user have to pay?

simianwords 4mo ago
> How much would it cost you to serve, say, 10k simultaneous users with a SOTA model? And if you wanted to go cash positive after a year, how much would each user have to pay? My post has this same argument - we have multiple third party companies running open weight models. They are obviously not subsidised. And people are willing to pay for it. And these models are as good as the SOTA models from last year. So this kinda proves my point that SOTA is sustainable.
- emil-lp 4mo ago
  I didn't find the answer there, that's why I asked. What hardware is needed, how much of it, cooling, and what does it all cost you? Or are you saying I can take my old desktop and serve Deepseek v3.2 to 10k users simultaneously and it would cost me about $1 per megatoken?
  - simianwords 4mo ago
    I'm simply saying this: there are third party hosters of Open Weight models like deepseek and they have been doing this for a while. Obviously they are not subsidised, do you disagree? If you agree, they have a way to price it at a point that people wanna pay for it and also they aren't losing money. So there's nothing inherent about inference that makes it too costly or whatever.
    - oblio 4mo ago
      > I'm simply saying this: there are third party hosters of Open Weight models like deepseek and they have been doing this for a while. > Obviously they are not subsidised, do you disagree? If you agree, they have a way to price it at a point that people wanna pay for it and also they aren't losing money. > So there's nothing inherent about inference that makes it too costly or whatever. Do we have audited GAAP financial data for any of these companies? If we don't, all these are... vibes, man.
      - simianwords 4mo ago
        Good questions. The post you are replying to is also based on vibes. So how about we make a bet and come back after IPO filings? But isn't it okay to suggest that random 3rd party hosting companies are ... not losing money? Why?
        oblio 4mo ago
        I'll bet 50€ that big AI companies (OpenAI, Anthropic, xAI) have bad regarding their core financials. Excluding one time deals such as renting hardware, external cash infusions, the core AI model business (training models and selling inference) is unprofitable and will be be at the expected scales (50 billion € or more) for the next 5 years at least. I bet that within 5 years they will be sold for scrap to bigger companies and will become divisions inside them.
        simianwords 4mo ago
        This is not fair, lets just speak about Inference margins. You are bringing 100 other things to make it more ambgigous. Statement: margins API prices of all models are greater than 10% in Anthropic. Feel free to either agree to what I'm saying or bet otherwise.
        oblio 4mo ago
        1. You can't have a model without training. Training is part of the cost. A model trained 5 years ago is borderline useless. Also models cannot be retrained continuously, we do not have the technology (even if we had it, it would just increase the ongoing operational cost). 2. Rephrasing your statement: At June 2026 prices, profit margins for all Anthropic models are >10%. That claim is super defensive. Serving a model can only be done if the model has been built and trained, can't have it any other way. Building it and training it costs lots of money. Even so, fine, I'll take that bet. Anthropic inference prices are still marginally subsdizided. Once they're public they will hike their API prices several times over the next 24 months. Even that might not save them, because when we take all their expenses into account, they will probably need to raise prices 2-3x compared to their June 2026 prices.

by emil-lp 4mo ago

simianwords 4mo ago

> How much would it cost you to serve, say, 10k simultaneous users with a SOTA model? And if you wanted to go cash positive after a year, how much would each user have to pay? My post has this same argument - we have multiple third party companies running open weight models. They are obviously not subsidised. And people are willing to pay for it. And these models are as good as the SOTA models from last year. So this kinda proves my point that SOTA is sustainable.

emil-lp 4mo ago
I didn't find the answer there, that's why I asked. What hardware is needed, how much of it, cooling, and what does it all cost you? Or are you saying I can take my old desktop and serve Deepseek v3.2 to 10k users simultaneously and it would cost me about $1 per megatoken?
- simianwords 4mo ago
  I'm simply saying this: there are third party hosters of Open Weight models like deepseek and they have been doing this for a while. Obviously they are not subsidised, do you disagree? If you agree, they have a way to price it at a point that people wanna pay for it and also they aren't losing money. So there's nothing inherent about inference that makes it too costly or whatever.
  - oblio 4mo ago
    > I'm simply saying this: there are third party hosters of Open Weight models like deepseek and they have been doing this for a while. > Obviously they are not subsidised, do you disagree? If you agree, they have a way to price it at a point that people wanna pay for it and also they aren't losing money. > So there's nothing inherent about inference that makes it too costly or whatever. Do we have audited GAAP financial data for any of these companies? If we don't, all these are... vibes, man.
    - simianwords 4mo ago
      Good questions. The post you are replying to is also based on vibes. So how about we make a bet and come back after IPO filings? But isn't it okay to suggest that random 3rd party hosting companies are ... not losing money? Why?
      - oblio 4mo ago
        I'll bet 50€ that big AI companies (OpenAI, Anthropic, xAI) have bad regarding their core financials. Excluding one time deals such as renting hardware, external cash infusions, the core AI model business (training models and selling inference) is unprofitable and will be be at the expected scales (50 billion € or more) for the next 5 years at least. I bet that within 5 years they will be sold for scrap to bigger companies and will become divisions inside them.
        simianwords 4mo ago
        This is not fair, lets just speak about Inference margins. You are bringing 100 other things to make it more ambgigous. Statement: margins API prices of all models are greater than 10% in Anthropic. Feel free to either agree to what I'm saying or bet otherwise.
        oblio 4mo ago
        1. You can't have a model without training. Training is part of the cost. A model trained 5 years ago is borderline useless. Also models cannot be retrained continuously, we do not have the technology (even if we had it, it would just increase the ongoing operational cost). 2. Rephrasing your statement: At June 2026 prices, profit margins for all Anthropic models are >10%. That claim is super defensive. Serving a model can only be done if the model has been built and trained, can't have it any other way. Building it and training it costs lots of money. Even so, fine, I'll take that bet. Anthropic inference prices are still marginally subsdizided. Once they're public they will hike their API prices several times over the next 24 months. Even that might not save them, because when we take all their expenses into account, they will probably need to raise prices 2-3x compared to their June 2026 prices.