4 ms·

> The demand for AI is currently overwhelming. Wait until they charge the real pice, if I sold a dollar for 10ct I'd also have a lot of demand. I'm burning bi

> The demand for AI is currently overwhelming. Wait until they charge the real pice, if I sold a dollar for 10ct I'd also have a lot of demand. I'm burning billions of tokens on chatgpt "deepresearch Pro extended" for things I wouldn't even bother googling, the second I have to pay even 2x the price I won't use that anymore

vanuatu 5mo ago
I hear this analogy (selling a dollar for 10ct) but it's unclear to me how we can cleanly map intelligence to cents. If the LLM was GPT-1, most people wouldn't even use it for free. So clearly there's another axis here?
- Micrococonut 5mo ago
  The analogy is implying that the revenue generated by providers is dwarfed by the total expenditure on inference & continuously training the next best model. These providers have large operational costs and the presumption is that they are providing a dollar ~worth~ of product for 10 cents. Worth being calculable based on the actual capital & operational costs of providing the service.
  - vanuatu 5mo ago
    anthropic models are profitable fully loaded (rev - inference cost - training cost)
    - Micrococonut 5mo ago
      Yea I don't know if that's true or not. I'm just saying that is what they are getting at.
    - toasty228 5mo ago
      According to who? Remember that they told us gpt 3 was "too dangerous to release" Until their finances are open you can't trust anything they say
ls612 5mo ago
The estimates I've seen are that running inference at scale on a Deepseek V3 sized model (so 700B parameters) costs roughly $0.70/mtok or so given current H100 rental costs. Sonnet charges $15/mtok on the API so the delta between the true cost and the API cost is quite large, to the point where even many subscription users are likely profitable.
hootz 5mo ago
Can't that be countered by the fact that you can pay a reasonable price (something like 20 or 30 bucks) for small businesses independent flat-rate inference subscriptions of models like GLM-5.1? They aren't being subsidized, they just balance normal and power users around their flat rate. Just check something like synthetic.new, Ollama Cloud or OpenCode Go.

vanuatu 5mo ago

I hear this analogy (selling a dollar for 10ct) but it's unclear to me how we can cleanly map intelligence to cents. If the LLM was GPT-1, most people wouldn't even use it for free. So clearly there's another axis here?

Micrococonut 5mo ago
The analogy is implying that the revenue generated by providers is dwarfed by the total expenditure on inference & continuously training the next best model. These providers have large operational costs and the presumption is that they are providing a dollar ~worth~ of product for 10 cents. Worth being calculable based on the actual capital & operational costs of providing the service.
- vanuatu 5mo ago
  anthropic models are profitable fully loaded (rev - inference cost - training cost)
  - Micrococonut 5mo ago
    Yea I don't know if that's true or not. I'm just saying that is what they are getting at.
  - toasty228 5mo ago
    According to who? Remember that they told us gpt 3 was "too dangerous to release" Until their finances are open you can't trust anything they say

ls612 5mo ago

The estimates I've seen are that running inference at scale on a Deepseek V3 sized model (so 700B parameters) costs roughly $0.70/mtok or so given current H100 rental costs. Sonnet charges $15/mtok on the API so the delta between the true cost and the API cost is quite large, to the point where even many subscription users are likely profitable.

hootz 5mo ago

Can't that be countered by the fact that you can pay a reasonable price (something like 20 or 30 bucks) for small businesses independent flat-rate inference subscriptions of models like GLM-5.1? They aren't being subsidized, they just balance normal and power users around their flat rate. Just check something like synthetic.new, Ollama Cloud or OpenCode Go.