3 ms·

Are you guys ready for the bifurcation when the top models are prohibitively expensive to normal users? If your AI budget $2000+ a month? Or are you going to be

Are you guys ready for the bifurcation when the top models are prohibitively expensive to normal users? If your AI budget $2000+ a month? Or are you going to be part of the permanent free tier underclass?

adi_kurian 6mo ago
If one is to believe the API prices are reasonable representation of non subsidized "real world pricing" (with model training being the big exception), then the models are getting cheaper over time. GPT 4.5 was $150.00 / 1M tokens IIRC. GPT o1-pro was $600 / 1M tokens.
- vonneumannstan 6mo ago
  You can check the hardware costs for self hosting a high end open source model and compare that to the tiers available from the big providers. Pretty hard to believe its not massively subsidized. 2 years of Claude Max costs you 2,400. There is no hardware/model combination that gets you close to that price for that level of performance.
  - adi_kurian 6mo ago
    Yes that's why I said API price. I once used the API like I use my subscription and it was an eye watering bill. More than that 2 year price in... a very short amount of time. With no automations/openclaw.
  - lostmsu 6mo ago
    Are you considering batch inference?
OsrsNeedsf2P 6mo ago
Inference for the same results has been dropping 10x year over year[0] [0] https://ziva.sh/blogs/llm-pricing-decline-analysis https://ziva.sh/blogs/llm-pricing-decline-analysis
- ceejayoz 6mo ago
  Sure, but "the same results" will rapidly become unacceptable results if much better results are available.
  - esafak 6mo ago
    Or will they rapidly become indistinguishable since they both get the job done?
  - hibikir 6mo ago
    When we go with any other good in the economy, price is always relevant: After all, the price is a key part of any offering. There are $80-100k workstations out there, but most of us don't buy them, because the extra capabilities just aren't worth it vs, say a $3000 computer, and or even a $500 one. Do I need a top specialist to consult for a stomachache, at $1000 a visit? Definitely not at first. There's a practical difference to how much better certain kinds of results can be. We already see coding harnesses offloading simple things to simpler models because they are accurate enough. Other things dropped straight to normal programs, because they are that much more efficient than letting the LLM do all the things. There will always be problems where money is basically irrelevant, and a model that costs tens of thousand dollars of compute per answer is seen as a great investment, but as long as there's a big price difference, in most questions, price and time to results are key features that cannot be ignored.
  - swader999 6mo ago
    Yes, it will always be an arms race game.
asadm 6mo ago
if it can pay my rent, why not?

adi_kurian 6mo ago

If one is to believe the API prices are reasonable representation of non subsidized "real world pricing" (with model training being the big exception), then the models are getting cheaper over time. GPT 4.5 was $150.00 / 1M tokens IIRC. GPT o1-pro was $600 / 1M tokens.

vonneumannstan 6mo ago
You can check the hardware costs for self hosting a high end open source model and compare that to the tiers available from the big providers. Pretty hard to believe its not massively subsidized. 2 years of Claude Max costs you 2,400. There is no hardware/model combination that gets you close to that price for that level of performance.
- adi_kurian 6mo ago
  Yes that's why I said API price. I once used the API like I use my subscription and it was an eye watering bill. More than that 2 year price in... a very short amount of time. With no automations/openclaw.
- lostmsu 6mo ago
  Are you considering batch inference?

OsrsNeedsf2P 6mo ago

Inference for the same results has been dropping 10x year over year[0] [0] https://ziva.sh/blogs/llm-pricing-decline-analysis https://ziva.sh/blogs/llm-pricing-decline-analysis

ceejayoz 6mo ago
Sure, but "the same results" will rapidly become unacceptable results if much better results are available.
- esafak 6mo ago
  Or will they rapidly become indistinguishable since they both get the job done?
- hibikir 6mo ago
  When we go with any other good in the economy, price is always relevant: After all, the price is a key part of any offering. There are $80-100k workstations out there, but most of us don't buy them, because the extra capabilities just aren't worth it vs, say a $3000 computer, and or even a $500 one. Do I need a top specialist to consult for a stomachache, at $1000 a visit? Definitely not at first. There's a practical difference to how much better certain kinds of results can be. We already see coding harnesses offloading simple things to simpler models because they are accurate enough. Other things dropped straight to normal programs, because they are that much more efficient than letting the LLM do all the things. There will always be problems where money is basically irrelevant, and a model that costs tens of thousand dollars of compute per answer is seen as a great investment, but as long as there's a big price difference, in most questions, price and time to results are key features that cannot be ignored.
- swader999 6mo ago
  Yes, it will always be an arms race game.

asadm 6mo ago

if it can pay my rent, why not?