4 ms·

The tech is way underpriced right now. It's basically a subsidized market right now, with the money flowing in coming from the private sector. The problem here

by hvb2 11mo ago

The tech is way underpriced right now. It's basically a subsidized market right now, with the money flowing in coming from the private sector. The problem here is that it remains to be seen who is willing to pay for the service once it's priced at cost or even with a margin. And based on valuations of AI companies one would expect a huge margin.

causal 11mo ago
I wonder if a price correction would be a boon for open source, with the economics of smaller / self hosted models making a lot more sense when API prices have to surge.
- refulgentis 11mo ago
  It's not actually subsidized and the economics of smaller/self-hosted models are a much, much, worse nightmare (source: guy who spent last 2 years maintaining llama.cpp && any provider you can think of) (why is it bad? same reason why 20 cars vs. 1 bus is bad. same reason why only being able to use transportation if you own a car would be bad)
  - causal 11mo ago
    > It's not actually subsidized Source?
    - anoncareer0212 11mo ago
      Source on it being subsidized? :) (there isn't one, other than an aggro subset of people lying to eachother that somehow literally everyone is losing money, while posting record profit margins) (https://en.wikipedia.org/wiki/Hitchens%27s_razor https://en.wikipedia.org/wiki/Hitchens%27s_razor)
      - causal 11mo ago
        If it's not profitable, it's running on capital. Subsidized.
AnimalMuppet 11mo ago
A huge margin or a huge market at a moderate margin. But yes, the net profit has to be huge.
ForHackernews 11mo ago
It's hard for me to imagine paying real money for something that gives me a maybe-hallucinated answer that I need to check every single time. A flaky test is worse than a failing test.
- danielbln 11mo ago
  _sigh_ Yes LLMs hallucinate, no it's no longer 2022 and ChatGPT (gpt-3.5) is the pinnacle of LLM tech. Modern LLMs in an agentic loop can self correct, you still need to be on guard but if used correctly (yes, yes, holding it wrong etc. etc.) can do many, many tasks that do not suffer from "need to check every single time".
  - chmod775 11mo ago
    I must be holding it wrong then, because in my ChatGPT history I've abandoned 2/3rds of my conversations recently because it wasn't coming up with anything useful. Granted, most of that was debugging some rather complicated typescript types in a custom JSX namespace, which would probably be considered hard even for most humans as well as there being comparatively few resources on it to be found online, but the issue is that overall it wasted more of my time than it saved with its confidently wrong answers. When I look at my history I don't see anything that would be worth twenty bucks - what I see makes me think that I should be the one getting paid.
  - dwa3592 11mo ago
    really? >>many tasks that do not suffer from "need to check every single time" like which tasks? How do you decide whether you need to check or not? If you're asking it to complete 100 sequences, and if the error rate is 5%, which 5% of the sequences do you think it messed up or _thought_ otherwise? if the 5% is in the middle, would the next 50 sequences be okay?
    - palmotea 11mo ago
      > really? >>many tasks that do not suffer from "need to check every single time" > like which tasks? Making slop.
mistrial9 11mo ago
> it remains to be seen who is forced to pay via govt relationships, long term irreplaceable services, debt or convictions.. Also don't forget the surveillance budgets and the best spigots there, win.
energy123 11mo ago
You're saying the unit economics are bad?
refulgentis 11mo ago
It's not subsidized, lol. Generally, I worry HN is in a dark place with this stuff - look how this thread goes, ex. descendant of yours is at "Why would I ever pay for this when it hallucinates." I don't understand how you can be a software engineer and afford to have opinions like that. I'm worried for those who do, genuinely, I hope transitions out there are slow enough, due to obstinance, that they're not cast out suddenly without the skills to get something else.
- philipwhiuk 11mo ago
  > It's not subsidized, lol. It's subsidised by VC funding. At some point the gravy train stops and they have to pivot to profit so that the VCs deliver return-on-investment. Look at Facebook shoving in adverts, Uber jacking up the price, etc. > I don't understand how you can be a software engineer and afford to have opinions like that I don't know how you can afford not to realise that there's a fixed value prop here for the current behaviour and that it's potentially not as high as it needs to be for OpenAI to turn a profit. OpenAI's ridiculous investment ability is based on a future potential it probably will never hit. Assuming it does not, the whole stack of cards falls down real quick. (You can Ctrl-C/Ctrl-V OpenAI for all the big AI providers)
  - anoncareer0212 11mo ago
    This is all about OpenAI, not about AI being subsidized...with some sort of directive to copy/paste "OpenAI" for all the big AI providers? (presumably you meant s/OpenAI/$PROVIDER?) If that's what you meant: Google. Boom. Also, perhaps you're a bit new to industry, but that's how these things go. They burn a lot of capital building it out b/c they can always fire everyone and just serve at cost -- i.e. subsidizing business development is different from subsiziding inference, unless you're just sort of confused and angry at the whole situation and it all collapses into everyone's losing money and no one will admit it.
    - hvb2 11mo ago
      You're replying to a story about a hyperscaler worrying investors about how much they're leveraging themselves for a small number of companies. From the article: > OpenAI faces questions about how it plans to meet its commitments to spend $1.4tn on AI infrastructure over the next eight years. Someone needs to pay for that 1.4 trillion, that's 2/3 of what Microsoft makes this year. If you think they'll make that from revenue, that's fine. I don't. And that's just the infra.
conartist6 11mo ago
And really the reason that it would be like that is that the models don't learn, per se, within their lifetime. I'm told that each model is cashflow positive over its lifetime, which suggests that if the companies could just stop training new models the money would come raining down. If they have to keep training new models though to keep pace with the changes in the world though then token costs would be only maybe 30% electricity and 70% model depreciation -- i.e. the costs of training the next generation of model so that model users don't become stranded 10 years in the past.

by hvb2 11mo ago

causal 11mo ago

I wonder if a price correction would be a boon for open source, with the economics of smaller / self hosted models making a lot more sense when API prices have to surge.

refulgentis 11mo ago
It's not actually subsidized and the economics of smaller/self-hosted models are a much, much, worse nightmare (source: guy who spent last 2 years maintaining llama.cpp && any provider you can think of) (why is it bad? same reason why 20 cars vs. 1 bus is bad. same reason why only being able to use transportation if you own a car would be bad)
- causal 11mo ago
  > It's not actually subsidized Source?
  - anoncareer0212 11mo ago
    Source on it being subsidized? :) (there isn't one, other than an aggro subset of people lying to eachother that somehow literally everyone is losing money, while posting record profit margins) (https://en.wikipedia.org/wiki/Hitchens%27s_razor https://en.wikipedia.org/wiki/Hitchens%27s_razor)
    - causal 11mo ago
      If it's not profitable, it's running on capital. Subsidized.

AnimalMuppet 11mo ago

A huge margin or a huge market at a moderate margin. But yes, the net profit has to be huge.

ForHackernews 11mo ago

It's hard for me to imagine paying real money for something that gives me a maybe-hallucinated answer that I need to check every single time. A flaky test is worse than a failing test.

danielbln 11mo ago
_sigh_ Yes LLMs hallucinate, no it's no longer 2022 and ChatGPT (gpt-3.5) is the pinnacle of LLM tech. Modern LLMs in an agentic loop can self correct, you still need to be on guard but if used correctly (yes, yes, holding it wrong etc. etc.) can do many, many tasks that do not suffer from "need to check every single time".
- chmod775 11mo ago
  I must be holding it wrong then, because in my ChatGPT history I've abandoned 2/3rds of my conversations recently because it wasn't coming up with anything useful. Granted, most of that was debugging some rather complicated typescript types in a custom JSX namespace, which would probably be considered hard even for most humans as well as there being comparatively few resources on it to be found online, but the issue is that overall it wasted more of my time than it saved with its confidently wrong answers. When I look at my history I don't see anything that would be worth twenty bucks - what I see makes me think that I should be the one getting paid.
- dwa3592 11mo ago
  really? >>many tasks that do not suffer from "need to check every single time" like which tasks? How do you decide whether you need to check or not? If you're asking it to complete 100 sequences, and if the error rate is 5%, which 5% of the sequences do you think it messed up or _thought_ otherwise? if the 5% is in the middle, would the next 50 sequences be okay?
  - palmotea 11mo ago
    > really? >>many tasks that do not suffer from "need to check every single time" > like which tasks? Making slop.

mistrial9 11mo ago

> it remains to be seen who is forced to pay via govt relationships, long term irreplaceable services, debt or convictions.. Also don't forget the surveillance budgets and the best spigots there, win.

energy123 11mo ago

You're saying the unit economics are bad?

refulgentis 11mo ago

It's not subsidized, lol. Generally, I worry HN is in a dark place with this stuff - look how this thread goes, ex. descendant of yours is at "Why would I ever pay for this when it hallucinates." I don't understand how you can be a software engineer and afford to have opinions like that. I'm worried for those who do, genuinely, I hope transitions out there are slow enough, due to obstinance, that they're not cast out suddenly without the skills to get something else.

philipwhiuk 11mo ago
> It's not subsidized, lol. It's subsidised by VC funding. At some point the gravy train stops and they have to pivot to profit so that the VCs deliver return-on-investment. Look at Facebook shoving in adverts, Uber jacking up the price, etc. > I don't understand how you can be a software engineer and afford to have opinions like that I don't know how you can afford not to realise that there's a fixed value prop here for the current behaviour and that it's potentially not as high as it needs to be for OpenAI to turn a profit. OpenAI's ridiculous investment ability is based on a future potential it probably will never hit. Assuming it does not, the whole stack of cards falls down real quick. (You can Ctrl-C/Ctrl-V OpenAI for all the big AI providers)
- anoncareer0212 11mo ago
  This is all about OpenAI, not about AI being subsidized...with some sort of directive to copy/paste "OpenAI" for all the big AI providers? (presumably you meant s/OpenAI/$PROVIDER?) If that's what you meant: Google. Boom. Also, perhaps you're a bit new to industry, but that's how these things go. They burn a lot of capital building it out b/c they can always fire everyone and just serve at cost -- i.e. subsidizing business development is different from subsiziding inference, unless you're just sort of confused and angry at the whole situation and it all collapses into everyone's losing money and no one will admit it.
  - hvb2 11mo ago
    You're replying to a story about a hyperscaler worrying investors about how much they're leveraging themselves for a small number of companies. From the article: > OpenAI faces questions about how it plans to meet its commitments to spend $1.4tn on AI infrastructure over the next eight years. Someone needs to pay for that 1.4 trillion, that's 2/3 of what Microsoft makes this year. If you think they'll make that from revenue, that's fine. I don't. And that's just the infra.

conartist6 11mo ago

And really the reason that it would be like that is that the models don't learn, per se, within their lifetime. I'm told that each model is cashflow positive over its lifetime, which suggests that if the companies could just stop training new models the money would come raining down. If they have to keep training new models though to keep pace with the changes in the world though then token costs would be only maybe 30% electricity and 70% model depreciation -- i.e. the costs of training the next generation of model so that model users don't become stranded 10 years in the past.