4 ms·
But doesn't $15M/day of inference cost imply "demand" from users? If this is the case, it's just a matter of time until costs can be reduced.
by santiagobasulto 7mo ago
But doesn't $15M/day of inference cost imply "demand" from users? If this is the case, it's just a matter of time until costs can be reduced.
- abuani 7mo agoCosts were reduced to $0. Can't get better then that for a product that OpenAI had no clue how to monetize
- newsclues 7mo agoI’ll eat lots of free samples at Costco of foods I’d never pay money for
- bonesss 7mo agoA box that says "Free T-Shirts" can create near-infinite demand. Paying near-infinity dollars for T-Shirts people want for $0 isn't a profitable business model. Demand side price sensitivity impacts potential supply side margins.
- vinni2 7mo agoIt could be over provisioned or that cost is supposed to be minimum cost with some minimum capacity which was never reached.
- _qib3 7mo agoWhy would demand imply costs will be reduced? If you're making an economies of scale argument, there's plenty scale right now, and costs don't seem to be trending down.
- MrGilbert 7mo ago> If this is the case, it's just a matter of time until costs can be reduced. Is it, though? We cannot predict technological advancement, and the times of ̶M̶u̶r̶p̶h̶y̶'̶s̶ ̶L̶a̶w̶ Moore's Law* for computational power are long gone. There is simply no guarantee that the costs will go down enough. * thanks lucianbr!
- kukkamario 7mo agoI think there is plenty of room to make AI inference much more energy efficient. For example, there are companies testing creating custom silicon to run the model. Once that technology matures and we have some "good enough" models for normal use, inference cost for non-bleeding-edge models can come way down. I don't expect bleeding-edge models to become any cheaper, but previous generation models can potentially be really cheap.
- deleted 7mo ago[deleted]
- lucianbr 7mo agoMoore's Law. The times for Murphy's Law for computational power are just beginning.
- PunchyHamster 7mo agoThe "matter of time" is getting more and more expensive, not cheaper, at least for next 2 years
- brazzy 7mo agoNot sure what you're referring to. If you're talking about inference cost for frontier models, that's going up because researchers keep pushing those frontiers, often without considering cost. And while they're subsidized (to gain market share), users have no reason NOT to use the crazy expensive frontier models. Once the market consolidates, and users get used to the idea of using models that are "good enough" because frontier models are too expensive, there's no reason AI cannot be profitable.
- zarzavat 7mo agoThere's not much profit in inference, it's heavily commoditized. There is an illusion of potential profitability because the closed-weight models are currently a step ahead of the open-weight models. However, if you ignore the closed-weight models, then the open-weight models are also getting better every year. In the limit, the open-weight models will end up just as good as the closed-weight models. AI is an inverse gold rush, the people who are getting rich off it are the people using it. The shovel-sellers are screwed.
- PunchyHamster 7mo ago> Not sure what you're referring to. If you're talking about inference cost for frontier models, that's going up because researchers keep pushing those frontiers, often without considering cost. And while they're subsidized (to gain market share), users have no reason NOT to use the crazy expensive frontier models. The price of GPUs and the price of RAM to put in the servers.
- written-beyond 7mo agoYes but it was capped by their restriction on sign ups/registrations. That could've easily been in the hundreds of millions if the app had public signups. Idk if Instagram would exist if they were spending hundreds of millions a day.
- vinodpandey7 7mo ago[flagged]
- yifanl 7mo agoDisneyworld has lines longer than the park can manage for decades, do you expect it to just be a matter of time until park management finally figures out how to queue people efficiently enough, or do you think the solution will be once again raising costs for the customer.
- pjc50 7mo agoThe actual revenue was quoted at $2.1m .. total. Ever. It would require multiple order of magnitude cost reductions to make that worthwhile. Maybe another few decades of Moore's law, if we have that left. This was the Moviepass model of selling $10 bills for $9.
- rsynnott 7mo agoMuch worse, really; it was selling $10 bills for half a cent. The Moviepass thing, I think if you were kinda gullible you could maybe buy into it eventually working on scale. This could never work on scale.
- hk__2 7mo ago> But doesn't $15M/day of inference cost imply "demand" from users? If this is the case, it's just a matter of time until costs can be reduced. If you build a website that gives $100 for free to each one of your users, you’ll quickly have "demand" but that’s not "a matter of time until costs can be reduced".
- imron 7mo agoMake a loss on every sale but make it up in volume!
- rsynnott 7mo agoCosts would have to be reduced about 2,000 times just to break even, assuming that inference was the only cost, which of course it was not.
- mingus88 7mo agoMany things are only in demand because they are free. And if nobody is willing to pay for it, it hardly matters how low you bring down cost, because it’s always a net negative. Just look at the general state of the internet over the past two decades. Do you think it would work for Sora to insert ads into the slop?
- array4277 7mo agoHa, that's a good one!