4 ms·

> Models are not free. Downloading them is free. Running them is not. Is this really different from traditional software? Downloading postgres is free. Running

> Models are not free. Downloading them is free. Running them is not. Is this really different from traditional software? Downloading postgres is free. Running it is not. You either buy hardware and assume the costs of owning and running that, or you pay to run it in the cloud.

aurareturn 3mo ago
I think the point here is that it takes the same hardware to inference an open source model as OpenAI/Anthropic inference their models. IE, a lower param OpenAI/Anthropic model can compete with a higher param open source model. So even if you are an American company who downloaded Chinese models in hopes of saving in cost, you still have to beat OpenAI and Anthropic in $/task which is very tough to do over the long run.
- pishpash 3mo ago
  No they don't. Are OpenAI and Anthropic charging at cost, or wish to? No.
  - aurareturn 3mo ago
    If you have more efficient models, you can reduce price and grab market share or keep price and increase your margins. Over time, this compounds. More profits means more investments. More market share means more control.
- Gigachad 3mo ago
  The problem is these AI companies are running at a loss with the hopes of pumping the prices after everyone is hooked. Now they are locked in at selling API access at rock bottom price. The valuations won't hold.
  - elictronic 3mo ago
    It’s even worse. There operating costs are rising because of their own demand claims while those same rock bottom api prices are locked in. There being squeezed by their own stock pumping and SpaceX pretending to be an AI company is driving down the exit strategy. I’m guessing one starts going full Theranos and begins claiming full AGI or gets the US government to government cheese then hard. It’s going to be a few crazy months.
charlieflowers 3mo ago
But inference costs scale per task whereas platform services like postgres typically amortize across tasks. If you are selling tasks done by inference, then compute is part of your COGS and it scales per task.
- SoftTalker 3mo ago
  Much more so, I would agree. A bit like how mainframe batch jobs might have once been charged back based on "cycles" used for a job or other tracked metrics.

aurareturn 3mo ago

I think the point here is that it takes the same hardware to inference an open source model as OpenAI/Anthropic inference their models. IE, a lower param OpenAI/Anthropic model can compete with a higher param open source model. So even if you are an American company who downloaded Chinese models in hopes of saving in cost, you still have to beat OpenAI and Anthropic in $/task which is very tough to do over the long run.

pishpash 3mo ago
No they don't. Are OpenAI and Anthropic charging at cost, or wish to? No.
- aurareturn 3mo ago
  If you have more efficient models, you can reduce price and grab market share or keep price and increase your margins. Over time, this compounds. More profits means more investments. More market share means more control.
Gigachad 3mo ago
The problem is these AI companies are running at a loss with the hopes of pumping the prices after everyone is hooked. Now they are locked in at selling API access at rock bottom price. The valuations won't hold.
- elictronic 3mo ago
  It’s even worse. There operating costs are rising because of their own demand claims while those same rock bottom api prices are locked in. There being squeezed by their own stock pumping and SpaceX pretending to be an AI company is driving down the exit strategy. I’m guessing one starts going full Theranos and begins claiming full AGI or gets the US government to government cheese then hard. It’s going to be a few crazy months.

charlieflowers 3mo ago

But inference costs scale per task whereas platform services like postgres typically amortize across tasks. If you are selling tasks done by inference, then compute is part of your COGS and it scales per task.

SoftTalker 3mo ago
Much more so, I would agree. A bit like how mainframe batch jobs might have once been charged back based on "cycles" used for a job or other tracked metrics.