5 ms·
> If OpenAI or Anthropic could squeeze the same output out of smaller GPUs and servers they'd be doing it for themselves. First, they do this; that's why they
by Majromax 11mo ago
> If OpenAI or Anthropic could squeeze the same output out of smaller GPUs and servers they'd be doing it for themselves.
First, they do this; that's why they release models at different price points. It's also why GPT-5 tries auto-routing requests to the most cost-effective model.
Second, be careful about considering the incentives of these companies. They all act as if they're in an existential race to deliver 'the' best model; the winner-take-all model justifies their collective trillion dollar-ish valuation. In that race, delivering 97% of the performance at 10% of the cost is a distraction.
- cubefox 11mo ago> > If OpenAI or Anthropic could squeeze the same output out of smaller GPUs and servers they'd be doing it for themselves. > First, they do this; that's why they release models at different price points. No, those don't deliver the same output. The cheaper models are worse. > It's also why GPT-5 tries auto-routing requests to the most cost-effective model. These are likely the same size, just one uses reasoning and the other doesn't. Not using reasoning is cheaper, but not because the model is smaller.
- gunalx 11mo agoBut they also squesed a 80% cut in O3 at some point, supposedly purely on inference or infra optimization
- anabis 11mo ago> delivering 97% of the performance at 10% of the cost is a distraction. Not if you are running RL on that model, and need to do many roll-outs.