3 ms·

> On many task lengths (including those near their plateau) they cost 10 to 100 times as much per hour. For instance, Grok 4 is at $0.40 per hour at its sweet s

> On many task lengths (including those near their plateau) they cost 10 to 100 times as much per hour. For instance, Grok 4 is at $0.40 per hour at its sweet spot, but $13 per hour at the start of its final plateau. GPT-5 is about $13 per hour for tasks that take about 45 minutes, but $120 per hour for tasks that take 2 hours. And o3 actually costs $350 per hour (more than the human price) to achieve tasks at its full 1.5 hour task horizon. This is a lot of money to pay for an agent that fails at the task you’ve just paid for 50% of the time — especially in cases where failure is much worse than not having tried at all.

boxedemp 6mo ago
If you gave me an agent that succeeded 50% of tasks I gave it, I could take over the world in a week. Faster if I wasn't so lazy. I think you're overestimating, or oversimplifying. Maybe both.
- raincole 6mo ago
  No one is claiming an agent can do 50% of arbitrary tasks. It's just 50% of METR's benchmark set. > I think you're overestimating, or oversimplifying Yeah if you only read comments on HN but not the actual linked article you will get oversimplified conclusion. Like, duh?
  - boxedemp 6mo ago
    Sorry for stating something so obvious. I'll comment less from now on.
  - TeMPOraL 6mo ago
    > Yeah if you only read comments on HN but not the actual linked article you will get oversimplified conclusion. Like, duh? Curiously, for most submissions it's the opposite - comments are much more useful and nuanced than the source being discussed.
- jurgenburgen 6mo ago
  > If you gave me an agent that succeeded 50% of tasks I gave it, I could take over the world in a week. Faster if I wasn't so lazy. Assuming you used o3, that would cost $58800 per week. That’s an expensive bet for only 50% odds in your favor. Of course the agents are only that good on benchmarks, in reality your odds are worse. Maybe roulette instead?
nopinsight 6mo ago
Ord's frontier-cost argument is right as far as it goes, but the piece doesn't engage with the counter-trend: inference cost for a fixed capability level has been falling faster than Moore's law. Pushing the frontier will likely keep getting more expensive and concentrated among a few players, while the intelligence needed for more mundane tasks keeps getting cheaper. That raises a question: if practical-tier inference commoditizes, how does any company justify the ever-larger capex to push the frontier? OpenAI's pitch is that their business model should "scale with the value intelligence delivers." Concretely, that means moving beyond API fees into licensing and outcome-based pricing in high-value R&D sectors like drug discovery and materials science, where a single breakthrough dwarfs compute cost. That's one possible answer, though it's unclear whether the mechanism will work in practice.
- popcorncowboy 6mo ago
  > how does any company justify the ever-larger capex to push the frontier AGI. [waves hands at the infinite money machine]
zozbot234 6mo ago
This effect is likely even larger when you consider that the raw cost per inferred token grows linearly with context, rather than being constant. So longer tasks performed with higher-context models will cost quadratically more. The computational cost also grows super-linearly with model parameter size: a 20B-active model is more than four times the cost of a 5B-active model.
- tibbar 6mo ago
  Doesn't context cacheing mostly eliminate this problem? (I suppose for enough context the 90% discount is eventually a lot anyway)
  - zozbot234 6mo ago
    Context caching is really storing the KV-cache for reuse. It saves running prefill for that part of the context, but tokens referencing that KV-cache will still cost more.
naveen99 6mo ago
Where are you getting hourly costs for private models ? The rate limits are pretty arbitrary. If you max out by api tokens it would be like $10k / hour

boxedemp 6mo ago

If you gave me an agent that succeeded 50% of tasks I gave it, I could take over the world in a week. Faster if I wasn't so lazy. I think you're overestimating, or oversimplifying. Maybe both.

raincole 6mo ago
No one is claiming an agent can do 50% of arbitrary tasks. It's just 50% of METR's benchmark set. > I think you're overestimating, or oversimplifying Yeah if you only read comments on HN but not the actual linked article you will get oversimplified conclusion. Like, duh?
- boxedemp 6mo ago
  Sorry for stating something so obvious. I'll comment less from now on.
- TeMPOraL 6mo ago
  > Yeah if you only read comments on HN but not the actual linked article you will get oversimplified conclusion. Like, duh? Curiously, for most submissions it's the opposite - comments are much more useful and nuanced than the source being discussed.
jurgenburgen 6mo ago
> If you gave me an agent that succeeded 50% of tasks I gave it, I could take over the world in a week. Faster if I wasn't so lazy. Assuming you used o3, that would cost $58800 per week. That’s an expensive bet for only 50% odds in your favor. Of course the agents are only that good on benchmarks, in reality your odds are worse. Maybe roulette instead?

nopinsight 6mo ago

Ord's frontier-cost argument is right as far as it goes, but the piece doesn't engage with the counter-trend: inference cost for a fixed capability level has been falling faster than Moore's law. Pushing the frontier will likely keep getting more expensive and concentrated among a few players, while the intelligence needed for more mundane tasks keeps getting cheaper. That raises a question: if practical-tier inference commoditizes, how does any company justify the ever-larger capex to push the frontier? OpenAI's pitch is that their business model should "scale with the value intelligence delivers." Concretely, that means moving beyond API fees into licensing and outcome-based pricing in high-value R&D sectors like drug discovery and materials science, where a single breakthrough dwarfs compute cost. That's one possible answer, though it's unclear whether the mechanism will work in practice.

popcorncowboy 6mo ago
> how does any company justify the ever-larger capex to push the frontier AGI. [waves hands at the infinite money machine]

zozbot234 6mo ago

This effect is likely even larger when you consider that the raw cost per inferred token grows linearly with context, rather than being constant. So longer tasks performed with higher-context models will cost quadratically more. The computational cost also grows super-linearly with model parameter size: a 20B-active model is more than four times the cost of a 5B-active model.

tibbar 6mo ago
Doesn't context cacheing mostly eliminate this problem? (I suppose for enough context the 90% discount is eventually a lot anyway)
- zozbot234 6mo ago
  Context caching is really storing the KV-cache for reuse. It saves running prefill for that part of the context, but tokens referencing that KV-cache will still cost more.

naveen99 6mo ago

Where are you getting hourly costs for private models ? The rate limits are pretty arbitrary. If you max out by api tokens it would be like $10k / hour