6 ms·

It's nice that people are genuinely curious about this. - All of your observations are absolutely dead on - Yet, we have very very very robust scaling laws th

It's nice that people are genuinely curious about this. - All of your observations are absolutely dead on - Yet, we have very very very robust scaling laws that as Dario points out we've had and validated for over a decade. This extends to downstream measures like METR time horizon and compsosite benchmarks like the epoch capability index. - If you look at where you're at now, which is again dead on, you're looking at a point on a curve that is quite easy to extrapolate, but less easy to tell when exactly on the curve a certain capability or use case undergoes a step change from error rates dropping below a threshold that is hard to anticipate in advance. So while Dario / other frontier CEOs are understandably unpalatable, they are absolutely spot on with a call out that all of this is bound to happen and happen quickly, and that's without solving several core problems that haven't been solved yet (e.g. continual learning). In 2023, coding agents were just laughable. Yet they followed the same predictable training curves. Anyone looking at the data can see the obvious, and anyone reading newspaper headlines or hacker news comments would get a very different impression.

oudlys 4mo ago
Are we plotting against cost? How is the capability advancement vs dollars paid for development? By my read of the (very sparse) data, we're getting linear improvements in capability for super-linear increases in costs. [1] Indicates that by 2027 models will cost $1 billon to train. Dario estimates that model runs will cost $10 billion in 2026 [2]. That to me indicates costs are potentially growing faster than capability. Maybe by quite a bit. If the value prop of LLMs doesn't prove out, that won't last. I'm of the opinion there is no data that shows actual economic value being delivered by models. The best data shows that LLM use might be destroying value [3]. [1] https://epoch.ai/publications/how-much-does-it-cost-to-train-frontier-ai-models https://epoch.ai/publications/how-much-does-it-cost-to-train... [2] https://lexfridman.com/dario-amodei-transcript/ https://lexfridman.com/dario-amodei-transcript/ [3] https://unessays.substack.com/p/talk-is-cheap https://unessays.substack.com/p/talk-is-cheap
- simianwords 4mo ago
  >By my read of the (very sparse) data, we're getting linear improvements in capability for super-linear increases in costs. [1] Indicates that by 2027 models will cost $1 billon to train. Dario estimates that model runs will cost $10 billion in 2026 [2]. That to me indicates costs are potentially growing faster than capability. Maybe by quite a bit. This is true and well established. As long as you get any improvement whatsoever, it is worth spending to train since it pays off during. Imagine training was not $1 billion but $100 billion but the performance improved by just 10%. This is still worth it because you can squeeze out the profits across years and years right? The improvement is ever lasting. > The best data shows that LLM use might be destroying value [3]. This is basically a conspiracy theory and if you really believed this, you should not have led with "How is the capability advancement vs dollars paid for development?" because if there were no value, it doesn't really matter how much you invest.
  - oudlys 4mo ago
    >This is basically a conspiracy theory I think this is pretty uncharitable, especially when I've provided you with a dataset you can evaluate yourself and an argument you can review for logical inconsistency. I have worked quite hard to locate data that supports your thesis, I can't find it. I've at least gone to the effort of documenting that search. Before you throw around such strong convictions, I suggest you actually look for yourself.
    - simianwords 4mo ago
      Respectfully, your link is not very convincing. But what’s interesting is that you are commenting on a post where Dario is suggesting that LLMs are so extremely powerful that they can take over, help synthesise bioweapons, help in warfare, help in drug discovery — the whole post here is to try and regulate this. If you believe AI can’t even create positive value let alone discover new things then your problem is somewhere else and not in something like “but training costs a lot”. So it is absolutely strange and contrasting to see you believe that LLMs are so weak as to create negative value while the CEO is asking about regulations because AI is too powerful. I don’t think I can convince you that AI is actually that powerful. But let me ask you something directly: if you believe what you believe, you should also acknowledge that AI doesn’t need regulations in the context Dario is proposing since obviously AI can’t do anything he predicts. Do you agree?
- aspenmartin 4mo ago
  I appreciate the data here but I don't think the read is quite right; Saying we have linear capability for super-linear cost compares an unbounded variable (dollars) to bounded instruments (because benchmarks saturate). On unbounded measures, growth is exponential; you can see METR time horizons double every ~4-7 months (https://metr.org/blog/2026-1-29-time-horizon-1-1/ https://metr.org/blog/2026-1-29-time-horizon-1-1/). And capability being proportional to log(compute) is what the scaling law predicts. Epoch puts training cost growth at ~2.4x/year as your link shows. Meanwhile cost for fixed capability falls ~10-40x/year (https://epoch.ai/data-insights/llm-inference-price-trends https://epoch.ai/data-insights/llm-inference-price-trends), and lab revenue is growing ~10x/year! Anthropic went from $1B to $9B to $30B+ run rate in ~15 months, OpenAI ~$25B. On [3]: the "destroying value" conclusion flips sign on an assumed 15% baseline rework rate. The report's most direct metric is +16% merged PRs per dev. The RCT evidence is genuinely mixed (METR: -19%, with n = 20 and Claude 3.x; Cui et al: +26%) but its just super hard to do this well, I think Faros stuff was pretty cool, I haven't seen this before so thank you for the reference.
  - balefulboy 4mo ago
    METR's time horizon is not a reliable metric of LLM capability growth: https://www.transformernews.ai/p/against-the-metr-graph-coding-capabilities-software-jobs-task-ai https://www.transformernews.ai/p/against-the-metr-graph-codi...
    - oudlys 4mo ago
      Wow. This deserves to be much more widely read. Thank you for this.
    - aspenmartin 4mo ago
      Yes I've seen this before, and while the critiques are fair and high quality (and unfortunately not unique to METR) we're missing the forest for the trees here. First of all, if you take the articles critiques and work out the implications on the METR graph, all you're doing is shifting the curve up or down, it doesn't change the fact that progress is scaling exponentially. While it is technically possible the universe could be throwing a massive pathological curveball to change the conclusion from METR data (which is we've been seeing exponential growth over the last 6 years), I think that seems very far from likely. The fact that we see the same behavior from a variety of sources over a wide variety of tasks and domains is a pretty clear indication that METR while certainly far from perfect is actually painting a consistent picture at least in terms of the rate of progress. You can look at ECI for a summary benchmark statistic, which does NOT use METR's benchmark, and you see a similar trend. Same with SWE-bench where the task distribution is far more in domain for real world problems. It is a bummer that this METR data can't be better funded. It would probably take $1M or so to really beef it up properly which any of these labs probably have in their couch cushions.
snaking0776 4mo ago
That’s interesting. I commented something about this elsewhere but to me part of the exponential argument that loses me though is that it can often seem like a way to distract from issues that already exist which we should be working to fix. Things like autonomous weapons or mass surveillance are already here and rather terrifying and I would hope that we would dedicate our time to fixing those rather than having industry leaders focus so much on hypotheticals. While I guess the hypothetical scenario could be so bad that we must focus on it, I imagine a world which can’t come up with a way to spread wealth more equally or prevent mass proliferation of surveillance technology through profit seeking behavior will not be able to handle a digital super intelligence. So I keep coming back to the question: why is all I hear these industry leaders talking about is the threat of extinction? Maybe it’s just news coverage but I would love to see a leading lab release research on the health effects of subaudible sound in datacenters or other immediately present issues which would build good will towards these further out concerns.
- hollerith 4mo ago
  >why is all I hear these industry leaders talking about is the threat of extinction? . . . I would love to see a leading lab release research on the health effects of subaudible sound in datacenters It is straightforward for industry leaders to avoid living near data centers, but there's no way for them to insulate themselves from the extinction threat -- no way short of somehow eliminating the danger for everybody, which seems quite hard to do. Since industry leaders are as self-centered as everyone else, the extinction threat is what they think about. Also, you describe the extinction threat as "further out". A lot of us think there is already some small amount of AI extinction risk being incurred every day. I.e., we think the period of danger has already begun.
  - snaking0776 4mo ago
    I see. I wonder how this works out in terms of risk/reward. I suppose if you take extinction as -infinite cost than it would be the only issue worth thinking about. Where I think this line of thinking gets challenging is when you need to take in terms of a counter factual. A lot of these were already risks prior to AI (bioweapons, nukes, etc) so what’s the marginal increase in probability as a result of AI I guess is the question which matters. I could get more around this way of framing it than saying that AI itself is the problem. It’s just the being more capable as a species increases risks. I think a lot of these pushback comes from the fact that it’s often the CEO who stands to gain huge by saying his tool is going to end the world so we need public buyin to supporting it. If instead it was just framed as “general technological advancement” is dangerous but potentially worthwhile I think more would be on board.

oudlys 4mo ago

Are we plotting against cost? How is the capability advancement vs dollars paid for development? By my read of the (very sparse) data, we're getting linear improvements in capability for super-linear increases in costs. [1] Indicates that by 2027 models will cost $1 billon to train. Dario estimates that model runs will cost $10 billion in 2026 [2]. That to me indicates costs are potentially growing faster than capability. Maybe by quite a bit. If the value prop of LLMs doesn't prove out, that won't last. I'm of the opinion there is no data that shows actual economic value being delivered by models. The best data shows that LLM use might be destroying value [3]. [1] https://epoch.ai/publications/how-much-does-it-cost-to-train-frontier-ai-models https://epoch.ai/publications/how-much-does-it-cost-to-train... [2] https://lexfridman.com/dario-amodei-transcript/ https://lexfridman.com/dario-amodei-transcript/ [3] https://unessays.substack.com/p/talk-is-cheap https://unessays.substack.com/p/talk-is-cheap

simianwords 4mo ago
>By my read of the (very sparse) data, we're getting linear improvements in capability for super-linear increases in costs. [1] Indicates that by 2027 models will cost $1 billon to train. Dario estimates that model runs will cost $10 billion in 2026 [2]. That to me indicates costs are potentially growing faster than capability. Maybe by quite a bit. This is true and well established. As long as you get any improvement whatsoever, it is worth spending to train since it pays off during. Imagine training was not $1 billion but $100 billion but the performance improved by just 10%. This is still worth it because you can squeeze out the profits across years and years right? The improvement is ever lasting. > The best data shows that LLM use might be destroying value [3]. This is basically a conspiracy theory and if you really believed this, you should not have led with "How is the capability advancement vs dollars paid for development?" because if there were no value, it doesn't really matter how much you invest.
- oudlys 4mo ago
  >This is basically a conspiracy theory I think this is pretty uncharitable, especially when I've provided you with a dataset you can evaluate yourself and an argument you can review for logical inconsistency. I have worked quite hard to locate data that supports your thesis, I can't find it. I've at least gone to the effort of documenting that search. Before you throw around such strong convictions, I suggest you actually look for yourself.
  - simianwords 4mo ago
    Respectfully, your link is not very convincing. But what’s interesting is that you are commenting on a post where Dario is suggesting that LLMs are so extremely powerful that they can take over, help synthesise bioweapons, help in warfare, help in drug discovery — the whole post here is to try and regulate this. If you believe AI can’t even create positive value let alone discover new things then your problem is somewhere else and not in something like “but training costs a lot”. So it is absolutely strange and contrasting to see you believe that LLMs are so weak as to create negative value while the CEO is asking about regulations because AI is too powerful. I don’t think I can convince you that AI is actually that powerful. But let me ask you something directly: if you believe what you believe, you should also acknowledge that AI doesn’t need regulations in the context Dario is proposing since obviously AI can’t do anything he predicts. Do you agree?
aspenmartin 4mo ago
I appreciate the data here but I don't think the read is quite right; Saying we have linear capability for super-linear cost compares an unbounded variable (dollars) to bounded instruments (because benchmarks saturate). On unbounded measures, growth is exponential; you can see METR time horizons double every ~4-7 months (https://metr.org/blog/2026-1-29-time-horizon-1-1/ https://metr.org/blog/2026-1-29-time-horizon-1-1/). And capability being proportional to log(compute) is what the scaling law predicts. Epoch puts training cost growth at ~2.4x/year as your link shows. Meanwhile cost for fixed capability falls ~10-40x/year (https://epoch.ai/data-insights/llm-inference-price-trends https://epoch.ai/data-insights/llm-inference-price-trends), and lab revenue is growing ~10x/year! Anthropic went from $1B to $9B to $30B+ run rate in ~15 months, OpenAI ~$25B. On [3]: the "destroying value" conclusion flips sign on an assumed 15% baseline rework rate. The report's most direct metric is +16% merged PRs per dev. The RCT evidence is genuinely mixed (METR: -19%, with n = 20 and Claude 3.x; Cui et al: +26%) but its just super hard to do this well, I think Faros stuff was pretty cool, I haven't seen this before so thank you for the reference.
- balefulboy 4mo ago
  METR's time horizon is not a reliable metric of LLM capability growth: https://www.transformernews.ai/p/against-the-metr-graph-coding-capabilities-software-jobs-task-ai https://www.transformernews.ai/p/against-the-metr-graph-codi...
  - oudlys 4mo ago
    Wow. This deserves to be much more widely read. Thank you for this.
  - aspenmartin 4mo ago
    Yes I've seen this before, and while the critiques are fair and high quality (and unfortunately not unique to METR) we're missing the forest for the trees here. First of all, if you take the articles critiques and work out the implications on the METR graph, all you're doing is shifting the curve up or down, it doesn't change the fact that progress is scaling exponentially. While it is technically possible the universe could be throwing a massive pathological curveball to change the conclusion from METR data (which is we've been seeing exponential growth over the last 6 years), I think that seems very far from likely. The fact that we see the same behavior from a variety of sources over a wide variety of tasks and domains is a pretty clear indication that METR while certainly far from perfect is actually painting a consistent picture at least in terms of the rate of progress. You can look at ECI for a summary benchmark statistic, which does NOT use METR's benchmark, and you see a similar trend. Same with SWE-bench where the task distribution is far more in domain for real world problems. It is a bummer that this METR data can't be better funded. It would probably take $1M or so to really beef it up properly which any of these labs probably have in their couch cushions.

snaking0776 4mo ago

That’s interesting. I commented something about this elsewhere but to me part of the exponential argument that loses me though is that it can often seem like a way to distract from issues that already exist which we should be working to fix. Things like autonomous weapons or mass surveillance are already here and rather terrifying and I would hope that we would dedicate our time to fixing those rather than having industry leaders focus so much on hypotheticals. While I guess the hypothetical scenario could be so bad that we must focus on it, I imagine a world which can’t come up with a way to spread wealth more equally or prevent mass proliferation of surveillance technology through profit seeking behavior will not be able to handle a digital super intelligence. So I keep coming back to the question: why is all I hear these industry leaders talking about is the threat of extinction? Maybe it’s just news coverage but I would love to see a leading lab release research on the health effects of subaudible sound in datacenters or other immediately present issues which would build good will towards these further out concerns.

hollerith 4mo ago
>why is all I hear these industry leaders talking about is the threat of extinction? . . . I would love to see a leading lab release research on the health effects of subaudible sound in datacenters It is straightforward for industry leaders to avoid living near data centers, but there's no way for them to insulate themselves from the extinction threat -- no way short of somehow eliminating the danger for everybody, which seems quite hard to do. Since industry leaders are as self-centered as everyone else, the extinction threat is what they think about. Also, you describe the extinction threat as "further out". A lot of us think there is already some small amount of AI extinction risk being incurred every day. I.e., we think the period of danger has already begun.
- snaking0776 4mo ago
  I see. I wonder how this works out in terms of risk/reward. I suppose if you take extinction as -infinite cost than it would be the only issue worth thinking about. Where I think this line of thinking gets challenging is when you need to take in terms of a counter factual. A lot of these were already risks prior to AI (bioweapons, nukes, etc) so what’s the marginal increase in probability as a result of AI I guess is the question which matters. I could get more around this way of framing it than saying that AI itself is the problem. It’s just the being more capable as a species increases risks. I think a lot of these pushback comes from the fact that it’s often the CEO who stands to gain huge by saying his tool is going to end the world so we need public buyin to supporting it. If instead it was just framed as “general technological advancement” is dangerous but potentially worthwhile I think more would be on board.