12 ms·

The sigmoids won't save you

308 points by Tomte 5mo ago

philipallstar 5mo ago
But they do explain the improvement of AI driving 2017-2021 vs 2022-2026.
nathan_compton 5mo ago
A lot of words to say "The initial part of a sigmoidal curve is not very informative about the parameters of the sigmoid function in question."
- inglor_cz 5mo ago
  That is true, but I generally enjoy reading a lot of words from Scott, who has a talent for writing. The entire plot of the Lord of the Rings could probably be compressed into less than 10 kB of text too. Edit: this seems to be a controversial comment, but IMHO a blog of Scott Alexander's type is an art form, not just a communication channel.
  - jeffreyrogers 5mo ago
    I find him more interesting when he talks about non-AI topics. Lots of other interesting people are like this too. I'd rather get my knowledge on AI from people who have unique insights into it. Scott has a lot of unique perspectives of his own, but his views on AI are bog-standard for his social group.
    - inglor_cz 5mo ago
      Frankly, me too, but he is still smart enough to introduce some grains of original thought even into those bog-standard views.
andai 5mo ago
Well, curve shape aside, the high watermark might be lower than where it tapers off. https://news.ycombinator.com/item?id=46199723 https://news.ycombinator.com/item?id=46199723
BoredPositron 5mo ago
If you use the log scale you'll see that the time horizon of opus 4.6 was as expected...
- afthonos 5mo ago
  As expected by the exponential. The Wharton study was predicting when the exponential would turn into a sigmoid.
- ReptileMan 5mo ago
  Everything is linear on a log log scale with a fat marker.
gm678 5mo ago
I don't know what the Y-axis is supposed to be on that Wharton AI capabilities graph, but I am not really convinced that Opus 4.6 has more than double the intelligence/capability/whatever of GPT 5.1 Max.
- BoredPositron 5mo ago
  https://metr.org/time-horizons/ https://metr.org/time-horizons/ on linear scale. Clickbait garbage article as most of his in the last year.
  - afthonos 5mo ago
    …yeah, that’s where you see the exponential?
- NitpickLawyer 5mo ago
  IIRC that graph tracks capabilities as time_to_solve a task for humans (i.e. the model can now handle tasks that usually take a human ~8h). Which, depending on what tasks you look at, could be a reasonable finding. I could see Opus 4.6 handling tasks that take ~8h for humans, and that 5.1 couldn't previously handle (with 5.1 being "limited" at 4h tasks let's say). It is a bit arbitrary, but I think this is what they're tracking.
  - lukan 5mo ago
    "It is a bit arbitrary, but I think this is what they're tracking." I don't know if they can get their numbers right this way, but this seems a way more useful metric, than theoretic capabilities.
    - cyanydeez 5mo ago
      ok, but arn't you just measuring efficiency and not the big I in AGI improvements.
      - 5mo ago
devmor 5mo ago
"Exponentials all tend to become sigmoids but you can't predict exactly when" is a true statement, but I'm not sure it needed an article. This doesn't say much, and the author fights their own points a couple times, suggesting that they maybe didn't think through what they wanted to write until they were in the middle of writing it and started realizing their assumptions didn't match what they expected the data to say. I really don't get the point of what I just read.
- aspenmartin 5mo ago
  The point is the tiring arguments from AI skeptics saying “things are flattening, they have to” which while technically correct says nothing because no one knows when that will happen and we see no mechanism for this yet. Lindy’s law as a reasonable prediction under total uncertainty is interesting and insightful and a lot of people don’t know about it or why it holds. I did enjoy the reference to this!
  - devmor 5mo ago
    But those skeptics are initially responding to the constant AI hype claims that we are exponentially growing to AGI. So this article is in fact just a (very poorly thought through) attempt at saying “nuh uh, the hype might be true, you can’t prove it’s not yet!
    - aspenmartin 5mo ago
      Yet the evidence is on the side of the hype? We don’t see any mechanism or cogent framework for what limits exist here theoretically that I’m aware of, are you? Epoch had a great article a year ago looking at several bottlenecks in terms of scale and back then we were about 4 orders of magnitude away from hitting them. We’re probably now closer to 3. Yet scale is only part of the performance equation, a fairly big chunk of progress is from algorithmic or curation related contributions. The point of the article is: > But those skeptics are initially responding to the constant AI hype claims that we are exponentially growing to AGI. This is a meaningless statement or at best just strawmanning.
addaon 5mo ago
https://xkcd.com/605/ https://xkcd.com/605/
- deleted 5mo ago
  [deleted]
inglor_cz 5mo ago
Hmmm, this is quite an interesting take by Scott. Lindy's Law is not actually a law and many exact minds will be provoked by the very name; it also fails spectacularly in certain contexts (e.g. lifetime of a single organism, though not necessarily existence of entire species). But at the same time, I am willing to take its invocation in the context of AI somewhat seriously. There is an international arms race with China, which has less compute, but more engineers and scientists. This sort of intellectual arms race does not exhaust itself easily. A similar space race in the 1950s and 1960s progressed from first unmanned spaceflight to a moonwalk in mere 12 years, which is probably less than what it takes to approve a bicycle lane in Chicago now.
- krupan 5mo ago
  "There is an international arms race with China" I keep seeing this. Where did it come from? Has China said that they intend to attack other countries using AI? Have other countries declared that they intend to attack China with AI? Also, why does anyone believe that AI could actually be that dangerous, given it's inherent unpredictable and unreliable performance? I would be terrified to rely on AI in a life or death situation.
  - inglor_cz 5mo ago
    It was a metaphor. I meant, and later clarified, an intellectual arms race. BTW your handle is an actual Czech word, minus a diacritic sign ("křupan"), and a bit amusing one. It basically means hillbilly. Not that it matters, just FYI. Anyway: AI will be used in military context, and it probably already is. Both for target acquisition and maybe even driving the weapon itself. As of now, the Ukrainians are almost certainly operating some AI-enabled killer drones.
    - krupan 5mo ago
      That's funny, I was told my real last name is a swear word in Czech
      - 5mo ago
theturtle 5mo ago
[dead]
krupan 5mo ago
News flash: predicting the future is hard
- energy123 5mo ago
  The individual who is the best at predicting the future is predicting ASI and full labor automation by 2040: https://xcancel.com/peterwildeford/status/2029636662322446613 https://xcancel.com/peterwildeford/status/202963666232244661...
  - gerikson 5mo ago
    Past results is no guarantee of future performance.
  - Aurornis 5mo ago
    > The individual who is the best at predicting the future Going to need a big citation for that claim
    - margalabargala 5mo ago
      Source: trust me bro
    - hirvi74 5mo ago
      No need. That man predicted he would be the best at predicting the future.
  - margalabargala 5mo ago
    > The individual who is the best at predicting the future Lol
  - layer8 5mo ago
    Predicting who will predict the future best is hard.
    - deleted 5mo ago
      [deleted]
kubb 5mo ago
If the scary AI is so inevitable, why do you feel such an overwhelming need to convince people about that? Surely you can just wait a bit, and they'll see for themselves.
- adleyjulian 5mo ago
  1. It's not inevitable. 2. Those that see AI as an existential risk don't generally think it's a guarantee, but if it's say a 5% chance then that's worth addressing/mitigating. 3. That's not what this article was even about.
  - kubb 5mo ago
    Sounds like the burden is on you to explain either 1. If you're not treating my claim as a black box, explain explicitly what is your model of what the article was about? Are you aware, for example of the last paragraph of the article? I think that WAS what the article was about. Do you have specific opinions on e.g. how I went wrong and where my model differs? 2. If you are treating it as a black box, what's your default expectation based on the law of Nothing Ever Happens? Just kidding, you don't need to explain anything. A"I" fearmongers should though.
    - adleyjulian 5mo ago
      The point of the article is that people are historically bad at predicting when exponential curves plateau, even if they're correct that there will be a plateau. This does *not* imply the inevitability of AGI. It does not imply AGI is necessarily bad. It does mean that "the capabilities of AI will eventually plateau" offers no meaningful predictive power or relevance to the overall AI discussion.
- mitthrowaway2 5mo ago
  By that reasoning, why even warn people about anything? Why do road construction crews put up signs saying "ROAD CLOSED AHEAD" when you can just drive on and see for yourself?
LarsDu88 5mo ago
I think an interesting thing about recent AI developments is that its all happening right as we hit the diminishing returns side of another "exponential that's actually a sigmoid" which is Moore's law. The naive expectation is that AI will slow down b/c Moore's law is coming to an end, but if you really think about the models and how they are currently implemented in silicon, they are still inefficient as hell. At some point someone will build a tensor processing chip that replaces all the digital matmuls with analogue logamp matmuls, or some breakthrough in memristors will start breaking down the barrier between memory and compute. With the right level of research funding in hardware, the ceiling for AI can be very high.
- cyanydeez 5mo ago
  they already did put a model into the silicon and it's crazy fast. https://chatjimmy.ai/ https://chatjimmy.ai/ I'm pretty sure there's a 3 year design goal starting this year that'll do that to any of the qwen, deepseek, etc models. There's a lot you could do with sped up models of these quality. It might even be bad enough that the real bubble is how much we don't need giant data centers when 80-90% of use cases could just be a silicon chip with a model rather than as you say, bloated SOTA
  - clickety_clack 5mo ago
    It would be pretty cool to have interchangeable usb keys with models on them.
  - LarsDu88 5mo ago
    And this is an asic that is still operating digitally. Imagine a chip with baked it weights that does its math analogue with 20x reduction in number of circuit elements needed to do a multiplication op. If there's a breakthrough in memristors, you could end up with another 20x reduction in circuit elements (get rid of memory bottlnecks, start doing multiplication ops as log transform voltage addition) The ceiling is ultra high for how far AI can go.
    - mordae 5mo ago
Brendinooo 5mo ago
> then what is their model? My mental model has been 3D computer graphics: doubling the polygon count had huge returns early on but delivered diminishing returns over time. Ultimately, you can't make something look more realistic than real. I don't know what the future holds, but the answer to the question "can LLMs be more realistic than real" will determine much about whether or not you think the curve will level off soon.
- the8472 5mo ago
  The equivalent bar in this domain would be human intelligence, and we already have growing lists of tasks where machines outperform humans. We even known of natural systems that outperform humans on some metrics, e.g. bird-brains have higher neuron density than ours because evolution had to optimize more for weight.
- kilpikaarna 5mo ago
  In 3D graphics there's diminishing returns on investment for the technology itself, but the real limit is one of economics. How to create all those assets required by the rendering tech, and make your money back. Preferably while also keeping your customers interested long term, by not becoming risk averse. In the same fashion, LLMs have to pay for themselves to keep the trendlines going. In a whole-systems -sense, mind, not "$2000/month is cheaper than hiring a developer" while the rest of the economy collapses.
btilly 5mo ago
Lindy’s Law is an absolute gem, that I'm keeping. If we don't understand the fundamental limits to any particular kind of trend, our default assumption should be that it will continue for about as long as it has gone on already. We can, in fact, easily put a confidence interval on this. With 90% odds we're not in the first 5% of the trend, or the last 5% of the trend. Therefore it will probably go on between 1/19th longer, and 19 times longer. With a median of as long as it has gone on so far. This is deeply counterintuitive. When we expect something to last a finite time, every year it goes on, brings us a year closer to when it stops. But every year that it goes on properly brings the expectation that it will go on for a year longer still. We're looking at a trend. We believe that it will be finite. Our intuition for that is that every year spent, is a year closer to the end. But our expectation becomes that every year spent, means that it will last yet another year more! How can we apply that? A simple way is stocks. How long should we expect a rapidly growing company, to continue growing rapidly?
- jerf 5mo ago
  It's an interesting idea, and it may be something that could be mathematically justified, but I do think this is an abuse of Lindy's Law in the absence of such a justification. Per Wikipedia [1]: "The Lindy effect applies to non-perishable items, like books, those that do not have an "unavoidable expiration date"." And later in the article you can see the mathematical formulation which says the law holds for things with a Pareto distribution [2]. I'd want to see some sort of good analysis that "the life span of exponential growth curves" is drawn from some Pareto distribution. I don't think it's completely out of the question. But I'm also nowhere near confident enough that it is a true statement to casually apply Lindy's Law to it. [1]: https://en.wikipedia.org/wiki/Lindy_effect https://en.wikipedia.org/wiki/Lindy_effect [2]: https://en.wikipedia.org/wiki/Pareto_distribution https://en.wikipedia.org/wiki/Pareto_distribution
  - btilly 5mo ago
    The analysis in the article explains why it applies to any phenomena that we might be able to notice. The argument given is the same as the one that I first ran across, not by that name, in https://www.nature.com/articles/363315a0 https://www.nature.com/articles/363315a0. https://en.wikipedia.org/wiki/Doomsday_argument https://en.wikipedia.org/wiki/Doomsday_argument claims that it was a rediscovery of something that was hypothesized a decade article. I hadn't tried to give it a name, or thought to apply it outside of that context. As for the mathematical qualms, I'm a big believer in not letting formal mathematical technicalities get in the way of adopting an effective heuristic. And the heuristic reasoning here is compelling enough that I would like to adopt it.
itkovian_ 5mo ago
The other thing people don’t understand is exponential curves are self similar. The start of an exponential looks like an exponential. People always look at and think ‘well that’s it it’s exponential now, have missed it, can’t sustain’. Nope. Good example of this is number of submissions to neurips/icml/iclr. In 2017 that curve was exponential.
zkmon 5mo ago
The curve is a smoothed step curve (y=1 if x>1 otherwise 0). Nature doesn't allow any change to happen instantly at any degree of rate of change. The curveis just a manifestation a change with exponential smoothening of the sharp corners. For example, When a car starts, it's speed and acceleration become more than zero. But what about rate of change in higher degrees? It suddenly doesn't change from zero acceleration to non-zero. That means the car has a non-zero derivative at all degrees. In other words, the movement is exponential. The same thing happens in reverse when the car reaches a constant speed.
patrickmay 5mo ago
Stein's Law: "If something cannot go on forever, it will stop."
- skybrian 5mo ago
  Yes, but figuring out when is the hard part.
OscarCunningham 5mo ago
John D Cook gives more technical details here: "Trying to fit a logistic curve" https://www.johndcook.com/blog/2025/12/20/fit-logistic-curve/ https://www.johndcook.com/blog/2025/12/20/fit-logistic-curve...
bedobi 5mo ago
[flagged]
- simianparrot 5mo ago
  Because HN is YCombinator which has invested in probably hundreds of «AI» firms by now. Including OpenAI. Allowing slop articles like this literally prints them evaluation money.
  - t43562 5mo ago
    Yes, this is not the place to express skepticism of any kind.
    - simianparrot 5mo ago
      The subtle but consistent downvoting to a score of 0 to -1 feels very botted. No matter how I write the comments, anything counter the "AI" propaganda nets me a 0 or -1 total score. I don't care about the points whatsoever, but I find it interesting how there's never a massive downvote, just enough to keep such comments at 0 to -1. May be reading into things too much, but it is a bit odd.
- ngriffiths 5mo ago
  I think there are many ways someone with his lack of expertise can still be valuable, including: - Making connections to other subjects that an expert would miss. The hall of fame of sigmoid predictions is just excellent, I already know I'm going to be reminded of it some time in the future. Very entertaining way to get the point across. - Writing about tricky concepts in a very accessible and elegant way, which experts are notoriously bad at doing themselves - they are often optimizing for other specialists. - Being able to write with an air of speculation and experimentation with ideas that experts and institutions often can't afford. Experts have to maintain their track record; Scott Alexander can say "lol just double the timeline"
  - bedobi 5mo ago
janalsncm 5mo ago
> What if you don’t fully understand the process? AI forecasters know some things (like how data centers work and how much it costs to build them). But they’re unsure about other things (researchers keep inventing new paradigms of data generation that get over data walls, but for how long?), and other things are entirely opaque (What is intelligence really? Why do scaling laws work? Might they just stop working at some point?) Is there anything you can do here? This is the crux of the article. To a large extent continued progress depends on a stable increase in compute, an increase in training data, and an increase in good ideas to squeeze more out of both of them. One calculation you could do is a survival function: for each of the above, how long before it is disrupted? For example, China could crack down on AI or invade Taiwan. Or data centers become politically unpopular in the US. Or, we could run out of great ideas. Very hard to predict.
dsign 5mo ago
We did hit the sigmoid's plateau on airplane speed, but the applications of airplane speed are still coming (how fast can a Chinese company airship the PCB you ordered three minutes ago?). I expect the the same will happen with LLMs, though I also happen to believe things are just getting started on end capabilities.
jsmcgd 5mo ago
> It’s true that birth rates must eventually flatten out and become sigmoid All positive growth eventually flattens out and becomes sigmoid, but a lot of phenomena experience negative growth and nose dive. No gentle curve, but a hard kink and perfect flat line at zero. Forever. I think it would be a stretch to categorize that pattern as sigmoid. Predicting a sigmoid pattern for negative growth implies some sort of a soft landing (depending on your definition of soft). We can think of many populations that are no longer with us. So just a caution about over applying this reasoning in the negative case.
- Qem 5mo ago
  > All positive growth eventually flattens out and becomes sigmoid, but a lot of phenomena experience negative growth and nose dive. https://en.wikipedia.org/wiki/Seneca_effect https://en.wikipedia.org/wiki/Seneca_effect
ngruhn 5mo ago
> all exponentials eventually become sigmoids Except innovation. When one sigmoid tapers off we keep finding new ones to keep the climb going.
stymaar 5mo ago
I don't know when the sigmoid is going to kick in, but Nvidia's Quaterly datacenters revenues have been grown 15 folds over the past 3 years[1], and nobody including Scott believes this is sustainable for 3 more years otherwise Nvidia's market cap would conservatively be at least an order of magnitude higher than it is. All exponential eventually becomes a sigmoid because exponential growth always expose limiting factors that weren't limiting at the beginning. Silicon manufacturing had lots of room for high-margin customers like Nvidia even a year ago (by the mere virtue of outbidding lower-margin customers), but now it is mostly gone, and no amount of money will make fabs build themselves overnight. [1]: https://stockanalysis.com/stocks/nvda/metrics/revenue-by-segment/ https://stockanalysis.com/stocks/nvda/metrics/revenue-by-seg...
pyrale 5mo ago
Such a long article to say that neither side has a fucking idea about what will happen next. While we're at it, the "exponentials are actually sigmoïds" meme is not necessarily true. While exponentials are never exponentials, sigmoids are not guaranteed. Overshoot-and-collapse examples also happen in tech, e.g. the dotcom bubble, or the successive AI winters.
- andrewflnr 5mo ago
  It's really not that long, and is quite clear that its main point is about how to reason when you realize no one actually knows what's going on.
dreambuffer 5mo ago
FYI: The author has predicted that "AGI" will be here in 1-2 years and has staked his public reputation on it. He is personally invested in trendlines being lindy rather than sigmoid. I don't think you can use lindy on trends as if trends are static objects, but that's another conversation.
- throwawayk7h 5mo ago
  Mind you, he is only personally invested insofar as he's staked his reputation on it. Throughout his writing, he expresses the same point over and over again: desperately wants AI to slow down, advocates for politics that would slow it down, and most likely nothing would bring him greater peace than to see a sigmoid curve appear.
  - bombcar 5mo ago
    How convenient; when AGI doesn’t appear in 1-2 years his reputation is pristine because he slowed it down.
    - wmf 5mo ago
      To make that argument you'd want to show some causal link which so far we haven't seen.
    - achierius 5mo ago
      What do you want? This sounds like you have something against people making a claim in public, at all, on any topic of importance.
- woeirua 5mo ago
  Ok, but you can just look at the METR curve. Mythos saturated the 50% time horizon. The 80% is now at 3 hours. The rate of progress is accelerating not slowing down. There’s no indication yet that this is a sigmoid!
  - yorwba 5mo ago
andy99 5mo ago
AI has scaled well according to convenient measures. It (neural networks) have the property that whatever you define, they can rapidly be trained master it. We’re able to show that various tasks of increasing complication do not require intelligence and can be framed as autoregressive RL problems. I personally don’t think AI is any closer to sentient intelligence than LeNet; it’s almost trivially clear, we know how it works. So we’re measuring something orthogonal, basically how well a universal function approximator can fit to a function we define, given arbitrary computing power, and calling that progress. What will be really interesting is if we’re able to find a way to properly measure what they can’t do and what’s different about real intelligence. Edit: in particular I don’t agree with But if someone claims that the trend toward increasing AI capabilities will never reach some particular scary level... One has to agree that the benchmark results are getting “scarier”, which is not automatically implied by finding more goals to optimize for
- ordu 5mo ago
  > We’re able to show that various tasks of increasing complication do not require intelligence and can be framed as autoregressive RL problems. The important thing we can show it in hindsight only. We don't know which other tasks we are currently mistaken about requiring intelligence. Maybe none of them are? We don't know. We don't know what intelligence is. If we look at decades and even centuries of attempts to define intelligence, it is all looks like a goalposts moving. When a definition of intelligence starts to include people or things we don't like to think as of intelligent ones, we change the definition.
  - ryeights 5mo ago
    “AI is whatever hasn't been done yet” — Larry Tesler
  - winwang 5mo ago
    Yep. No one bats an eye at eyewitnesses "hallucinating" details, or that I'd rather have Opus as a coworker vs a random middle schooler (err, labor laws notwithstanding). I think perhaps too much of the dialogue around intelligence has to do with the word (and its connotations) itself. The poster you replied to even used the word "sentient", which is quite interesting (warning: opinionated tangent ahead). Merriam-Webster defines it as "capable of sensing or feeling: conscious of or responsive to the sensations of seeing, hearing, feeling, tasting, or smelling". Feels like qualia. Or if we don't want to go the qualia route... Of course, we wouldn't call Helen Keller non-sentient, so presumably we "really" mean "can it sense or feel" -- well, sense is just "act/feel according to the environment", which you could argue in the case of an LLM would be their context... so we should "really" remove "sense" from the definition, probably. So "do LLMs feel" is probably closer to what "sentient" is being used for here. Since we don't have the obvious symmetry of "you are like me and I feel (therefore you probably feel)", it's way better/easier/feel-good-ier to prefer "LLMs don't feel" rather than "oh shit, it feels and model training is actually just torturing it into the right shape". LLMs as fundamentally non-intelligent also avoids the problems of "what does that say about people" or "we may have made 'AGI' and it wasn't what we thought it would be" or "we're not ready to talk about this yet".
baxtr 5mo ago
> The moral of the story is that, even though all exponentials eventually become sigmoids, this doesn’t necessarily happen at the exact moment you’re doing your analysis. Sometimes they stay exponential for much longer than that! All exponentials eventually become sigmoids? Don’t think this can be true without qualifiers.
- jvanderbot 5mo ago
  All models are wrong, of course, but this is kind of "common sense" so it's not hard to accept as true in a natural system. How can something continue on exponential growth forever without reaching a new blocker that causes slowdown or encountering pushback that makes it an oscillator. A pendulum looks exponential when it is at its peak and accelerating down. The issue is that the exponential-looking part of the sigmoid might contain all of human history, sure, but most folks who espouse this theory probably agree that over time everything reaches a steady-enough state to be considered non-exponential, or become oscillatory.
jrflowers 5mo ago
I like this article about how we should assume, at any given point, that we are exactly halfway through a phenomenon which relies on a single data point on a graph —-that apparently doesn’t need its relevance or importance explained— to illustrate that this is obviously true for AI in particular
- Smaug123 5mo ago
  I think you may have read a different article from me. The thesis of the article is summarised at the end: > But if someone claims that the trend toward [X] will never reach some particular scary level, then the burden is on them to explain either: > If they’re not treating [X] as a black box, and claim to be modeling the dynamics explicitly, then what is their model? Have they calculated the obvious things… > If they are treating [X] as a black box, why isn’t their default expectation based on Lindy’s Law? Like, the whole point is that in real life we do actually know things about situations and can model them; we fall back to Lindy's law when we know nothing at all. Further, arguments have justification to deviate from Lindy only when they give specifics about the situation they're modelling.
noosphr 5mo ago
This article answers the question in the second paragraph then completely ignores the answer for the rest of it. >My understanding is that this represents 3-4 “generations” of different technology (propellers, turbojets, etc). Each technology went through normal iterative improvement, then, when it reached its fundamental limits, got replaced by a better technology. The last technology, ramjets, reached its limit at about 3500 km/h, and there wasn’t the economic/regulatory will to develop anything better, so the record stands. You don't have one sigmoid, you have multiple each stacked on top of each other. Airplanes aren't just one technology they are multiple technologies that happen to do the same thing. Each one is following a sigmoid perfectly. It only looks exponential(ish) because of unpredictable discoveries that let you switch to another sigmoid that has a higher maximum potential. The same is true in AI. If you used the same architecture as GPT2 today you're in for a bad time training a new frontier model. It's only because we have dozens of breakthroughs that the capabilities of models have improved as much as they have. That said exponential and sigmoids are the wrong model to use for growth. Growth is a differential equation. It has independent inputs, it has outputs and some of those outputs are dependent inputs again through causal chains of arbitrary complexity. What happens depends entirely on what the specific DE that governs the given technology is. We can easily have a chaotic system with completely random booms and busts which have no deep fundamental rhyme or reason. We currently call that the economy.
- Sniffnoy 5mo ago
  Yes, I was surprised he never discussed the idea that such exponentials are typically made of stacked sigmoids. That said... if the exponential is made of stacked sigmoids, it's still an exponential on the whole! The fact that it's made of stacked sigmoids is relevant to the engineers making it, but not so relevant to the users or those otherwise affected by it.
  - noosphr 5mo ago
    Only so long as you can keep inventing the next sigmoid in the stack.
    - 5mo ago
pron 5mo ago
1. Scott Alexander is famous for writing about topics he knows little about. I'm glad to see he's found a subject he knows little about but so does everyone else. 2. What's even worse than predicting that some growth curve flattens before X happens is predicting it will flatten before X happens but after Y happens, which is what we see when it comes to AI in software development. Too many people predict that AI will be able to effectively write most software, replacing software engineers, yet not be able to replace the people who originate the ideas for the software or the people who use them. I see no reason why AI capability growth should stop after the point it's able to write air-traffic control or medical diagnosis software yet before the point where it's able to replace air traffic controllers and doctors. 3. While we don't know much about AI (or, indeed, intelligence in general), we do know something about computational complexity. Some predictions about "scary things" happening (the ones I'm guessing Alexander is alluding to, though I can't be certain) do hit known computational complexity limits. Most systems affecting people are nonlinear (from weather to the economy). Predicting them requires not intelligence but computational resources. Controlling them, similarly, requires not intelligence but either computational resources or other resources. It's possible that people choose to give control over resources to computers (although probably not enough to answer many tough, important questions), although given how some countries choose to give control to people with below-average intelligence (looking at you, America), I don't see why super-human intelligence (if such a thing even exists) would be, in itself, exceptionally risky.
- ryeights 5mo ago
  >1. Scott Alexander is famous for writing about topics he knows little about. I'm glad to see he's found a subject he knows little about but so does everyone else. This is kinda laughable. Scott has been thinking and writing about AI for a long time
  - pron 5mo ago
    That's what I'm saying. I'm glad to see that he's found a subject where his lack of knowledge isn't a glaring handicap. When I come across his posts I usually feel uncomfortable because they read like a bright 4-year-old child trying hard to explain how a car works, only not cute (yeah, I know that trying to see what conclusions you can come to, Aristotle-like, from a basis of ignorance and without careful study is the whole point, but I never found this Memento-style game appealing).
leoc 5mo ago
Hmm. What’s the general belief about Toby Ord’s “Are the Costs of AI Agents Also Rising Exponentially?” https://www.tobyord.com/writing/hourly-costs-for-ai-agents https://www.tobyord.com/writing/hourly-costs-for-ai-agents among those who are well-equipped to judge? Is it seen as wrong or disproven or unlikely? Because if not—if indeed recent LLM capability advances have likely relied on increases in inference cost per run which can’t be much further sustained—then it seems remiss not to mention that if you point to those advances to claim that the exponential trend remains on track.
lovich 5mo ago
I wonder how the graph would look like if cost and/or profitability was taken into account. I could probably make increasingly larger fires for years if I was willing to burn the entire world.
whatshisface 5mo ago
If you want a model, here's one: LLMs have never demonstrated the ability to go obviously beyond interpolating their training data. It takes an army of paid data producers solving homework problems to give ChatGPT the ability to do your homework. All vibecoded apps that turned out to be successful could put on a geological soil chart with other apps, probably on GitHub somewhere, on the corners. The prediction? They won't. In this model, the exponential growth that everybody is freaking out about is only the realization of the modular software dream ("we'll only have to write an ORM once for all of human history!") and the sheer amount of knowledge in libraries. It's at least falsifiable.
- boxed 5mo ago
  Just to play devils advocate: are we sure humans have demonstrated the ability to go beyond their training data? Like.. are we sure-sure about that?
  - 1attice 5mo ago
    Idk, I mean, Shakespeare never read Shakespeare, so, I mean, unless aliens?
    - boxed 5mo ago
      AI images are all unique and new.
      - 1attice 5mo ago
        Not nearly as new as Shakespeare was when he single handedly reinvented English vocabulary, for example.
        boxed 5mo ago
        He accidentally formalized spelling. You can't invent vocabulary when you're writing plays for a mass audience and want the audience to actually understand what is being said. That makes no sense.
stego-tech 5mo ago
I felt the better takeaway from this was that it's impossible to know for certainty how long this will or will not continue regardless of the data or models you're using, because if you (or anyone else) could predict that accurately they'd be one of the richest people on the planet. I don't know when (or if) AI will implode or succeed with any degree of provable certainty, because that's not my area of expertise. Rather, I can point out and discuss flaws in the common booster and doomer arguments, and identify problems neither side seems willing to discuss. That brings me cold comfort, but it's not enough to stake my money on one direction or another with any degree of certainty - thus I limit my exposure to specific companies, and target indices or funds that will see uplift if things go well, or minimize losses if things go pear-shaped. I also think relying on such mathematics to justify a position in the first place is kind of silly, especially for technical people. Mathematical models work until they don't, at which point entirely new models must be designed to capture our new knowledge. On the other hand, logical arguments are more readily adapted to new data, and represent critical, rather than mathematical, thinking and reasoning. Saying AI is going boom/bust because of sigmoids or Lindy's Law or whathaveyou is not an argument, it's an excuse. The real argument is why those things may or may not emerge, and how do we address their consequences within areas inside and outside of AI through regulation, innovation, or policy.
- bjackman 5mo ago
  I think his agenda here is to point out that your probability distribution for AI outcomes should be broad (what you said), but most importantly: this means you must take seriously the possibility that we are gonna get superintelligence quite soon. Basically a lot of people say "but isn't it also pretty likely that we DON'T get superintelligence?" And, yes, it is. But superintelligence being even a remotely plausible outcome is a big fucking deal. Your investment choices in that context are not important. People really struggle to think rationally in the face of this shape of uncertainty.
  - stego-tech 5mo ago
    You’re 100% correct, which is why I opted for a broad investment approach rather than trying to pick “winners”. My thought process RE: superintelligence/AGI is generally this: * I personally don’t believe it’s likely to happen with silicon-based computing due to the immense power and resource costs involved just to get to where we are now; hence why I invest broadly to capitalize on what gains we actually attain using this current branch of AI research across all possible sectors and exposure rates * If we do achieve AGI using silicon-based computing, its limited scale (requiring vast amounts of compute only deliverable via city-scale data centers) will limit its broader utility until more optimizations can be achieved or a superior compute platform delivered that improves access and dramatically lowers cost; again, investing broadly covers a general uplift rather than hoping for a specific winner * If AGI is achieved, nobody - doomer or booster alike - will know what comes next other than complete and total destruction of existing societal structures or institutions. The stock market won’t explode with growth so much as immediately collapse from the disintegration of the consumptive base as a result of AGI quite literally annihilating a planet’s worth of jobs and associated business transactions. In this case, a broad spread protects me from harm by spreading the risk around; AGI will annihilate the market globally, but not all at once barring a significant global catastrophe instigated by it * Which brings me to the worst outcome, where AGI follows the “if anybody builds it everyone dies” thought process: investment is irrelevant because we’re all fucked anyway. And that’s just my investment approach. I’m too pragmatic to believe we’re at the bottom of the sigmoid curve, but too wise to begin guessing where we actually exist on it at present or how much is left in the current LLM-arm of AI research; I’m an IT dinosaur, not an AI scientist. What I can point to is the continued demand destruction of consumer compute through higher costs and limited availability due to rampant AI speculation as proof that the harm is already here in a manner most weren’t predicting, while at the same time actual job displacement by AI is limited to the empty boasting of executives using it as a smoke screen for layoffs after RTO mandates failed to thin headcount sufficiently. In the USA in particular, we’re facing a perfect storm of: * consumer confidence collapse leading to a decline in spending on all goods, especially luxury ones, by all but the most monied demographics * data center-driven cost increases (energy) and resource destruction (land, water, fossil fuel use) * the eradication of government support for renewable energy that would’ve kept these costs in check * the widening wealth gaps creating a new underclass not seen since before WW2 In other words, most of the discourse continues to revolve around hypotheticals of tomorrow rather than realities of today. That would be the lesson I’d hope more people take away from something like this, so we can finally begin addressing issues themselves rather than empty online circle jerking about who is right or wrong.
dnnddidiej 5mo ago
Tldr you cant accurately predict the future of complex systems. Corollary... you cant accurately predict the future of complex systems using sigmoid. Attention is all you need took us by surprise and we don't know how big the wave is let alone if there are other waves behind it.
graphememes 5mo ago
line can go up, line can go sideways, line can go up sideways, line can go up sideways up, line go where line go
jasongi 5mo ago
First sigmoid was transformers allow us to rapidly scale to our already abundant data until we tapped it out, the second is/was reasoning, allowing us to scale to our available compute (and compute manufacturing capacity). Correct me if I'm wrong but we don't have candidate for the third sigmoid, and scaling inference is hitting real-world supply chain constraints - electricity and chips. Short of a third sigmoid appearing in the ML CompSci space, perhaps in the form of ongoing, repeated step-optimisations which will also have diminishing returns, intelligence growth is now limited a few scaling problems that have already been worked on for a very long time. Transistors, which have been doubling for almost a century now, but Moores Law has already plateaued and reached limits on energy efficiency, and simply building new fabs is not something that we can do exponentially. And the other growth limiter is electricity - there is no exponential supply of fossil fuels or power plants. Although manufacturing has scaled, PV tech improvements are also plateauing - and while storage is getting cheaper, it's still not economical vs fossil fuels (meaning: when we have to switch to it, the growth slows down further) and we are unlikely to see battery efficiency sigmoid enough to maintain the AI sigmoid. I don't mean to be bearish here. There's so much money sloshing around that we can afford to put the smartest people, using unlimited tokens, on the task of finding small, incremental gains on the CompSci side of things that will have large monetary payoffs - hopefully allowing further scaling and increased emergent abilities of LLMs. Maybe we can squeeze the algos for quite a while. But I don't see that maintaining the same level of exponential as unlocking unlimited data or maxxing out the world's energy/fab capacity for long. And I don't see why this is a massive issue except for the people who want to have some god-like super AI? Frontier LLMs are genuinely magic. Not "won't delete your production database" magic, but definitely a massive productivity gain for competent knowledge workers.
- foobar10000 5mo ago
  We are multiple orders of magnitude away from Landauer limits - so next big thing in matmul could be photonic multipliers - there’s a bunch of them coming up in the next 3? years. So that’s a 2-4 order of magnitude improvement. Sigmoid?
- ianm218 5mo ago
  Reinforcement learning has become a huge portion of compute used during training runs [1] and synthetic data is letting us get lots more mileage out of the existing data. Additionally, there is lots of new, high quality data being created and collected each day. I think the "running out of data" thing was pretty poorly reported by mainstream media. [1]. https://www.dwarkesh.com/p/dario-amodei-2 https://www.dwarkesh.com/p/dario-amodei-2
mgraczyk 5mo ago
Don't bet on the sigmoid flattening out any time soon. I don't know when it will flatten, but it definitely won't be within the next 2 years
overgard 5mo ago
This feels like a really verbose way of saying "things have been growing fast for a while so they should continue to grow as fast for just as long", and then he places the burden on people to prove him wrong. Um, no, the burden of proof is shared, "this will just keep going" requires just as much proof as "this is going to level off" if you're just looking at trend lines. It's better to look at the underlying factors. Money sources are drying up, nobody is making a profit outside of nVidia, most blackwell GPU's are likely not even installed yet and will probably be 2 generations behind when they finally are being used, data centers are hitting all sorts of obstacles getting built and powered and they're getting built slowly, most AI researchers seem to think that LLMs are a dead end, the newer models seem to be getting more expensive and sometimes worse, or even potentially are showing signs of model collapse (goblins..), the supposed productivity gains are not materializing.. AI has worse public sentiment than congress.. I could keep going. Some obscure "law" seems to pale in comparison to the hard evidence that the status quo is utterly unsustainable and none of these companies seem to have a realistic plan other than trying to become too big to fail essentially. I like some of this guy's writing on other topics, but to me this is a prime example of what happens when you get public "intellectuals" talking about subjects far outside of their area of expertise. It's not as bad as Richard Dawkins latest fall into psychosis but it's basically the same phenonmenon.
plomme 5mo ago
I’m not saying he’s wrong about the core thesis here, but using Claude Opus 4.6 as a “mic drop” with a chart showing it being twice as good as the last model feels in my experience way off.
Permit 5mo ago
> But if someone claims that the trend toward increasing AI capabilities will never reach some particular scary level, then the burden is on them to explain either:… This is not the context in which I hear about sigmoids vs exponentials. I hear it in regards to “the singularity”, not that AI won’t reach some pre-specified level. You may get AGI, you aren’t getting a singularity.
izucken 5mo ago
Births is, sans miscalculation, a number that tracks exact events. Is the "capability" number on these LLM strengh graphs as tangible? I think it would be interesting to visit a reality that obeys arbitrary abstractions, but I would personally never go there.
mordae 5mo ago
I am 100% certain it's a sigmoid. Speed of light is finite. :-)
niemandhier 5mo ago
If you look at problems that can be solved by reasoning in text form or maybe even images, I am more than willing to accept that we simply cannot know when the curve will level. The situation is drastically different for problems that require interaction was the physical world to determine success. As soon as you add a powerful simulator for physical problems to the self learning experience of the AI, you are extremely hampered by the large amount of needed computation.
mentalgear 5mo ago
> Why do scaling laws work? Strictly speaking, the original paradigm of scaling laws doesn't work any more. The assumption that we could achieve better performance simply through "vertical scaling" ie infusing models with exponentially more parameters and pre-training data, is no longer the driving force of AI progress. Instead, the industry has pivoted toward inference-time scaling. Rather than relying solely on a massive, static neural network, modern architectures allocate more compute during the actual generation process, allowing the model to "think" and verify its logic dynamically. Furthermore, the latest state-of-the-art models are no longer pure LLMs; they are compound neuro-symbolic systems that integrate external tools like REPLs, databases, and structured skill documentation to archive things pure LLM vertical parameter scaling was not able to do.
- yorwba 5mo ago
  The "law" part of scaling laws is about predicting validation cross-entropy loss from the training configuration, analogous to physical laws allowing to predict one quantity based on the measurement of another. Most scaling laws take the form of an irreducible error plus additional terms that asymptotically decay to zero. So that there is a wall you can approach but not cross (the irreducible error) is an integrated part of the scaling law paradigm. That it isn't economical to keep increasing model size to squeeze out a few more drops of cross-entropy doesn't mean scaling laws stopped working. Strictly speaking, "Why do scaling laws work?" is a question about the theoretical reasons the asymptotic decay takes the particular mathematical shape that it does.
Staross 5mo ago
You should really do a Bayesian fit for such predictions and give confidence intervals, it would probably show that the uncertainty is very high in these cases.
amelius 5mo ago
The exponentials won't save you either. In fact it's more likely that a sigmoid takes over at some point.
AvAn12 5mo ago
Forecasts are a thought exercise, not the revelation of something foretold. Best thing to do is think of the outcome you wish for and then try to take whatever actions you can to help make it so. Like with climate change for example.
- torginus 5mo ago
  It can be both. In the field of high-technology (like semiconductor) manufacturing, the techinques of making chips with feature sizes many generations ahead of current SOTA exist in various states of completion, so extrapolation can be made from that. But what's going on in here is not that - it's reading tea leaves for maximum dramatic effect.
FrustratedMonky 5mo ago
Black Swan Events probably look like nothing just before they happen.
theuniverseson 5mo ago
[flagged]
tim-projects 5mo ago
Does hype follow a sigmoid curve?
dwa3592 5mo ago
I am not disagreeing with the conclusion of the article but the way they reached the conclusion is problematic (and wrong) and here is my nuanced explanation of that : There is no attention paid to the origin (0,0) in any of the plots in the article. To see a meaningful trend you have to be be able to zoom out and zoom in, which means your origin is not a fixed point, ever. This also means that plots with fixed point origins don't tell us anything important at a societal level. For example: the flight airspeed plot, it starts at ~1900. Now of course it should start there bc we did not have planes before that. But let's change the plot heading from airspeed to human speed a.k.a at what speed could humans move. now you can change the origin meaningfully as we had chariots, horses, bicycles, ships before that. If you instead create a plot for last 5000 years you would see that the speed at which humans are able to move is rising exponentially, from walking on foot in a radius of 1000-2000feet in a thick jungle 5000 years ago to reentering earth at 25000 miles/hour in 1969 (yeah, read that again). Even for AI, if you zoom out the plot to last 70 years it will look exponential, if you zoom in to last 2 months it will look absolutely flat. The point is that the whole sigmoid/exponential argument is a function of the origin (0,0).
naorsabag 5mo ago
[flagged]
fph 5mo ago
As a wise man once said, "anyone who believes exponential growth can go on forever in a finite world is either a madman or an economist".
morpheos137 5mo ago
The Yudkowskian 'Bayesians' are a scifi cult. Intelligence as it exists in nature is neither arteficial nor general nor 'super.' All human technology is natural. It certainly is not supernatural.
jgalt212 5mo ago
There are practitioners and pontificaters, and never the twain shall meet.

308 points by Tomte 5mo ago

But they do explain the improvement of AI driving 2017-2021 vs 2022-2026.

A lot of words to say "The initial part of a sigmoidal curve is not very informative about the parameters of the sigmoid function in question."

inglor_cz 5mo ago
That is true, but I generally enjoy reading a lot of words from Scott, who has a talent for writing. The entire plot of the Lord of the Rings could probably be compressed into less than 10 kB of text too. Edit: this seems to be a controversial comment, but IMHO a blog of Scott Alexander's type is an art form, not just a communication channel.
- jeffreyrogers 5mo ago
  I find him more interesting when he talks about non-AI topics. Lots of other interesting people are like this too. I'd rather get my knowledge on AI from people who have unique insights into it. Scott has a lot of unique perspectives of his own, but his views on AI are bog-standard for his social group.
  - inglor_cz 5mo ago
    Frankly, me too, but he is still smart enough to introduce some grains of original thought even into those bog-standard views.

andai 5mo ago

Well, curve shape aside, the high watermark might be lower than where it tapers off. https://news.ycombinator.com/item?id=46199723 https://news.ycombinator.com/item?id=46199723

BoredPositron 5mo ago

If you use the log scale you'll see that the time horizon of opus 4.6 was as expected...

afthonos 5mo ago
As expected by the exponential. The Wharton study was predicting when the exponential would turn into a sigmoid.
ReptileMan 5mo ago
Everything is linear on a log log scale with a fat marker.

gm678 5mo ago

I don't know what the Y-axis is supposed to be on that Wharton AI capabilities graph, but I am not really convinced that Opus 4.6 has more than double the intelligence/capability/whatever of GPT 5.1 Max.

BoredPositron 5mo ago
https://metr.org/time-horizons/ https://metr.org/time-horizons/ on linear scale. Clickbait garbage article as most of his in the last year.
- afthonos 5mo ago
  …yeah, that’s where you see the exponential?
NitpickLawyer 5mo ago
IIRC that graph tracks capabilities as time_to_solve a task for humans (i.e. the model can now handle tasks that usually take a human ~8h). Which, depending on what tasks you look at, could be a reasonable finding. I could see Opus 4.6 handling tasks that take ~8h for humans, and that 5.1 couldn't previously handle (with 5.1 being "limited" at 4h tasks let's say). It is a bit arbitrary, but I think this is what they're tracking.
- lukan 5mo ago
  "It is a bit arbitrary, but I think this is what they're tracking." I don't know if they can get their numbers right this way, but this seems a way more useful metric, than theoretic capabilities.
  - cyanydeez 5mo ago
    ok, but arn't you just measuring efficiency and not the big I in AGI improvements.
    - 5mo ago

devmor 5mo ago

"Exponentials all tend to become sigmoids but you can't predict exactly when" is a true statement, but I'm not sure it needed an article. This doesn't say much, and the author fights their own points a couple times, suggesting that they maybe didn't think through what they wanted to write until they were in the middle of writing it and started realizing their assumptions didn't match what they expected the data to say. I really don't get the point of what I just read.

aspenmartin 5mo ago
The point is the tiring arguments from AI skeptics saying “things are flattening, they have to” which while technically correct says nothing because no one knows when that will happen and we see no mechanism for this yet. Lindy’s law as a reasonable prediction under total uncertainty is interesting and insightful and a lot of people don’t know about it or why it holds. I did enjoy the reference to this!
- devmor 5mo ago
  But those skeptics are initially responding to the constant AI hype claims that we are exponentially growing to AGI. So this article is in fact just a (very poorly thought through) attempt at saying “nuh uh, the hype might be true, you can’t prove it’s not yet!
  - aspenmartin 5mo ago
    Yet the evidence is on the side of the hype? We don’t see any mechanism or cogent framework for what limits exist here theoretically that I’m aware of, are you? Epoch had a great article a year ago looking at several bottlenecks in terms of scale and back then we were about 4 orders of magnitude away from hitting them. We’re probably now closer to 3. Yet scale is only part of the performance equation, a fairly big chunk of progress is from algorithmic or curation related contributions. The point of the article is: > But those skeptics are initially responding to the constant AI hype claims that we are exponentially growing to AGI. This is a meaningless statement or at best just strawmanning.

addaon 5mo ago

https://xkcd.com/605/ https://xkcd.com/605/

deleted 5mo ago
[deleted]

inglor_cz 5mo ago

Hmmm, this is quite an interesting take by Scott. Lindy's Law is not actually a law and many exact minds will be provoked by the very name; it also fails spectacularly in certain contexts (e.g. lifetime of a single organism, though not necessarily existence of entire species). But at the same time, I am willing to take its invocation in the context of AI somewhat seriously. There is an international arms race with China, which has less compute, but more engineers and scientists. This sort of intellectual arms race does not exhaust itself easily. A similar space race in the 1950s and 1960s progressed from first unmanned spaceflight to a moonwalk in mere 12 years, which is probably less than what it takes to approve a bicycle lane in Chicago now.

krupan 5mo ago
"There is an international arms race with China" I keep seeing this. Where did it come from? Has China said that they intend to attack other countries using AI? Have other countries declared that they intend to attack China with AI? Also, why does anyone believe that AI could actually be that dangerous, given it's inherent unpredictable and unreliable performance? I would be terrified to rely on AI in a life or death situation.
- inglor_cz 5mo ago
  It was a metaphor. I meant, and later clarified, an intellectual arms race. BTW your handle is an actual Czech word, minus a diacritic sign ("křupan"), and a bit amusing one. It basically means hillbilly. Not that it matters, just FYI. Anyway: AI will be used in military context, and it probably already is. Both for target acquisition and maybe even driving the weapon itself. As of now, the Ukrainians are almost certainly operating some AI-enabled killer drones.
  - krupan 5mo ago
    That's funny, I was told my real last name is a swear word in Czech
    - 5mo ago

theturtle 5mo ago

[dead]

krupan 5mo ago

News flash: predicting the future is hard

energy123 5mo ago
The individual who is the best at predicting the future is predicting ASI and full labor automation by 2040: https://xcancel.com/peterwildeford/status/2029636662322446613 https://xcancel.com/peterwildeford/status/202963666232244661...
- gerikson 5mo ago
  Past results is no guarantee of future performance.
- Aurornis 5mo ago
  > The individual who is the best at predicting the future Going to need a big citation for that claim
  - margalabargala 5mo ago
    Source: trust me bro
  - hirvi74 5mo ago
    No need. That man predicted he would be the best at predicting the future.
- margalabargala 5mo ago
  > The individual who is the best at predicting the future Lol
- layer8 5mo ago
  Predicting who will predict the future best is hard.
  - deleted 5mo ago
    [deleted]

kubb 5mo ago

If the scary AI is so inevitable, why do you feel such an overwhelming need to convince people about that? Surely you can just wait a bit, and they'll see for themselves.

adleyjulian 5mo ago
1. It's not inevitable. 2. Those that see AI as an existential risk don't generally think it's a guarantee, but if it's say a 5% chance then that's worth addressing/mitigating. 3. That's not what this article was even about.
- kubb 5mo ago
  Sounds like the burden is on you to explain either 1. If you're not treating my claim as a black box, explain explicitly what is your model of what the article was about? Are you aware, for example of the last paragraph of the article? I think that WAS what the article was about. Do you have specific opinions on e.g. how I went wrong and where my model differs? 2. If you are treating it as a black box, what's your default expectation based on the law of Nothing Ever Happens? Just kidding, you don't need to explain anything. A"I" fearmongers should though.
  - adleyjulian 5mo ago
    The point of the article is that people are historically bad at predicting when exponential curves plateau, even if they're correct that there will be a plateau. This does *not* imply the inevitability of AGI. It does not imply AGI is necessarily bad. It does mean that "the capabilities of AI will eventually plateau" offers no meaningful predictive power or relevance to the overall AI discussion.
mitthrowaway2 5mo ago
By that reasoning, why even warn people about anything? Why do road construction crews put up signs saying "ROAD CLOSED AHEAD" when you can just drive on and see for yourself?

LarsDu88 5mo ago

I think an interesting thing about recent AI developments is that its all happening right as we hit the diminishing returns side of another "exponential that's actually a sigmoid" which is Moore's law. The naive expectation is that AI will slow down b/c Moore's law is coming to an end, but if you really think about the models and how they are currently implemented in silicon, they are still inefficient as hell. At some point someone will build a tensor processing chip that replaces all the digital matmuls with analogue logamp matmuls, or some breakthrough in memristors will start breaking down the barrier between memory and compute. With the right level of research funding in hardware, the ceiling for AI can be very high.

cyanydeez 5mo ago
they already did put a model into the silicon and it's crazy fast. https://chatjimmy.ai/ https://chatjimmy.ai/ I'm pretty sure there's a 3 year design goal starting this year that'll do that to any of the qwen, deepseek, etc models. There's a lot you could do with sped up models of these quality. It might even be bad enough that the real bubble is how much we don't need giant data centers when 80-90% of use cases could just be a silicon chip with a model rather than as you say, bloated SOTA
- clickety_clack 5mo ago
  It would be pretty cool to have interchangeable usb keys with models on them.
- LarsDu88 5mo ago
  And this is an asic that is still operating digitally. Imagine a chip with baked it weights that does its math analogue with 20x reduction in number of circuit elements needed to do a multiplication op. If there's a breakthrough in memristors, you could end up with another 20x reduction in circuit elements (get rid of memory bottlnecks, start doing multiplication ops as log transform voltage addition) The ceiling is ultra high for how far AI can go.
  - mordae 5mo ago

Brendinooo 5mo ago

> then what is their model? My mental model has been 3D computer graphics: doubling the polygon count had huge returns early on but delivered diminishing returns over time. Ultimately, you can't make something look more realistic than real. I don't know what the future holds, but the answer to the question "can LLMs be more realistic than real" will determine much about whether or not you think the curve will level off soon.

the8472 5mo ago
The equivalent bar in this domain would be human intelligence, and we already have growing lists of tasks where machines outperform humans. We even known of natural systems that outperform humans on some metrics, e.g. bird-brains have higher neuron density than ours because evolution had to optimize more for weight.
kilpikaarna 5mo ago
In 3D graphics there's diminishing returns on investment for the technology itself, but the real limit is one of economics. How to create all those assets required by the rendering tech, and make your money back. Preferably while also keeping your customers interested long term, by not becoming risk averse. In the same fashion, LLMs have to pay for themselves to keep the trendlines going. In a whole-systems -sense, mind, not "$2000/month is cheaper than hiring a developer" while the rest of the economy collapses.

btilly 5mo ago

Lindy’s Law is an absolute gem, that I'm keeping. If we don't understand the fundamental limits to any particular kind of trend, our default assumption should be that it will continue for about as long as it has gone on already. We can, in fact, easily put a confidence interval on this. With 90% odds we're not in the first 5% of the trend, or the last 5% of the trend. Therefore it will probably go on between 1/19th longer, and 19 times longer. With a median of as long as it has gone on so far. This is deeply counterintuitive. When we expect something to last a finite time, every year it goes on, brings us a year closer to when it stops. But every year that it goes on properly brings the expectation that it will go on for a year longer still. We're looking at a trend. We believe that it will be finite. Our intuition for that is that every year spent, is a year closer to the end. But our expectation becomes that every year spent, means that it will last yet another year more! How can we apply that? A simple way is stocks. How long should we expect a rapidly growing company, to continue growing rapidly?

jerf 5mo ago
It's an interesting idea, and it may be something that could be mathematically justified, but I do think this is an abuse of Lindy's Law in the absence of such a justification. Per Wikipedia [1]: "The Lindy effect applies to non-perishable items, like books, those that do not have an "unavoidable expiration date"." And later in the article you can see the mathematical formulation which says the law holds for things with a Pareto distribution [2]. I'd want to see some sort of good analysis that "the life span of exponential growth curves" is drawn from some Pareto distribution. I don't think it's completely out of the question. But I'm also nowhere near confident enough that it is a true statement to casually apply Lindy's Law to it. [1]: https://en.wikipedia.org/wiki/Lindy_effect https://en.wikipedia.org/wiki/Lindy_effect [2]: https://en.wikipedia.org/wiki/Pareto_distribution https://en.wikipedia.org/wiki/Pareto_distribution
- btilly 5mo ago
  The analysis in the article explains why it applies to any phenomena that we might be able to notice. The argument given is the same as the one that I first ran across, not by that name, in https://www.nature.com/articles/363315a0 https://www.nature.com/articles/363315a0. https://en.wikipedia.org/wiki/Doomsday_argument https://en.wikipedia.org/wiki/Doomsday_argument claims that it was a rediscovery of something that was hypothesized a decade article. I hadn't tried to give it a name, or thought to apply it outside of that context. As for the mathematical qualms, I'm a big believer in not letting formal mathematical technicalities get in the way of adopting an effective heuristic. And the heuristic reasoning here is compelling enough that I would like to adopt it.

itkovian_ 5mo ago

The other thing people don’t understand is exponential curves are self similar. The start of an exponential looks like an exponential. People always look at and think ‘well that’s it it’s exponential now, have missed it, can’t sustain’. Nope. Good example of this is number of submissions to neurips/icml/iclr. In 2017 that curve was exponential.

zkmon 5mo ago

The curve is a smoothed step curve (y=1 if x>1 otherwise 0). Nature doesn't allow any change to happen instantly at any degree of rate of change. The curveis just a manifestation a change with exponential smoothening of the sharp corners. For example, When a car starts, it's speed and acceleration become more than zero. But what about rate of change in higher degrees? It suddenly doesn't change from zero acceleration to non-zero. That means the car has a non-zero derivative at all degrees. In other words, the movement is exponential. The same thing happens in reverse when the car reaches a constant speed.

patrickmay 5mo ago

Stein's Law: "If something cannot go on forever, it will stop."

skybrian 5mo ago
Yes, but figuring out when is the hard part.

OscarCunningham 5mo ago

John D Cook gives more technical details here: "Trying to fit a logistic curve" https://www.johndcook.com/blog/2025/12/20/fit-logistic-curve/ https://www.johndcook.com/blog/2025/12/20/fit-logistic-curve...

bedobi 5mo ago

[flagged]

simianparrot 5mo ago
Because HN is YCombinator which has invested in probably hundreds of «AI» firms by now. Including OpenAI. Allowing slop articles like this literally prints them evaluation money.
- t43562 5mo ago
  Yes, this is not the place to express skepticism of any kind.
  - simianparrot 5mo ago
    The subtle but consistent downvoting to a score of 0 to -1 feels very botted. No matter how I write the comments, anything counter the "AI" propaganda nets me a 0 or -1 total score. I don't care about the points whatsoever, but I find it interesting how there's never a massive downvote, just enough to keep such comments at 0 to -1. May be reading into things too much, but it is a bit odd.
ngriffiths 5mo ago
I think there are many ways someone with his lack of expertise can still be valuable, including: - Making connections to other subjects that an expert would miss. The hall of fame of sigmoid predictions is just excellent, I already know I'm going to be reminded of it some time in the future. Very entertaining way to get the point across. - Writing about tricky concepts in a very accessible and elegant way, which experts are notoriously bad at doing themselves - they are often optimizing for other specialists. - Being able to write with an air of speculation and experimentation with ideas that experts and institutions often can't afford. Experts have to maintain their track record; Scott Alexander can say "lol just double the timeline"
- bedobi 5mo ago

janalsncm 5mo ago

> What if you don’t fully understand the process? AI forecasters know some things (like how data centers work and how much it costs to build them). But they’re unsure about other things (researchers keep inventing new paradigms of data generation that get over data walls, but for how long?), and other things are entirely opaque (What is intelligence really? Why do scaling laws work? Might they just stop working at some point?) Is there anything you can do here? This is the crux of the article. To a large extent continued progress depends on a stable increase in compute, an increase in training data, and an increase in good ideas to squeeze more out of both of them. One calculation you could do is a survival function: for each of the above, how long before it is disrupted? For example, China could crack down on AI or invade Taiwan. Or data centers become politically unpopular in the US. Or, we could run out of great ideas. Very hard to predict.

dsign 5mo ago

We did hit the sigmoid's plateau on airplane speed, but the applications of airplane speed are still coming (how fast can a Chinese company airship the PCB you ordered three minutes ago?). I expect the the same will happen with LLMs, though I also happen to believe things are just getting started on end capabilities.

jsmcgd 5mo ago

> It’s true that birth rates must eventually flatten out and become sigmoid All positive growth eventually flattens out and becomes sigmoid, but a lot of phenomena experience negative growth and nose dive. No gentle curve, but a hard kink and perfect flat line at zero. Forever. I think it would be a stretch to categorize that pattern as sigmoid. Predicting a sigmoid pattern for negative growth implies some sort of a soft landing (depending on your definition of soft). We can think of many populations that are no longer with us. So just a caution about over applying this reasoning in the negative case.

Qem 5mo ago
> All positive growth eventually flattens out and becomes sigmoid, but a lot of phenomena experience negative growth and nose dive. https://en.wikipedia.org/wiki/Seneca_effect https://en.wikipedia.org/wiki/Seneca_effect

ngruhn 5mo ago

> all exponentials eventually become sigmoids Except innovation. When one sigmoid tapers off we keep finding new ones to keep the climb going.

stymaar 5mo ago

I don't know when the sigmoid is going to kick in, but Nvidia's Quaterly datacenters revenues have been grown 15 folds over the past 3 years[1], and nobody including Scott believes this is sustainable for 3 more years otherwise Nvidia's market cap would conservatively be at least an order of magnitude higher than it is. All exponential eventually becomes a sigmoid because exponential growth always expose limiting factors that weren't limiting at the beginning. Silicon manufacturing had lots of room for high-margin customers like Nvidia even a year ago (by the mere virtue of outbidding lower-margin customers), but now it is mostly gone, and no amount of money will make fabs build themselves overnight. [1]: https://stockanalysis.com/stocks/nvda/metrics/revenue-by-segment/ https://stockanalysis.com/stocks/nvda/metrics/revenue-by-seg...

pyrale 5mo ago

Such a long article to say that neither side has a fucking idea about what will happen next. While we're at it, the "exponentials are actually sigmoïds" meme is not necessarily true. While exponentials are never exponentials, sigmoids are not guaranteed. Overshoot-and-collapse examples also happen in tech, e.g. the dotcom bubble, or the successive AI winters.

andrewflnr 5mo ago
It's really not that long, and is quite clear that its main point is about how to reason when you realize no one actually knows what's going on.

dreambuffer 5mo ago

FYI: The author has predicted that "AGI" will be here in 1-2 years and has staked his public reputation on it. He is personally invested in trendlines being lindy rather than sigmoid. I don't think you can use lindy on trends as if trends are static objects, but that's another conversation.

throwawayk7h 5mo ago
Mind you, he is only personally invested insofar as he's staked his reputation on it. Throughout his writing, he expresses the same point over and over again: desperately wants AI to slow down, advocates for politics that would slow it down, and most likely nothing would bring him greater peace than to see a sigmoid curve appear.
- bombcar 5mo ago
  How convenient; when AGI doesn’t appear in 1-2 years his reputation is pristine because he slowed it down.
  - wmf 5mo ago
    To make that argument you'd want to show some causal link which so far we haven't seen.
  - achierius 5mo ago
    What do you want? This sounds like you have something against people making a claim in public, at all, on any topic of importance.
woeirua 5mo ago
Ok, but you can just look at the METR curve. Mythos saturated the 50% time horizon. The 80% is now at 3 hours. The rate of progress is accelerating not slowing down. There’s no indication yet that this is a sigmoid!
- yorwba 5mo ago

andy99 5mo ago

AI has scaled well according to convenient measures. It (neural networks) have the property that whatever you define, they can rapidly be trained master it. We’re able to show that various tasks of increasing complication do not require intelligence and can be framed as autoregressive RL problems. I personally don’t think AI is any closer to sentient intelligence than LeNet; it’s almost trivially clear, we know how it works. So we’re measuring something orthogonal, basically how well a universal function approximator can fit to a function we define, given arbitrary computing power, and calling that progress. What will be really interesting is if we’re able to find a way to properly measure what they can’t do and what’s different about real intelligence. Edit: in particular I don’t agree with But if someone claims that the trend toward increasing AI capabilities will never reach some particular scary level... One has to agree that the benchmark results are getting “scarier”, which is not automatically implied by finding more goals to optimize for

ordu 5mo ago
> We’re able to show that various tasks of increasing complication do not require intelligence and can be framed as autoregressive RL problems. The important thing we can show it in hindsight only. We don't know which other tasks we are currently mistaken about requiring intelligence. Maybe none of them are? We don't know. We don't know what intelligence is. If we look at decades and even centuries of attempts to define intelligence, it is all looks like a goalposts moving. When a definition of intelligence starts to include people or things we don't like to think as of intelligent ones, we change the definition.
- ryeights 5mo ago
  “AI is whatever hasn't been done yet” — Larry Tesler
- winwang 5mo ago
  Yep. No one bats an eye at eyewitnesses "hallucinating" details, or that I'd rather have Opus as a coworker vs a random middle schooler (err, labor laws notwithstanding). I think perhaps too much of the dialogue around intelligence has to do with the word (and its connotations) itself. The poster you replied to even used the word "sentient", which is quite interesting (warning: opinionated tangent ahead). Merriam-Webster defines it as "capable of sensing or feeling: conscious of or responsive to the sensations of seeing, hearing, feeling, tasting, or smelling". Feels like qualia. Or if we don't want to go the qualia route... Of course, we wouldn't call Helen Keller non-sentient, so presumably we "really" mean "can it sense or feel" -- well, sense is just "act/feel according to the environment", which you could argue in the case of an LLM would be their context... so we should "really" remove "sense" from the definition, probably. So "do LLMs feel" is probably closer to what "sentient" is being used for here. Since we don't have the obvious symmetry of "you are like me and I feel (therefore you probably feel)", it's way better/easier/feel-good-ier to prefer "LLMs don't feel" rather than "oh shit, it feels and model training is actually just torturing it into the right shape". LLMs as fundamentally non-intelligent also avoids the problems of "what does that say about people" or "we may have made 'AGI' and it wasn't what we thought it would be" or "we're not ready to talk about this yet".

baxtr 5mo ago

> The moral of the story is that, even though all exponentials eventually become sigmoids, this doesn’t necessarily happen at the exact moment you’re doing your analysis. Sometimes they stay exponential for much longer than that! All exponentials eventually become sigmoids? Don’t think this can be true without qualifiers.

jvanderbot 5mo ago
All models are wrong, of course, but this is kind of "common sense" so it's not hard to accept as true in a natural system. How can something continue on exponential growth forever without reaching a new blocker that causes slowdown or encountering pushback that makes it an oscillator. A pendulum looks exponential when it is at its peak and accelerating down. The issue is that the exponential-looking part of the sigmoid might contain all of human history, sure, but most folks who espouse this theory probably agree that over time everything reaches a steady-enough state to be considered non-exponential, or become oscillatory.

jrflowers 5mo ago

I like this article about how we should assume, at any given point, that we are exactly halfway through a phenomenon which relies on a single data point on a graph —-that apparently doesn’t need its relevance or importance explained— to illustrate that this is obviously true for AI in particular

Smaug123 5mo ago
I think you may have read a different article from me. The thesis of the article is summarised at the end: > But if someone claims that the trend toward [X] will never reach some particular scary level, then the burden is on them to explain either: > If they’re not treating [X] as a black box, and claim to be modeling the dynamics explicitly, then what is their model? Have they calculated the obvious things… > If they are treating [X] as a black box, why isn’t their default expectation based on Lindy’s Law? Like, the whole point is that in real life we do actually know things about situations and can model them; we fall back to Lindy's law when we know nothing at all. Further, arguments have justification to deviate from Lindy only when they give specifics about the situation they're modelling.

noosphr 5mo ago

This article answers the question in the second paragraph then completely ignores the answer for the rest of it. >My understanding is that this represents 3-4 “generations” of different technology (propellers, turbojets, etc). Each technology went through normal iterative improvement, then, when it reached its fundamental limits, got replaced by a better technology. The last technology, ramjets, reached its limit at about 3500 km/h, and there wasn’t the economic/regulatory will to develop anything better, so the record stands. You don't have one sigmoid, you have multiple each stacked on top of each other. Airplanes aren't just one technology they are multiple technologies that happen to do the same thing. Each one is following a sigmoid perfectly. It only looks exponential(ish) because of unpredictable discoveries that let you switch to another sigmoid that has a higher maximum potential. The same is true in AI. If you used the same architecture as GPT2 today you're in for a bad time training a new frontier model. It's only because we have dozens of breakthroughs that the capabilities of models have improved as much as they have. That said exponential and sigmoids are the wrong model to use for growth. Growth is a differential equation. It has independent inputs, it has outputs and some of those outputs are dependent inputs again through causal chains of arbitrary complexity. What happens depends entirely on what the specific DE that governs the given technology is. We can easily have a chaotic system with completely random booms and busts which have no deep fundamental rhyme or reason. We currently call that the economy.

Sniffnoy 5mo ago
Yes, I was surprised he never discussed the idea that such exponentials are typically made of stacked sigmoids. That said... if the exponential is made of stacked sigmoids, it's still an exponential on the whole! The fact that it's made of stacked sigmoids is relevant to the engineers making it, but not so relevant to the users or those otherwise affected by it.
- noosphr 5mo ago
  Only so long as you can keep inventing the next sigmoid in the stack.
  - 5mo ago

pron 5mo ago

1. Scott Alexander is famous for writing about topics he knows little about. I'm glad to see he's found a subject he knows little about but so does everyone else. 2. What's even worse than predicting that some growth curve flattens before X happens is predicting it will flatten before X happens but after Y happens, which is what we see when it comes to AI in software development. Too many people predict that AI will be able to effectively write most software, replacing software engineers, yet not be able to replace the people who originate the ideas for the software or the people who use them. I see no reason why AI capability growth should stop after the point it's able to write air-traffic control or medical diagnosis software yet before the point where it's able to replace air traffic controllers and doctors. 3. While we don't know much about AI (or, indeed, intelligence in general), we do know something about computational complexity. Some predictions about "scary things" happening (the ones I'm guessing Alexander is alluding to, though I can't be certain) do hit known computational complexity limits. Most systems affecting people are nonlinear (from weather to the economy). Predicting them requires not intelligence but computational resources. Controlling them, similarly, requires not intelligence but either computational resources or other resources. It's possible that people choose to give control over resources to computers (although probably not enough to answer many tough, important questions), although given how some countries choose to give control to people with below-average intelligence (looking at you, America), I don't see why super-human intelligence (if such a thing even exists) would be, in itself, exceptionally risky.

ryeights 5mo ago
>1. Scott Alexander is famous for writing about topics he knows little about. I'm glad to see he's found a subject he knows little about but so does everyone else. This is kinda laughable. Scott has been thinking and writing about AI for a long time
- pron 5mo ago
  That's what I'm saying. I'm glad to see that he's found a subject where his lack of knowledge isn't a glaring handicap. When I come across his posts I usually feel uncomfortable because they read like a bright 4-year-old child trying hard to explain how a car works, only not cute (yeah, I know that trying to see what conclusions you can come to, Aristotle-like, from a basis of ignorance and without careful study is the whole point, but I never found this Memento-style game appealing).

leoc 5mo ago

Hmm. What’s the general belief about Toby Ord’s “Are the Costs of AI Agents Also Rising Exponentially?” https://www.tobyord.com/writing/hourly-costs-for-ai-agents https://www.tobyord.com/writing/hourly-costs-for-ai-agents among those who are well-equipped to judge? Is it seen as wrong or disproven or unlikely? Because if not—if indeed recent LLM capability advances have likely relied on increases in inference cost per run which can’t be much further sustained—then it seems remiss not to mention that if you point to those advances to claim that the exponential trend remains on track.

lovich 5mo ago

I wonder how the graph would look like if cost and/or profitability was taken into account. I could probably make increasingly larger fires for years if I was willing to burn the entire world.

whatshisface 5mo ago

If you want a model, here's one: LLMs have never demonstrated the ability to go obviously beyond interpolating their training data. It takes an army of paid data producers solving homework problems to give ChatGPT the ability to do your homework. All vibecoded apps that turned out to be successful could put on a geological soil chart with other apps, probably on GitHub somewhere, on the corners. The prediction? They won't. In this model, the exponential growth that everybody is freaking out about is only the realization of the modular software dream ("we'll only have to write an ORM once for all of human history!") and the sheer amount of knowledge in libraries. It's at least falsifiable.

boxed 5mo ago
Just to play devils advocate: are we sure humans have demonstrated the ability to go beyond their training data? Like.. are we sure-sure about that?
- 1attice 5mo ago
  Idk, I mean, Shakespeare never read Shakespeare, so, I mean, unless aliens?
  - boxed 5mo ago
    AI images are all unique and new.
    - 1attice 5mo ago
      Not nearly as new as Shakespeare was when he single handedly reinvented English vocabulary, for example.
      - boxed 5mo ago
        He accidentally formalized spelling. You can't invent vocabulary when you're writing plays for a mass audience and want the audience to actually understand what is being said. That makes no sense.

stego-tech 5mo ago

I felt the better takeaway from this was that it's impossible to know for certainty how long this will or will not continue regardless of the data or models you're using, because if you (or anyone else) could predict that accurately they'd be one of the richest people on the planet. I don't know when (or if) AI will implode or succeed with any degree of provable certainty, because that's not my area of expertise. Rather, I can point out and discuss flaws in the common booster and doomer arguments, and identify problems neither side seems willing to discuss. That brings me cold comfort, but it's not enough to stake my money on one direction or another with any degree of certainty - thus I limit my exposure to specific companies, and target indices or funds that will see uplift if things go well, or minimize losses if things go pear-shaped. I also think relying on such mathematics to justify a position in the first place is kind of silly, especially for technical people. Mathematical models work until they don't, at which point entirely new models must be designed to capture our new knowledge. On the other hand, logical arguments are more readily adapted to new data, and represent critical, rather than mathematical, thinking and reasoning. Saying AI is going boom/bust because of sigmoids or Lindy's Law or whathaveyou is not an argument, it's an excuse. The real argument is why those things may or may not emerge, and how do we address their consequences within areas inside and outside of AI through regulation, innovation, or policy.

bjackman 5mo ago
I think his agenda here is to point out that your probability distribution for AI outcomes should be broad (what you said), but most importantly: this means you must take seriously the possibility that we are gonna get superintelligence quite soon. Basically a lot of people say "but isn't it also pretty likely that we DON'T get superintelligence?" And, yes, it is. But superintelligence being even a remotely plausible outcome is a big fucking deal. Your investment choices in that context are not important. People really struggle to think rationally in the face of this shape of uncertainty.
- stego-tech 5mo ago
  You’re 100% correct, which is why I opted for a broad investment approach rather than trying to pick “winners”. My thought process RE: superintelligence/AGI is generally this: * I personally don’t believe it’s likely to happen with silicon-based computing due to the immense power and resource costs involved just to get to where we are now; hence why I invest broadly to capitalize on what gains we actually attain using this current branch of AI research across all possible sectors and exposure rates * If we do achieve AGI using silicon-based computing, its limited scale (requiring vast amounts of compute only deliverable via city-scale data centers) will limit its broader utility until more optimizations can be achieved or a superior compute platform delivered that improves access and dramatically lowers cost; again, investing broadly covers a general uplift rather than hoping for a specific winner * If AGI is achieved, nobody - doomer or booster alike - will know what comes next other than complete and total destruction of existing societal structures or institutions. The stock market won’t explode with growth so much as immediately collapse from the disintegration of the consumptive base as a result of AGI quite literally annihilating a planet’s worth of jobs and associated business transactions. In this case, a broad spread protects me from harm by spreading the risk around; AGI will annihilate the market globally, but not all at once barring a significant global catastrophe instigated by it * Which brings me to the worst outcome, where AGI follows the “if anybody builds it everyone dies” thought process: investment is irrelevant because we’re all fucked anyway. And that’s just my investment approach. I’m too pragmatic to believe we’re at the bottom of the sigmoid curve, but too wise to begin guessing where we actually exist on it at present or how much is left in the current LLM-arm of AI research; I’m an IT dinosaur, not an AI scientist. What I can point to is the continued demand destruction of consumer compute through higher costs and limited availability due to rampant AI speculation as proof that the harm is already here in a manner most weren’t predicting, while at the same time actual job displacement by AI is limited to the empty boasting of executives using it as a smoke screen for layoffs after RTO mandates failed to thin headcount sufficiently. In the USA in particular, we’re facing a perfect storm of: * consumer confidence collapse leading to a decline in spending on all goods, especially luxury ones, by all but the most monied demographics * data center-driven cost increases (energy) and resource destruction (land, water, fossil fuel use) * the eradication of government support for renewable energy that would’ve kept these costs in check * the widening wealth gaps creating a new underclass not seen since before WW2 In other words, most of the discourse continues to revolve around hypotheticals of tomorrow rather than realities of today. That would be the lesson I’d hope more people take away from something like this, so we can finally begin addressing issues themselves rather than empty online circle jerking about who is right or wrong.

dnnddidiej 5mo ago

Tldr you cant accurately predict the future of complex systems. Corollary... you cant accurately predict the future of complex systems using sigmoid. Attention is all you need took us by surprise and we don't know how big the wave is let alone if there are other waves behind it.

graphememes 5mo ago

line can go up, line can go sideways, line can go up sideways, line can go up sideways up, line go where line go

jasongi 5mo ago

First sigmoid was transformers allow us to rapidly scale to our already abundant data until we tapped it out, the second is/was reasoning, allowing us to scale to our available compute (and compute manufacturing capacity). Correct me if I'm wrong but we don't have candidate for the third sigmoid, and scaling inference is hitting real-world supply chain constraints - electricity and chips. Short of a third sigmoid appearing in the ML CompSci space, perhaps in the form of ongoing, repeated step-optimisations which will also have diminishing returns, intelligence growth is now limited a few scaling problems that have already been worked on for a very long time. Transistors, which have been doubling for almost a century now, but Moores Law has already plateaued and reached limits on energy efficiency, and simply building new fabs is not something that we can do exponentially. And the other growth limiter is electricity - there is no exponential supply of fossil fuels or power plants. Although manufacturing has scaled, PV tech improvements are also plateauing - and while storage is getting cheaper, it's still not economical vs fossil fuels (meaning: when we have to switch to it, the growth slows down further) and we are unlikely to see battery efficiency sigmoid enough to maintain the AI sigmoid. I don't mean to be bearish here. There's so much money sloshing around that we can afford to put the smartest people, using unlimited tokens, on the task of finding small, incremental gains on the CompSci side of things that will have large monetary payoffs - hopefully allowing further scaling and increased emergent abilities of LLMs. Maybe we can squeeze the algos for quite a while. But I don't see that maintaining the same level of exponential as unlocking unlimited data or maxxing out the world's energy/fab capacity for long. And I don't see why this is a massive issue except for the people who want to have some god-like super AI? Frontier LLMs are genuinely magic. Not "won't delete your production database" magic, but definitely a massive productivity gain for competent knowledge workers.

foobar10000 5mo ago
We are multiple orders of magnitude away from Landauer limits - so next big thing in matmul could be photonic multipliers - there’s a bunch of them coming up in the next 3? years. So that’s a 2-4 order of magnitude improvement. Sigmoid?
ianm218 5mo ago
Reinforcement learning has become a huge portion of compute used during training runs [1] and synthetic data is letting us get lots more mileage out of the existing data. Additionally, there is lots of new, high quality data being created and collected each day. I think the "running out of data" thing was pretty poorly reported by mainstream media. [1]. https://www.dwarkesh.com/p/dario-amodei-2 https://www.dwarkesh.com/p/dario-amodei-2

mgraczyk 5mo ago

Don't bet on the sigmoid flattening out any time soon. I don't know when it will flatten, but it definitely won't be within the next 2 years

overgard 5mo ago

This feels like a really verbose way of saying "things have been growing fast for a while so they should continue to grow as fast for just as long", and then he places the burden on people to prove him wrong. Um, no, the burden of proof is shared, "this will just keep going" requires just as much proof as "this is going to level off" if you're just looking at trend lines. It's better to look at the underlying factors. Money sources are drying up, nobody is making a profit outside of nVidia, most blackwell GPU's are likely not even installed yet and will probably be 2 generations behind when they finally are being used, data centers are hitting all sorts of obstacles getting built and powered and they're getting built slowly, most AI researchers seem to think that LLMs are a dead end, the newer models seem to be getting more expensive and sometimes worse, or even potentially are showing signs of model collapse (goblins..), the supposed productivity gains are not materializing.. AI has worse public sentiment than congress.. I could keep going. Some obscure "law" seems to pale in comparison to the hard evidence that the status quo is utterly unsustainable and none of these companies seem to have a realistic plan other than trying to become too big to fail essentially. I like some of this guy's writing on other topics, but to me this is a prime example of what happens when you get public "intellectuals" talking about subjects far outside of their area of expertise. It's not as bad as Richard Dawkins latest fall into psychosis but it's basically the same phenonmenon.

plomme 5mo ago

I’m not saying he’s wrong about the core thesis here, but using Claude Opus 4.6 as a “mic drop” with a chart showing it being twice as good as the last model feels in my experience way off.

Permit 5mo ago

> But if someone claims that the trend toward increasing AI capabilities will never reach some particular scary level, then the burden is on them to explain either:… This is not the context in which I hear about sigmoids vs exponentials. I hear it in regards to “the singularity”, not that AI won’t reach some pre-specified level. You may get AGI, you aren’t getting a singularity.

izucken 5mo ago

Births is, sans miscalculation, a number that tracks exact events. Is the "capability" number on these LLM strengh graphs as tangible? I think it would be interesting to visit a reality that obeys arbitrary abstractions, but I would personally never go there.

mordae 5mo ago

I am 100% certain it's a sigmoid. Speed of light is finite. :-)

niemandhier 5mo ago

If you look at problems that can be solved by reasoning in text form or maybe even images, I am more than willing to accept that we simply cannot know when the curve will level. The situation is drastically different for problems that require interaction was the physical world to determine success. As soon as you add a powerful simulator for physical problems to the self learning experience of the AI, you are extremely hampered by the large amount of needed computation.

mentalgear 5mo ago

> Why do scaling laws work? Strictly speaking, the original paradigm of scaling laws doesn't work any more. The assumption that we could achieve better performance simply through "vertical scaling" ie infusing models with exponentially more parameters and pre-training data, is no longer the driving force of AI progress. Instead, the industry has pivoted toward inference-time scaling. Rather than relying solely on a massive, static neural network, modern architectures allocate more compute during the actual generation process, allowing the model to "think" and verify its logic dynamically. Furthermore, the latest state-of-the-art models are no longer pure LLMs; they are compound neuro-symbolic systems that integrate external tools like REPLs, databases, and structured skill documentation to archive things pure LLM vertical parameter scaling was not able to do.

yorwba 5mo ago
The "law" part of scaling laws is about predicting validation cross-entropy loss from the training configuration, analogous to physical laws allowing to predict one quantity based on the measurement of another. Most scaling laws take the form of an irreducible error plus additional terms that asymptotically decay to zero. So that there is a wall you can approach but not cross (the irreducible error) is an integrated part of the scaling law paradigm. That it isn't economical to keep increasing model size to squeeze out a few more drops of cross-entropy doesn't mean scaling laws stopped working. Strictly speaking, "Why do scaling laws work?" is a question about the theoretical reasons the asymptotic decay takes the particular mathematical shape that it does.

Staross 5mo ago

You should really do a Bayesian fit for such predictions and give confidence intervals, it would probably show that the uncertainty is very high in these cases.

amelius 5mo ago

The exponentials won't save you either. In fact it's more likely that a sigmoid takes over at some point.

AvAn12 5mo ago

Forecasts are a thought exercise, not the revelation of something foretold. Best thing to do is think of the outcome you wish for and then try to take whatever actions you can to help make it so. Like with climate change for example.

torginus 5mo ago
It can be both. In the field of high-technology (like semiconductor) manufacturing, the techinques of making chips with feature sizes many generations ahead of current SOTA exist in various states of completion, so extrapolation can be made from that. But what's going on in here is not that - it's reading tea leaves for maximum dramatic effect.

FrustratedMonky 5mo ago

Black Swan Events probably look like nothing just before they happen.

theuniverseson 5mo ago

[flagged]

tim-projects 5mo ago

Does hype follow a sigmoid curve?

dwa3592 5mo ago

I am not disagreeing with the conclusion of the article but the way they reached the conclusion is problematic (and wrong) and here is my nuanced explanation of that : There is no attention paid to the origin (0,0) in any of the plots in the article. To see a meaningful trend you have to be be able to zoom out and zoom in, which means your origin is not a fixed point, ever. This also means that plots with fixed point origins don't tell us anything important at a societal level. For example: the flight airspeed plot, it starts at ~1900. Now of course it should start there bc we did not have planes before that. But let's change the plot heading from airspeed to human speed a.k.a at what speed could humans move. now you can change the origin meaningfully as we had chariots, horses, bicycles, ships before that. If you instead create a plot for last 5000 years you would see that the speed at which humans are able to move is rising exponentially, from walking on foot in a radius of 1000-2000feet in a thick jungle 5000 years ago to reentering earth at 25000 miles/hour in 1969 (yeah, read that again). Even for AI, if you zoom out the plot to last 70 years it will look exponential, if you zoom in to last 2 months it will look absolutely flat. The point is that the whole sigmoid/exponential argument is a function of the origin (0,0).

naorsabag 5mo ago

[flagged]

fph 5mo ago

As a wise man once said, "anyone who believes exponential growth can go on forever in a finite world is either a madman or an economist".

morpheos137 5mo ago

The Yudkowskian 'Bayesians' are a scifi cult. Intelligence as it exists in nature is neither arteficial nor general nor 'super.' All human technology is natural. It certainly is not supernatural.

jgalt212 5mo ago

There are practitioners and pontificaters, and never the twain shall meet.