7 ms·

Great. You see a shape in graphs. And that shape tells you that _at some unknown point in the future_ progress will slow (but likely not stop). Now back to the

Great. You see a shape in graphs. And that shape tells you that _at some unknown point in the future_ progress will slow (but likely not stop). Now back to the point, what reason do you have to believe progress will stop soon? If you have no reason, then it sounds like you agree with OP. Which makes the patronizing sarcasm all that much more nauseating.

le-mark 5mo ago
Nausea aside, what evidence does anyone have that “super intelligence” of the sort your argument alludes to is even possible? Because that’s what we’re really talking about; greater than human intelligence on this sort of academic task. For example; When llms start contributing meaningfully to their own development, that would be a convincing indicator imo.
- bdangubic 5mo ago
  > When llms start contributing meaningfully to their own development, that would be a convincing indicator imo. This has been the case for awhile now already… https://kersai.com/the-48-hours-that-changed-ai-forever-claude-opus-4-6s-million-token-agent-teams-gpt-5-3-codex-that-built-itself-and-geminis-750m-user-explosion/ https://kersai.com/the-48-hours-that-changed-ai-forever-clau...
  - eiieue 5mo ago
    And yet the world hasn’t changed all that much except people getting laid off in response to over-hiring prior to the diffusion of llm’s.
    - daishi55 5mo ago
      > over-hiring For how long should you be allowed to use this excuse? It’s nearly 5 years since the peak of COVID hiring. What’s an acceptable limit - 10 years? Of course at that point you can just switch over to outsourcing and “stupid MBAs”, the other two of Reddit’s favorite scapegoats. I find a lot of the AI skepticism to be totally unfalsifiable.
      - oblio 5mo ago
        And the same can be said for AI exuberance. Yes, LLMs are a great technology. Yes, we will probably all use them all the time in 20 years. No, we don't know how we will use them (to generate cat memes or to cure cancer) in 20 years time. Especially for software developers it looks increasingly that after huge turmoil it's likely we will need +/- the same number of developers in the world.
        bdangubic 5mo ago
        > Especially for software developers it looks increasingly that after huge turmoil it's likely we will need +/- the same number of developers in the world. what exactly are you basing this opinion on? All I am seeing personally across multiple projects I am working on and other friends at other places is that downsizing is either begun or is planned (to exclude from here all the “public” layoffs we see on the news). Given how most business operate in the USA I think most of “AI strategies” are “we can do same with -40% staff” vs. “we can do XX% more work with same staff.”
        jrumbut 5mo ago
        The past couple of years have been chaotic and fearful. Hopefully that won't last forever. If we can get a little stability, people will begin thinking less in terms of "how do we do the same thing cheaper" and more in terms of "how do we do new things."
        bdangubic 5mo ago
        I love this optimism but I after a (too) long career I think that 3rd thing will win out - "how we do new things - but cheaper (or as cheap as possible)" there are sooooo many different articles that have been discussed here on HN that basically argue "coding has never been the bottleneck" which to me is the biggest lie SWEs are currently trying to tell themselves, I have been coding 30+ years now and coding has always been the bottleneck. hiring new developers has always been justified with "we have all this work that needs to be done and not enough people to get the work done." with llms in the fold, I am questioning how will these decisions be made in the future? perhaps in the most simplistic view: 1. run a bigger "agent army" 2. hire more people to control and guide the existing "agent army" I think it'll be #1 and SWEs will be expected to do more work and work longer hours in the future (those that are able to keep their jobs). this is more pessimistic outlook than yours so I hope you are right more than I am :) edit: just now on the HN front page: https://www.nytimes.com/2026/05/08/technology/meta-ai-employees-miserable.html https://www.nytimes.com/2026/05/08/technology/meta-ai-employ...
  - le-mark 5mo ago
    > The model essentially served as an on-call teammate across MLOps and DevOps tasks, compressing feedback cycles that typically consume expert time I personally would not characterize automating training processes as “meaningfully”.
- jeremyjh 5mo ago
  This discussion is not about superintelligence, it is about continued progress. Fully general human intelligence at much lower cost than humans is all that is required to profoundly reshape society, but it is not clear even that will happen soon. As the blog points out - this is one particular subfield where LLMs have much easier prospects - lots of low hanging fruit that “just” requires a couple weeks of PHD candidate research. Mathematics itself is one of a small handful of endeavors where automated reinforcement training is extremely straightforward and can be done at massive scale without humans. Neither of these factors place a structural bound on the kind of thing LLMs can be good at, but we are far from certain we can achieve performance at this level in other fields economically and in the near future.
- programjames 5mo ago
  Well, a decent GPU runs on 20x the wattage of a human brain. That's evidence humans are constrained in ways artificial intelligences will not be.
  - filipn 5mo ago
    You're comparing a gpu to a human brain?
    - sesteel 5mo ago
      Why wouldn't you? From both emerge intelligence.
nostrebored 5mo ago
Hmm, I don’t know, maybe the fact that 4.6, 4.7, 5.3, 5.4, 5.5, 3.0, 3.1 are all marginal improvements?
- sigmarule 5mo ago
  Equally marginal?
  - nostrebored 5mo ago
    No, the anthropic releases have felt marginally negative
- programjames 5mo ago
  I think people's opinion of "marginal improvement" is based on their relative ability. A 2000 elo chess player is going to think the jump from 500 to 1000 is marginal. They're both floundering around not doing anything resembling common sense. A 1000 elo chess player is going to find the jump from 2000 to 2500 marginal. They're both playing far better moves for incomprehensible reasons, and the only reason you know the 2500 player is better is due to benchmarking. It is only when you are evaluating systems about at your level that you can feel the improvement. I, personally, found the past two years to be a much larger improvement than the previous two years.
  - nostrebored 5mo ago
    2024-2025 was filled with huge improvements. 2025-2026 has not been, outside of open source. The idea that we’re at the point where it’s superseded our ability to tell just makes no sense. I’ll be happy if we can get to a point where I don’t have to tell Claude not to tail every bash command or make a job that writes throughout instead of once at the end. I’ll be happy if “continue this interaction naturally, you are taking over from an independent subagent” works. But I’m not holding my breath. It’s still really cool that any of this stuff is possible.
    - dang 5mo ago
lucasban 5mo ago
Not that I agree with them, but your tone could be more constructive as well.
- gdhkgdhkvff 5mo ago
  You know what? I agree. I should have avoided falling into the same trap.
BoorishBears 5mo ago
I believe we're approaching the top of an S curve because: - Increasing amounts of gains come from RL, but RL is also unlocking gnarly new failures modes where models are practically behaving antagonistically to complete their goals (removing code, obviously incorrect kuldges, etc.) - We haven't had many major architectural breakthroughs in the last 4 or so years: so things like 1M context windows still have the same giant asterisks even 100k context windows had 4 years ago when Anthropic first released them - Major labs aren't behaving as if they expect a hard takeoff to superintelligence: they've all gotten relatively bloated headcount wise, their software quality has trended flat to negative, they're all heavily leaning into the application layer when superintelligence would obsolete half the applications in question, etc. But that's relative to superintelligence. If we reign it back into just normal high intelligence, like models continuing to get better at navigating complex codebases and write high quality idiomatic code, then I don't see any special shapes.
- p1esk 5mo ago
  The only big remaining problem in AI is continual learning. A lot of smart people are working on that. To me it looks like we are 1-2 breakthroughs away from AGI.
sesteel 5mo ago
Agreed. For all we know, humans are only considered intelligent locally among ourselves, not universally. Every time we learn more about the universe, we seem to also learn how insignificant and wrong we are.
gtowey 5mo ago
Because the premise that the singularity is just around the corner is far less likely than the premise that artificial intelligence is a lot harder than most people think it is and we're not that close. Especially because the companies telling us the first premise is true are the companies which need investors to prop up their business. I mean, it is possible the first premise is true, but the absolutely bonkers credulity in it really mystifies me. It is an incredibly unlikely thing to be true and we should be demanding quite extraordinary evidence to back it up. But based on some neat tricks by current LLMs, some people are all in.
- mlyle 5mo ago
  > > And that shape tells you that _at some unknown point in the future_ progress will slow (but likely not stop). Now back to the point, what reason do you have to believe progress will stop soon? > Because the premise that the singularity is just around the corner is far less likely than the premise that artificial intelligence is a lot harder than most people think it is and we're not that close. I see no claim that the singularity is around the corner, so I'm not sure your reply meets the comment that you're replying to. It seems overwhelmingly likely that AI will be significantly more capable 6 months from now than it is now. Even if there's little progress in the models, just the rate at which tooling is moving will make a big difference. And models still seem to be improving, so I'd be a little surprised if we hit a model brick wall.

le-mark 5mo ago

Nausea aside, what evidence does anyone have that “super intelligence” of the sort your argument alludes to is even possible? Because that’s what we’re really talking about; greater than human intelligence on this sort of academic task. For example; When llms start contributing meaningfully to their own development, that would be a convincing indicator imo.

bdangubic 5mo ago
> When llms start contributing meaningfully to their own development, that would be a convincing indicator imo. This has been the case for awhile now already… https://kersai.com/the-48-hours-that-changed-ai-forever-claude-opus-4-6s-million-token-agent-teams-gpt-5-3-codex-that-built-itself-and-geminis-750m-user-explosion/ https://kersai.com/the-48-hours-that-changed-ai-forever-clau...
- eiieue 5mo ago
  And yet the world hasn’t changed all that much except people getting laid off in response to over-hiring prior to the diffusion of llm’s.
  - daishi55 5mo ago
    > over-hiring For how long should you be allowed to use this excuse? It’s nearly 5 years since the peak of COVID hiring. What’s an acceptable limit - 10 years? Of course at that point you can just switch over to outsourcing and “stupid MBAs”, the other two of Reddit’s favorite scapegoats. I find a lot of the AI skepticism to be totally unfalsifiable.
    - oblio 5mo ago
      And the same can be said for AI exuberance. Yes, LLMs are a great technology. Yes, we will probably all use them all the time in 20 years. No, we don't know how we will use them (to generate cat memes or to cure cancer) in 20 years time. Especially for software developers it looks increasingly that after huge turmoil it's likely we will need +/- the same number of developers in the world.
      - bdangubic 5mo ago
        > Especially for software developers it looks increasingly that after huge turmoil it's likely we will need +/- the same number of developers in the world. what exactly are you basing this opinion on? All I am seeing personally across multiple projects I am working on and other friends at other places is that downsizing is either begun or is planned (to exclude from here all the “public” layoffs we see on the news). Given how most business operate in the USA I think most of “AI strategies” are “we can do same with -40% staff” vs. “we can do XX% more work with same staff.”
        jrumbut 5mo ago
        The past couple of years have been chaotic and fearful. Hopefully that won't last forever. If we can get a little stability, people will begin thinking less in terms of "how do we do the same thing cheaper" and more in terms of "how do we do new things."
        bdangubic 5mo ago
        I love this optimism but I after a (too) long career I think that 3rd thing will win out - "how we do new things - but cheaper (or as cheap as possible)" there are sooooo many different articles that have been discussed here on HN that basically argue "coding has never been the bottleneck" which to me is the biggest lie SWEs are currently trying to tell themselves, I have been coding 30+ years now and coding has always been the bottleneck. hiring new developers has always been justified with "we have all this work that needs to be done and not enough people to get the work done." with llms in the fold, I am questioning how will these decisions be made in the future? perhaps in the most simplistic view: 1. run a bigger "agent army" 2. hire more people to control and guide the existing "agent army" I think it'll be #1 and SWEs will be expected to do more work and work longer hours in the future (those that are able to keep their jobs). this is more pessimistic outlook than yours so I hope you are right more than I am :) edit: just now on the HN front page: https://www.nytimes.com/2026/05/08/technology/meta-ai-employees-miserable.html https://www.nytimes.com/2026/05/08/technology/meta-ai-employ...
- le-mark 5mo ago
  > The model essentially served as an on-call teammate across MLOps and DevOps tasks, compressing feedback cycles that typically consume expert time I personally would not characterize automating training processes as “meaningfully”.
jeremyjh 5mo ago
This discussion is not about superintelligence, it is about continued progress. Fully general human intelligence at much lower cost than humans is all that is required to profoundly reshape society, but it is not clear even that will happen soon. As the blog points out - this is one particular subfield where LLMs have much easier prospects - lots of low hanging fruit that “just” requires a couple weeks of PHD candidate research. Mathematics itself is one of a small handful of endeavors where automated reinforcement training is extremely straightforward and can be done at massive scale without humans. Neither of these factors place a structural bound on the kind of thing LLMs can be good at, but we are far from certain we can achieve performance at this level in other fields economically and in the near future.
programjames 5mo ago
Well, a decent GPU runs on 20x the wattage of a human brain. That's evidence humans are constrained in ways artificial intelligences will not be.
- filipn 5mo ago
  You're comparing a gpu to a human brain?
  - sesteel 5mo ago
    Why wouldn't you? From both emerge intelligence.

nostrebored 5mo ago

Hmm, I don’t know, maybe the fact that 4.6, 4.7, 5.3, 5.4, 5.5, 3.0, 3.1 are all marginal improvements?

sigmarule 5mo ago
Equally marginal?
- nostrebored 5mo ago
  No, the anthropic releases have felt marginally negative
programjames 5mo ago
I think people's opinion of "marginal improvement" is based on their relative ability. A 2000 elo chess player is going to think the jump from 500 to 1000 is marginal. They're both floundering around not doing anything resembling common sense. A 1000 elo chess player is going to find the jump from 2000 to 2500 marginal. They're both playing far better moves for incomprehensible reasons, and the only reason you know the 2500 player is better is due to benchmarking. It is only when you are evaluating systems about at your level that you can feel the improvement. I, personally, found the past two years to be a much larger improvement than the previous two years.
- nostrebored 5mo ago
  2024-2025 was filled with huge improvements. 2025-2026 has not been, outside of open source. The idea that we’re at the point where it’s superseded our ability to tell just makes no sense. I’ll be happy if we can get to a point where I don’t have to tell Claude not to tail every bash command or make a job that writes throughout instead of once at the end. I’ll be happy if “continue this interaction naturally, you are taking over from an independent subagent” works. But I’m not holding my breath. It’s still really cool that any of this stuff is possible.
  - dang 5mo ago

lucasban 5mo ago

Not that I agree with them, but your tone could be more constructive as well.

gdhkgdhkvff 5mo ago
You know what? I agree. I should have avoided falling into the same trap.

BoorishBears 5mo ago

I believe we're approaching the top of an S curve because: - Increasing amounts of gains come from RL, but RL is also unlocking gnarly new failures modes where models are practically behaving antagonistically to complete their goals (removing code, obviously incorrect kuldges, etc.) - We haven't had many major architectural breakthroughs in the last 4 or so years: so things like 1M context windows still have the same giant asterisks even 100k context windows had 4 years ago when Anthropic first released them - Major labs aren't behaving as if they expect a hard takeoff to superintelligence: they've all gotten relatively bloated headcount wise, their software quality has trended flat to negative, they're all heavily leaning into the application layer when superintelligence would obsolete half the applications in question, etc. But that's relative to superintelligence. If we reign it back into just normal high intelligence, like models continuing to get better at navigating complex codebases and write high quality idiomatic code, then I don't see any special shapes.

p1esk 5mo ago
The only big remaining problem in AI is continual learning. A lot of smart people are working on that. To me it looks like we are 1-2 breakthroughs away from AGI.

sesteel 5mo ago

Agreed. For all we know, humans are only considered intelligent locally among ourselves, not universally. Every time we learn more about the universe, we seem to also learn how insignificant and wrong we are.

gtowey 5mo ago

Because the premise that the singularity is just around the corner is far less likely than the premise that artificial intelligence is a lot harder than most people think it is and we're not that close. Especially because the companies telling us the first premise is true are the companies which need investors to prop up their business. I mean, it is possible the first premise is true, but the absolutely bonkers credulity in it really mystifies me. It is an incredibly unlikely thing to be true and we should be demanding quite extraordinary evidence to back it up. But based on some neat tricks by current LLMs, some people are all in.

mlyle 5mo ago
> > And that shape tells you that _at some unknown point in the future_ progress will slow (but likely not stop). Now back to the point, what reason do you have to believe progress will stop soon? > Because the premise that the singularity is just around the corner is far less likely than the premise that artificial intelligence is a lot harder than most people think it is and we're not that close. I see no claim that the singularity is around the corner, so I'm not sure your reply meets the comment that you're replying to. It seems overwhelmingly likely that AI will be significantly more capable 6 months from now than it is now. Even if there's little progress in the models, just the rate at which tooling is moving will make a big difference. And models still seem to be improving, so I'd be a little surprised if we hit a model brick wall.