3 ms·

Do you have specific and objective examples of things people got right? From my perspective there’s very slow but very real progress happening in the AI space.

by Retric 4mo ago

Do you have specific and objective examples of things people got right? From my perspective there’s very slow but very real progress happening in the AI space. I see people making wild predictions in both directions, but in terms of actual unsupervised utility there’s definitely progress abet wildly slower than most hype.

MattRix 4mo ago
The idea that progress is “slow” in the AI space is absurd. These are some of the fastest growing products and companies of all time. The models are still improving a surprising amount.
- Retric 4mo ago
  I suspected you felt that way even though it hasn’t been my personal experience. I’ve heard people say older models can’t do X, when I used that way etc. I suspect people are applying their own learning curve as part of their assessment of progress, you get better at writing prompts and it feels like the model improved. Which is why I’m saying we need some objective metrics to judge predictions of actual capacity.
  - MattRix 4mo ago
    There are objective metrics, that’s what benchmarks are.
    - Retric 4mo ago
      Companies game benchmarks, but sure they are better than nothing.
      - MattRix 4mo ago
        I mean you wanted something objective, and they are. I don’t know why you’re being dismissive of them, they’re a huge element of what drives model development forward. These companies aren’t just making stuff up, they really do want to improve the models, and the models really are improving.
        Retric 4mo ago
        > I don’t know why you’re being dismissive of them I’m aware of multiple cases of benchmark cheating/“optimization”. So, taking benchmarks a face value seems laughable.
- dotdi 4mo ago
  It's not that absolute progress is slow, it's extremely slow compared to the predictions. It might be fast in absolute terms, but the "50% of coders will be obsolete by 2023" has been renewed every six months, and it's becoming increasingly clear that there's a real chance it might not ever happen.
  - mirekrusin 4mo ago
    „Coders being obsolete” is not a measure of AI capabilities. I see coders being more busy than ever before. I see people without coding knowledge getting more behind. The gap is widening, not shrinking.
    - Schlagbohrer 4mo ago
      I see people without coding ability catching up to learned coders. AI is a huge force multiplier for people who don't have hands on, detailed technical knowledge, AI can increasingly handle that and just needs a human to steer it more broadly.
      - mirekrusin 4mo ago
        It's huge force multiplier for people who have hands on, detailed technical knowledge as well. 3 * N < 42 * N 42 - 3 < N * (42 - 3) It helps to know layer you're working on. People seem to make mistake of thinking how good LLMs are around tasks that they are familiar with and extrapolating it to whole population. It's good mental exercise to think about how little you can do compared to expert on tasks you never thought of working. Ie. if you're programmer or know something about finance, don't think how much it enables you to do better coding or investing, think instead how much it doesn't enable you to work on something you don't know like maybe molecular biology or visual special effects – it's all there but it's much better multiplier for people who do know their shit. Knowing layer you're working on helps a lot, it gets multiplied. Knowing programming is becoming more fundamental skill than ever before as it lies at the foundation of almost everything else.
frognumber 4mo ago
This was their prediction for 2026: "The bet of using AI to speed up AI research is starting to pay off. OpenBrain continues to deploy the iteratively improving Agent-1 internally for AI R&D. Overall, they are making algorithmic progress 50% faster than they would without AI assistants—and more importantly, faster than their competitors. The AI R&D progress multiplier: what do we mean by 50% faster algorithmic progress? Several competing publicly released AIs now match or exceed Agent-0, including an open-weights model. OpenBrain responds by releasing Agent-1, which is more capable and reliable.28 People naturally try to compare Agent-1 to humans, but it has a very different skill profile. It knows more facts than any human, knows practically every programming language, and can solve well-specified coding problems extremely quickly. On the other hand, Agent-1 is bad at even simple long-horizon tasks, like beating video games it hasn’t played before. Still, the common workday is eight hours, and a day’s work can usually be separated into smaller chunks; you could think of Agent-1 as a scatterbrained employee who thrives under careful management.29 Savvy people find ways to automate routine parts of their jobs.30 OpenBrain’s executives turn consideration to an implication of automating AI R&D: security has become more important. In early 2025, the worst-case scenario was leaked algorithmic secrets; now, if China steals Agent-1’s weights, they could increase their research speed by nearly 50%.31 OpenBrain’s security level is typical of a fast-growing ~3,000 person tech company, secure only against low-priority attacks from capable cyber groups (RAND’s SL2).32 They are working hard to protect their weights and secrets from insider threats and top cybercrime syndicates (SL3),33 but defense against nation states (SL4&5) is barely on the horizon." https://ai-2027.com/ https://ai-2027.com/ That's precisely where we are. This is eerie. It's like a time traveler. The only delta is Anthropic is in the role of OpenAI.
- gensym 4mo ago
  So you think Anthropic is using internal AI assistants to pull away from competitors and the leapfrogging we've seen over the last several years is now done? That seems to me to be the most concrete and least obvious prediction in the quoted text. I don't think that's happening. If that were generally accepted as true I would expect OpenAI to be unable to successfully IPO.
  - 4mo ago

by Retric 4mo ago

MattRix 4mo ago

The idea that progress is “slow” in the AI space is absurd. These are some of the fastest growing products and companies of all time. The models are still improving a surprising amount.

Retric 4mo ago
I suspected you felt that way even though it hasn’t been my personal experience. I’ve heard people say older models can’t do X, when I used that way etc. I suspect people are applying their own learning curve as part of their assessment of progress, you get better at writing prompts and it feels like the model improved. Which is why I’m saying we need some objective metrics to judge predictions of actual capacity.
- MattRix 4mo ago
  There are objective metrics, that’s what benchmarks are.
  - Retric 4mo ago
    Companies game benchmarks, but sure they are better than nothing.
    - MattRix 4mo ago
      I mean you wanted something objective, and they are. I don’t know why you’re being dismissive of them, they’re a huge element of what drives model development forward. These companies aren’t just making stuff up, they really do want to improve the models, and the models really are improving.
      - Retric 4mo ago
        > I don’t know why you’re being dismissive of them I’m aware of multiple cases of benchmark cheating/“optimization”. So, taking benchmarks a face value seems laughable.
dotdi 4mo ago
It's not that absolute progress is slow, it's extremely slow compared to the predictions. It might be fast in absolute terms, but the "50% of coders will be obsolete by 2023" has been renewed every six months, and it's becoming increasingly clear that there's a real chance it might not ever happen.
- mirekrusin 4mo ago
  „Coders being obsolete” is not a measure of AI capabilities. I see coders being more busy than ever before. I see people without coding knowledge getting more behind. The gap is widening, not shrinking.
  - Schlagbohrer 4mo ago
    I see people without coding ability catching up to learned coders. AI is a huge force multiplier for people who don't have hands on, detailed technical knowledge, AI can increasingly handle that and just needs a human to steer it more broadly.
    - mirekrusin 4mo ago
      It's huge force multiplier for people who have hands on, detailed technical knowledge as well. 3 * N < 42 * N 42 - 3 < N * (42 - 3) It helps to know layer you're working on. People seem to make mistake of thinking how good LLMs are around tasks that they are familiar with and extrapolating it to whole population. It's good mental exercise to think about how little you can do compared to expert on tasks you never thought of working. Ie. if you're programmer or know something about finance, don't think how much it enables you to do better coding or investing, think instead how much it doesn't enable you to work on something you don't know like maybe molecular biology or visual special effects – it's all there but it's much better multiplier for people who do know their shit. Knowing layer you're working on helps a lot, it gets multiplied. Knowing programming is becoming more fundamental skill than ever before as it lies at the foundation of almost everything else.

frognumber 4mo ago

This was their prediction for 2026: "The bet of using AI to speed up AI research is starting to pay off. OpenBrain continues to deploy the iteratively improving Agent-1 internally for AI R&D. Overall, they are making algorithmic progress 50% faster than they would without AI assistants—and more importantly, faster than their competitors. The AI R&D progress multiplier: what do we mean by 50% faster algorithmic progress? Several competing publicly released AIs now match or exceed Agent-0, including an open-weights model. OpenBrain responds by releasing Agent-1, which is more capable and reliable.28 People naturally try to compare Agent-1 to humans, but it has a very different skill profile. It knows more facts than any human, knows practically every programming language, and can solve well-specified coding problems extremely quickly. On the other hand, Agent-1 is bad at even simple long-horizon tasks, like beating video games it hasn’t played before. Still, the common workday is eight hours, and a day’s work can usually be separated into smaller chunks; you could think of Agent-1 as a scatterbrained employee who thrives under careful management.29 Savvy people find ways to automate routine parts of their jobs.30 OpenBrain’s executives turn consideration to an implication of automating AI R&D: security has become more important. In early 2025, the worst-case scenario was leaked algorithmic secrets; now, if China steals Agent-1’s weights, they could increase their research speed by nearly 50%.31 OpenBrain’s security level is typical of a fast-growing ~3,000 person tech company, secure only against low-priority attacks from capable cyber groups (RAND’s SL2).32 They are working hard to protect their weights and secrets from insider threats and top cybercrime syndicates (SL3),33 but defense against nation states (SL4&5) is barely on the horizon." https://ai-2027.com/ https://ai-2027.com/ That's precisely where we are. This is eerie. It's like a time traveler. The only delta is Anthropic is in the role of OpenAI.

gensym 4mo ago
So you think Anthropic is using internal AI assistants to pull away from competitors and the leapfrogging we've seen over the last several years is now done? That seems to me to be the most concrete and least obvious prediction in the quoted text. I don't think that's happening. If that were generally accepted as true I would expect OpenAI to be unable to successfully IPO.
- 4mo ago