3 ms·

At least for me, the core criticism of AI 2027 was always that it was an extremely simplistic "number go up, therefore AGI", with some nice fiction-y words arou

At least for me, the core criticism of AI 2027 was always that it was an extremely simplistic "number go up, therefore AGI", with some nice fiction-y words around it. The scribble model kind-of hints at what a better forecast would've done - you start from the scribbles and ask "what would it take to get that line, and how'd we get there". And I love that the initial set of scribbles will, amongst other things, expose your biases. (Because you draw the set of scribbles that seems plausible to you, a priori) The fact that it can both guide you towards exploring alternatives and exposing biases, while being extremely simple - marvellous work. Definitely going to incorporate this into my reasoning toolkit!

ben_w 1y ago
To me, 2027 looks like a case of writing the conclusion first and then trying to explain backwards how it happens. If everything goes "perfectly", then the logic works (to an extent, but the increasing rate of returns is a suspicious assumption baked into it). But everything must go perfectly to do that, including all the productivity multipliers being independent and the USA deciding to take this genuinely seriously (not fake seriously in the form of politicians saying "we're taking this seriously" and not doing much), and therefore no-expenses-spared rush the target like it's actually an existential threat. I see no way this would be a baseline scenario.
- groby_b 1y ago
  It still misses the fact AI is nowhere close to self-improvement. In fact, there was a paper out on Friday that shows they're impressively bad at it: https://arxiv.org/abs/2506.22419 https://arxiv.org/abs/2506.22419
  - ben_w 1y ago
    Sure, but that's kinda what I'm saying they're doing wrong. One of the core claims 2027 is making is, to paraphrase, we get AI to help researchers do the research. If we just presume that this happens (which I'm saying is a mistake), then the AI helps researchers research how to make AI self-improve. But there's not any obvious reason for me to expect that. I mean, even aside from the narrow issue that the METR report earlier this year is showing that AI could (at the time) only do with 80% success tasks that would take a domain expert 15 minutes, and that this time horizon doubles every 7 months which would take them to being useful helpers for half-to-two-day tasks over 2027 which is still much less than needed for this kind of thing, there's still a lot of unknowns about where we are in what might be a sigmoid for unrealised efficiency gains in such code. Anyway, this is a much more thorough critique than I'm going to give: https://www.lesswrong.com/posts/PAYfmG2aRbdb74mEp/a-deep-critique-of-ai-2027-s-bad-timeline-models https://www.lesswrong.com/posts/PAYfmG2aRbdb74mEp/a-deep-cri...

ben_w 1y ago

To me, 2027 looks like a case of writing the conclusion first and then trying to explain backwards how it happens. If everything goes "perfectly", then the logic works (to an extent, but the increasing rate of returns is a suspicious assumption baked into it). But everything must go perfectly to do that, including all the productivity multipliers being independent and the USA deciding to take this genuinely seriously (not fake seriously in the form of politicians saying "we're taking this seriously" and not doing much), and therefore no-expenses-spared rush the target like it's actually an existential threat. I see no way this would be a baseline scenario.

groby_b 1y ago
It still misses the fact AI is nowhere close to self-improvement. In fact, there was a paper out on Friday that shows they're impressively bad at it: https://arxiv.org/abs/2506.22419 https://arxiv.org/abs/2506.22419
- ben_w 1y ago
  Sure, but that's kinda what I'm saying they're doing wrong. One of the core claims 2027 is making is, to paraphrase, we get AI to help researchers do the research. If we just presume that this happens (which I'm saying is a mistake), then the AI helps researchers research how to make AI self-improve. But there's not any obvious reason for me to expect that. I mean, even aside from the narrow issue that the METR report earlier this year is showing that AI could (at the time) only do with 80% success tasks that would take a domain expert 15 minutes, and that this time horizon doubles every 7 months which would take them to being useful helpers for half-to-two-day tasks over 2027 which is still much less than needed for this kind of thing, there's still a lot of unknowns about where we are in what might be a sigmoid for unrealised efficiency gains in such code. Anyway, this is a much more thorough critique than I'm going to give: https://www.lesswrong.com/posts/PAYfmG2aRbdb74mEp/a-deep-critique-of-ai-2027-s-bad-timeline-models https://www.lesswrong.com/posts/PAYfmG2aRbdb74mEp/a-deep-cri...