5 ms·

FYI: The author has predicted that "AGI" will be here in 1-2 years and has staked his public reputation on it. He is personally invested in trendlines being lin

FYI: The author has predicted that "AGI" will be here in 1-2 years and has staked his public reputation on it. He is personally invested in trendlines being lindy rather than sigmoid. I don't think you can use lindy on trends as if trends are static objects, but that's another conversation.

throwawayk7h 5mo ago
Mind you, he is only personally invested insofar as he's staked his reputation on it. Throughout his writing, he expresses the same point over and over again: desperately wants AI to slow down, advocates for politics that would slow it down, and most likely nothing would bring him greater peace than to see a sigmoid curve appear.
- bombcar 5mo ago
  How convenient; when AGI doesn’t appear in 1-2 years his reputation is pristine because he slowed it down.
  - wmf 5mo ago
    To make that argument you'd want to show some causal link which so far we haven't seen.
  - achierius 5mo ago
    What do you want? This sounds like you have something against people making a claim in public, at all, on any topic of importance.
woeirua 5mo ago
Ok, but you can just look at the METR curve. Mythos saturated the 50% time horizon. The 80% is now at 3 hours. The rate of progress is accelerating not slowing down. There’s no indication yet that this is a sigmoid!
- yorwba 5mo ago
  The METR task set contains no tasks with a duration greater than 32 hours (conservatively eyeballed from Figure 3: https://arxiv.org/abs/2503.17354 https://arxiv.org/abs/2503.17354 ), so any prediction that naively forecasts a longer time horizon is trivially incorrect. I guess that won't lead to a sigmoid-looking graph though, since METR will likely switch to a different evaluation methodology at that point and stop updating the old curve.
  - woeirua 5mo ago
    METR themselves say that any estimate >16 is highly suspect because there are too few tasks. I expect benchmarks like ProgramBench will replace METR this year.
paulpauper 5mo ago
He only has 1.5 more months. If he's wrong he needs to own it. Same for Eliezer Yudkowsky. But these people have too much riding on their brands. No one has the courage to fess up to being wrong. Given how many podcasts he and others have been on professing this belief, it will be hard to just pretend otherwise.
- quickthrowman 5mo ago
  Eliezer Yudkowsky, now there’s a name I haven’t heard in a while. What has he been up to since finishing the finest work of literature ever produced, Harry Potter and the Methods of Rationality? I’ve been patiently awaiting a sequel!
  - arjie 5mo ago
    His new book has billboards along the highway to the Bay Bridge. Surprised you haven’t heard of him recently. His fame has skyrocketed. If anyone builds it, everyone dies.
- hollerith 5mo ago
  Yudkowsky has never predicted that "AGI" will be here in 1-2 years. He has been saying frequently for years that it is easier to predict how the AI juggernaut will turn out (i.e., very badly for us) than to predict when the very bad things will happen. (I don't know about the other guy mentioned above.)
Sniffnoy 5mo ago
> FYI: The author has predicted that "AGI" will be here in 1-2 years and has staked his public reputation on it. He is personally invested in trendlines being lindy rather than sigmoid. I mean, that's called "having an opinion".
- paulpauper 5mo ago
  He wrote articles arguing that pro-AI people are dismissive of risks or even suggesting they are intellectually lazy. He's taken a side. if he's wrong I would hope he owns up to it
  - Sniffnoy 5mo ago
    > He's taken a side. Yes, that's called "having an opinion". Typically people writing argumentative pieces are doing so because they have a belief about the matter. I'm not sure what exactly you expect here. > if he's wrong I would hope he owns up to it I think Scott Alexander is pretty good about that.
  - boxed 5mo ago
    > He wrote articles arguing that pro-AI people are dismissive of risks or even suggesting they are intellectually lazy I mean.. this is 2026 right? You're not writing that comment from 2024 or something? We see massive problems already where photos are just not believable anymore, nor is audio, and not even video actually with many people falling for AI fake image clips from the Gaza war for example. And since then these tools are MASSIVELY more powerful. Disinformation is essentially free, and the cost of truth has been static. Meaning the "buying power" of truth has collapsed and is falling faster and faster. Anyone who dismissed AI risks a few years ago IS ALREADY PROVEN WRONG.
- dreambuffer 5mo ago
  He co-authored a report, which is something more than an opinion. It may be used to inspire policy. There should be greater reputational consequences for publishing something you spent a few months studying and writing about along with several experts. Just my opinion.
sigmoid10 5mo ago
AGI has become such a meaningless nondescript term, arguing when or how it is here has become pointless. Even OpenAI caved in and removed their AGI clause from their contract with Microsoft because they weren't fully sure that we are not there yet. The original ARC AGI was hailed as proof that AGI is not here yet, but now that ARC 1 and 2 got saturated, noone wanted to consider that perhaps we crossed the point where average humans are getting left behind. Frontier models are primarily limited by context and modality at this point, not by intelligence.
- Auracle 5mo ago
  To your point, if we had truly unlimited context to the point where at least that instance of a model could “learn” and have what seems like a continuous “consciousness” I think many of us would think that we’ve attained AGI. Right now we have an incredibly smart thing with severe short term memory loss, and it’s hard for us to reconcile that as it’s so different from us.
  - sigmoid10 5mo ago
    Quite a few people were already led to believe that these models are conscious when we had a fraction of current context lengths. Right now the biggest problem is that the "session" info in form of the current conversation gets lost too quickly, but that has become largely an implementation detail. You could fit an entire life's story into modern context windows. With some clever context management, you could probably build something that feels like what you describe. If we truly had this sort of short-term to long-term memory (i.e. from prompt context to weights) system on a technical foundation, we'd probably be closer to runaway superintelligence than mere AGI that could beat most humans on most tasks.
ToValueFunfetti 5mo ago
This is incorrect as written. The author contributed writing to AI-2027 but distanced himself from the underlying model. That model had 2027 as the modal year of AGI, not median or mean. The authors of that model revised it to a later date shortly after and (if I recall correctly) have since done so again. It is broadly true that Scott believes that AGI will come in the near future and from LLMs, although his reputation runs a ways deeper than that.
dannyobrien 5mo ago
So, this is not quite right: Alexander contributed to the report, but his personal opinion is more like the mid-2030s[1]. Freddie feels like this is him backing down from the original statement, but in fact he said this at the time the report was published, and in fact pointed out a graf below the quote that Freddie claims does tie him to 2027: > Do we really think things will move this fast? Sort of no - between the beginning of the project last summer and the present, Daniel’s median for the intelligence explosion shifted from 2027 to 2028. We keep the scenario centered around 2027 because it’s still his modal prediction (and because it would be annoying to change). Other members of the team (including me) have medians later in the 2020s or early 2030s, and also think automation will progress more slowly. So maybe think of this as a vision of what an 80th percentile fast scenario looks like - not our precise median, but also not something we feel safe ruling out. [2] I don't think this changes your observation that he is "personally invested" (i.e. believes this trendline will continue), but I'm pretty sure when AGI doesn't appear in 2027, many people will believe that this invalidates the arguments being made here (or in the report). The actual report was intended to give a feel for what a near-future "disaster" AGI scenario, and settled on a date to give that some concrete immediacy. The collective review that gave that as a possible, but not inevitable date is still ongoing (they originally pushed their best estimate out a bit further, but now they think, judging by the goals that are being hit, their scenario was a little too conservative). [3] [1] https://freddiedeboer.substack.com/p/im-offering-scott-alexander-a-wager#:~:text=said%20in%20the-,dwarkesh%20podcast,-and%20elsewhere%2C%20my https://freddiedeboer.substack.com/p/im-offering-scott-alexa... [2] https://www.astralcodexten.com/p/introducing-ai-2027 https://www.astralcodexten.com/p/introducing-ai-2027 [3] https://blog.aifutures.org/p/grading-ai-2027s-2025-predictions https://blog.aifutures.org/p/grading-ai-2027s-2025-predictio...
- grey-area 5mo ago
  AI boosters really are detached from reality. LLMs are nothing close to AGI and not going to lead to it, they can’t distinguish right from wrong, they can’t count, they can’t reason, they generate plausible text from a vast databank of connected text. Apparently that is enough to fool many people but it’s nothing close to AGI which would require internal models of the world, reasoning etc. We are nowhere close to AGI and the fools who predicted we were will unfortunately keep lying about their stated timelines when it inevitably doesn’t arrive. You’re already hedging and trying to caveat previous predictions, as OpenAI did with their AGI predictions which they’re now furiously back-pedalling on.
scott01 5mo ago
If I'm not mistaken, he's either affiliated with or otherwise connected to the effective altruist movement, hence he can't be unbiased. I find this article tells an interesting perspective on it: https://www.noemamag.com/the-politics-of-superintelligence/ https://www.noemamag.com/the-politics-of-superintelligence/
- naasking 5mo ago
  Nobody's unbiased.
  - scott01 5mo ago
    Obviously. I responded to the commenter with some context, but your comment doesn't bring anything to the discussion.
- goosejuice 5mo ago
  The whole rationalist movement is super bizarre to the "normies". It's not at all surprising that they are increasingly getting labeled a cult (they aren't by traditional definition but there are a lot similarities). I'm really surprised it hasn't hit the mainstream yet given the connections to Elon, Thiel, frontier labs, dark crypto funding, FTX/SBF, some suicides and some murders. It's all a little nuts. Meanwhile you got all the anti-democratic NRx people on the other side of it. I suspect this new doc coming out on HBO will spark a media frenzy.

throwawayk7h 5mo ago

Mind you, he is only personally invested insofar as he's staked his reputation on it. Throughout his writing, he expresses the same point over and over again: desperately wants AI to slow down, advocates for politics that would slow it down, and most likely nothing would bring him greater peace than to see a sigmoid curve appear.

bombcar 5mo ago
How convenient; when AGI doesn’t appear in 1-2 years his reputation is pristine because he slowed it down.
- wmf 5mo ago
  To make that argument you'd want to show some causal link which so far we haven't seen.
- achierius 5mo ago
  What do you want? This sounds like you have something against people making a claim in public, at all, on any topic of importance.

woeirua 5mo ago

Ok, but you can just look at the METR curve. Mythos saturated the 50% time horizon. The 80% is now at 3 hours. The rate of progress is accelerating not slowing down. There’s no indication yet that this is a sigmoid!

yorwba 5mo ago
The METR task set contains no tasks with a duration greater than 32 hours (conservatively eyeballed from Figure 3: https://arxiv.org/abs/2503.17354 https://arxiv.org/abs/2503.17354 ), so any prediction that naively forecasts a longer time horizon is trivially incorrect. I guess that won't lead to a sigmoid-looking graph though, since METR will likely switch to a different evaluation methodology at that point and stop updating the old curve.
- woeirua 5mo ago
  METR themselves say that any estimate >16 is highly suspect because there are too few tasks. I expect benchmarks like ProgramBench will replace METR this year.

paulpauper 5mo ago

He only has 1.5 more months. If he's wrong he needs to own it. Same for Eliezer Yudkowsky. But these people have too much riding on their brands. No one has the courage to fess up to being wrong. Given how many podcasts he and others have been on professing this belief, it will be hard to just pretend otherwise.

quickthrowman 5mo ago
Eliezer Yudkowsky, now there’s a name I haven’t heard in a while. What has he been up to since finishing the finest work of literature ever produced, Harry Potter and the Methods of Rationality? I’ve been patiently awaiting a sequel!
- arjie 5mo ago
  His new book has billboards along the highway to the Bay Bridge. Surprised you haven’t heard of him recently. His fame has skyrocketed. If anyone builds it, everyone dies.
hollerith 5mo ago
Yudkowsky has never predicted that "AGI" will be here in 1-2 years. He has been saying frequently for years that it is easier to predict how the AI juggernaut will turn out (i.e., very badly for us) than to predict when the very bad things will happen. (I don't know about the other guy mentioned above.)

Sniffnoy 5mo ago

> FYI: The author has predicted that "AGI" will be here in 1-2 years and has staked his public reputation on it. He is personally invested in trendlines being lindy rather than sigmoid. I mean, that's called "having an opinion".

paulpauper 5mo ago
He wrote articles arguing that pro-AI people are dismissive of risks or even suggesting they are intellectually lazy. He's taken a side. if he's wrong I would hope he owns up to it
- Sniffnoy 5mo ago
  > He's taken a side. Yes, that's called "having an opinion". Typically people writing argumentative pieces are doing so because they have a belief about the matter. I'm not sure what exactly you expect here. > if he's wrong I would hope he owns up to it I think Scott Alexander is pretty good about that.
- boxed 5mo ago
  > He wrote articles arguing that pro-AI people are dismissive of risks or even suggesting they are intellectually lazy I mean.. this is 2026 right? You're not writing that comment from 2024 or something? We see massive problems already where photos are just not believable anymore, nor is audio, and not even video actually with many people falling for AI fake image clips from the Gaza war for example. And since then these tools are MASSIVELY more powerful. Disinformation is essentially free, and the cost of truth has been static. Meaning the "buying power" of truth has collapsed and is falling faster and faster. Anyone who dismissed AI risks a few years ago IS ALREADY PROVEN WRONG.
dreambuffer 5mo ago
He co-authored a report, which is something more than an opinion. It may be used to inspire policy. There should be greater reputational consequences for publishing something you spent a few months studying and writing about along with several experts. Just my opinion.

sigmoid10 5mo ago

AGI has become such a meaningless nondescript term, arguing when or how it is here has become pointless. Even OpenAI caved in and removed their AGI clause from their contract with Microsoft because they weren't fully sure that we are not there yet. The original ARC AGI was hailed as proof that AGI is not here yet, but now that ARC 1 and 2 got saturated, noone wanted to consider that perhaps we crossed the point where average humans are getting left behind. Frontier models are primarily limited by context and modality at this point, not by intelligence.

Auracle 5mo ago
To your point, if we had truly unlimited context to the point where at least that instance of a model could “learn” and have what seems like a continuous “consciousness” I think many of us would think that we’ve attained AGI. Right now we have an incredibly smart thing with severe short term memory loss, and it’s hard for us to reconcile that as it’s so different from us.
- sigmoid10 5mo ago
  Quite a few people were already led to believe that these models are conscious when we had a fraction of current context lengths. Right now the biggest problem is that the "session" info in form of the current conversation gets lost too quickly, but that has become largely an implementation detail. You could fit an entire life's story into modern context windows. With some clever context management, you could probably build something that feels like what you describe. If we truly had this sort of short-term to long-term memory (i.e. from prompt context to weights) system on a technical foundation, we'd probably be closer to runaway superintelligence than mere AGI that could beat most humans on most tasks.

ToValueFunfetti 5mo ago

This is incorrect as written. The author contributed writing to AI-2027 but distanced himself from the underlying model. That model had 2027 as the modal year of AGI, not median or mean. The authors of that model revised it to a later date shortly after and (if I recall correctly) have since done so again. It is broadly true that Scott believes that AGI will come in the near future and from LLMs, although his reputation runs a ways deeper than that.

dannyobrien 5mo ago

So, this is not quite right: Alexander contributed to the report, but his personal opinion is more like the mid-2030s[1]. Freddie feels like this is him backing down from the original statement, but in fact he said this at the time the report was published, and in fact pointed out a graf below the quote that Freddie claims does tie him to 2027: > Do we really think things will move this fast? Sort of no - between the beginning of the project last summer and the present, Daniel’s median for the intelligence explosion shifted from 2027 to 2028. We keep the scenario centered around 2027 because it’s still his modal prediction (and because it would be annoying to change). Other members of the team (including me) have medians later in the 2020s or early 2030s, and also think automation will progress more slowly. So maybe think of this as a vision of what an 80th percentile fast scenario looks like - not our precise median, but also not something we feel safe ruling out. [2] I don't think this changes your observation that he is "personally invested" (i.e. believes this trendline will continue), but I'm pretty sure when AGI doesn't appear in 2027, many people will believe that this invalidates the arguments being made here (or in the report). The actual report was intended to give a feel for what a near-future "disaster" AGI scenario, and settled on a date to give that some concrete immediacy. The collective review that gave that as a possible, but not inevitable date is still ongoing (they originally pushed their best estimate out a bit further, but now they think, judging by the goals that are being hit, their scenario was a little too conservative). [3] [1] https://freddiedeboer.substack.com/p/im-offering-scott-alexander-a-wager#:~:text=said%20in%20the-,dwarkesh%20podcast,-and%20elsewhere%2C%20my https://freddiedeboer.substack.com/p/im-offering-scott-alexa... [2] https://www.astralcodexten.com/p/introducing-ai-2027 https://www.astralcodexten.com/p/introducing-ai-2027 [3] https://blog.aifutures.org/p/grading-ai-2027s-2025-predictions https://blog.aifutures.org/p/grading-ai-2027s-2025-predictio...

grey-area 5mo ago
AI boosters really are detached from reality. LLMs are nothing close to AGI and not going to lead to it, they can’t distinguish right from wrong, they can’t count, they can’t reason, they generate plausible text from a vast databank of connected text. Apparently that is enough to fool many people but it’s nothing close to AGI which would require internal models of the world, reasoning etc. We are nowhere close to AGI and the fools who predicted we were will unfortunately keep lying about their stated timelines when it inevitably doesn’t arrive. You’re already hedging and trying to caveat previous predictions, as OpenAI did with their AGI predictions which they’re now furiously back-pedalling on.

scott01 5mo ago

If I'm not mistaken, he's either affiliated with or otherwise connected to the effective altruist movement, hence he can't be unbiased. I find this article tells an interesting perspective on it: https://www.noemamag.com/the-politics-of-superintelligence/ https://www.noemamag.com/the-politics-of-superintelligence/

naasking 5mo ago
Nobody's unbiased.
- scott01 5mo ago
  Obviously. I responded to the commenter with some context, but your comment doesn't bring anything to the discussion.
goosejuice 5mo ago
The whole rationalist movement is super bizarre to the "normies". It's not at all surprising that they are increasingly getting labeled a cult (they aren't by traditional definition but there are a lot similarities). I'm really surprised it hasn't hit the mainstream yet given the connections to Elon, Thiel, frontier labs, dark crypto funding, FTX/SBF, some suicides and some murders. It's all a little nuts. Meanwhile you got all the anti-democratic NRx people on the other side of it. I suspect this new doc coming out on HBO will spark a media frenzy.