6 ms·
Some useful context from Scott Alexander's blog reveals that the authors don't actually believe the 2027 target: > Do we really think things will move this fas
by Aurornis 1y ago
Some useful context from Scott Alexander's blog reveals that the authors don't actually believe the 2027 target:
> Do we really think things will move this fast? Sort of no - between the beginning of the project last summer and the present, Daniel’s median for the intelligence explosion shifted from 2027 to 2028. We keep the scenario centered around 2027 because it’s still his modal prediction (and because it would be annoying to change). Other members of the team (including me) have medians later in the 2020s or early 2030s, and also think automation will progress more slowly. So maybe think of this as a vision of what an 80th percentile fast scenario looks like - not our precise median, but also not something we feel safe ruling out.
They went from "this represents roughly our median guess" in the website to "maybe think of it as an 80th percentile version of the fast scenario that we don't feel safe ruling out" in followup discussions.
Claiming that one reason they didn't change the website was because it would be "annoying" to change the date is a good barometer for how seriously anyone should be taking this exercise.
- throw310822 1y agoYes and no, is it actually important if it's 2027 or 28 or 2032? The scenario is such that a difference of a couple of years is basically irrelevant.
- Jensson 1y ago> The scenario is such that a difference of a couple of years is basically irrelevant. 2 years left and 7 years left is a massive difference, it is so much easier to deal with things 7 years in the future especially since its easier to see as we get closer.
- lm28469 1y agoYeah for example we had decades to tackle climate change and we easily over came the problem
- Jensson 1y agoLook at military tech 1941 and 1946, its a massive difference. 5 year time difference means a ton when people take it seriously. Only reason climate change isn't solved is that people don't care enough.
- lostmsu 1y ago> people don't care enough I think your parent meant exactly this.
- adrianN 1y agoThe reason climate change isn’t solved is that it’s a problem that can’t be solved by technology alone. We already have all the technology we need to decarbonize. We lack the political will to make the necessary investment.
- pinkmuffinere 1y agoYa, multiple failed predictions is an indicator of systemically bad predictors imo. That said, Scott Alexander usually does serious analysis instead of handwavey hype, so I tend to believe him more than many others in the space. My somewhat native take is that we’re still close to peak hype, AI will under deliver on the inflated expectations, and we’ll head into another “winter”. This pattern has repeated multiple times, so I think it’s fairly likely based on that alone. Real progress is made during each cycle, i think humans are just bad at containing excitement
- sigmaisaletter 1y agoI think you mean "somewhat naive" instead of "somewhat native". :) But, yes, this, in my mind the peak[1] bubble times ended with the DeepSeek shock earlier this year, and we are slowly on the downward trajectory now. It won't be slow for long, once people start realizing Sama was telling them a fairy tale, and AGI/ASI/singularity isn't "right around the corner", but (if achievable at all) at least two more technology triggers away. We got reasonably useful tools out of it, and thanks to Zuck, mostly for free (if you are an "investor", terms and conditions apply). [1] https://en.wikipedia.org/wiki/Gartner_hype_cycle https://en.wikipedia.org/wiki/Gartner_hype_cycle
- amarcheschi 1y agoThe other writings from Scott Alexander on scientific racism are also another good point imho
- A_D_E_P_T 1y agoWhat specifically would you highlight as being particularly egregious or wrong? As a general rule, "it's icky" doesn't make something false.
- amarcheschi 1y agoAnd it doesn't make it true either Human biodiversity theories are a bunch of dogwhistles for racism https://en.m.wikipedia.org/wiki/Human_Biodiversity_Institute https://en.m.wikipedia.org/wiki/Human_Biodiversity_Institute And his blog's survey reports a lot of users actually believing in those theories https://reflectivealtruism.com/2024/12/27/human-biodiversity-part-5-the-people-of-alexander/ https://reflectivealtruism.com/2024/12/27/human-biodiversity... (I wasn't referring to this Ai 2027 in specific)
- HDThoreaun 1y agoTry steel manning in order to effectively persuade. This comment does not address the argument being made it just calls a field of study icky. The unfortunate reality is that shouting down questions like this only empowers the racist HBI people who are effectively leeches
- amarcheschi 1y agoScott effectively defended Lynn study on iq here https://www.astralcodexten.com/p/how-to-stop-worrying-and-learn-to https://www.astralcodexten.com/p/how-to-stop-worrying-and-le... Citing another blog post that defends it, while conveniently ignoring every other point being made by researchers https://en.m.wikipedia.org/wiki/IQ_and_the_Wealth_of_Nations https://en.m.wikipedia.org/wiki/IQ_and_the_Wealth_of_Nations
- bpodgursky 1y agoDo you feel that you are shifting goalposts a bit when quibbling over whether AI will kill everyone in 2030 or 2035? As of 10 years ago, the entire conversation would have seemed ridiculous. Now we're talking about single digit timeline differences to the singularity or extinction. Come on man.
- ewoodrich 1y agoI'm in my 30s and remember my friend in middle school showing me a website he found with an ominous countdown to Kurzweil's "singularity" in 2045.
- throw310822 1y ago> ominous countdown to Kurzweil's "singularity" in 2045 And then it didn't happen?
- goatlover 1y agoNot between 2027 and 2032 anyway.
- SketchySeaBeast 1y agoWell, the first goal was 1997, but Skynet sure screwed that up.
- falcor84 1y agoWell, they can still go back in time and change things
- sigmaisaletter 1y ago> 10 years ago, the entire conversation would have seemed ridiculous Bostrom's book[1] is 11 years old. The Basilisk is 15 years old. The Singularity summit was nearly 20 years ago. And Yudkowsky was there for all of it. If you frequented LessWrong in the 2010s, most of this is very very old hat. [1]: https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang... [2]: Ford (2015) "Our Fear of Artificial Intelligence", MIT Tech Review: https://www.technologyreview.com/2015/02/11/169210/our-fear-of-artificial-intelligence/ https://www.technologyreview.com/2015/02/11/169210/our-fear-...
- magicalist 1y ago> They went from "this represents roughly our median guess" in the website to "maybe think of it as an 80th percentile version of the fast scenario that we don't feel safe ruling out" in followup discussions. His post also just reads like they think they're Hari Seldon (oh Daniel's modal prediction, whew, I was worried we were reading fanfic) while being horoscope-vague enough that almost any possible development will fit into the "predictions" in the post for the next decade. I really hope I don't have to keep reading references to this for the next decade.
- amarcheschi 1y agoYud is also something like 50% sure we'll die in a few years - if I'm not wrong I guess they'll have to update their a priori % if we survive
- ben_w 1y agoI think Yudkowsky is more like 90% sure of us all dying in a few (<10) years. I mean, this is their new book: https://ifanyonebuildsit.com/ https://ifanyonebuildsit.com/
- godelski 1y ago> We do not mean that as hyperbole. We are not exaggerating for effect. We think that is the most direct extrapolation from the knowledge, evidence, and institutional conduct around artificial intelligence today. In this book, we lay out our case. Take us seriously, buy our book! We're real researchers, so we make our definitely scientific case available to anyone who will give us $15-$30! It's the most important book ever, says some actor. Read it, so we all don't die! For Christ's sake, how does anyone take this Harry Potter fanfiction writer serially?
- ben_w 1y agoBecause of what else he writes besides the fanfic. (Better question is why anyone takes JKR herself seriously). But if you insist on only listening to people with academic acolades or industrial output, there's this other guy who got the Rumelhart Prize (2001), Turing Award (2018), Dickson Prize (2021), Princess of Asturias Award (2022), Nobel Prize in Physics (2024), VinFuture Prize (2024), Queen Elizabeth Prize for Engineering (2025), Order of Canada, Fellow of the Royal Society, and Fellow of the Royal Society of Canada That's one person with all that, and he says there's a "10 to 20 per cent chance" that AI would be the cause of human extinction within the following three decades, and "it is hard to see how you can prevent the bad actors from using [AI] for bad things.": https://en.wikipedia.org/wiki/Geoffrey_Hinton https://en.wikipedia.org/wiki/Geoffrey_Hinton Myself, I'm closer to Hinton's view than Yudkowsky's: path dependency, i.e. I expect that before we get existential threat from AI, we get catastrophic economic threat the precludes existential threat.
- merksittich 1y agoAlso, the relevant manifold prediction has low odds: https://manifold.markets/IsaacKing/ai-2027-reports-predictions-borne-o https://manifold.markets/IsaacKing/ai-2027-reports-predictio...
- AnimalMuppet 1y agoWell, if it happens in 2028 instead of 2027, I don't think anyone will be justified in yelling at them for being wrong. But I can't help notice that in one year, the expected arrival date slipped by... one year. That does not bode well...