19 ms·

This is religious fervor folks, as AI 2027 was. I grew up in evangelical christianity, and to them the end of the world is just around the corner, the same way

by taurath 3mo ago

This is religious fervor folks, as AI 2027 was. I grew up in evangelical christianity, and to them the end of the world is just around the corner, the same way it has been since I was a small child and likely will be when we are all gone. This isn't science. This isn't hypothesis experiment record results. This is very expensive astrology, shiny rock collecting, ritualistic meaning-making and self-justification. Yall, with your incredible wealth and resources you could do real good in this world and make society better, healthier, better educated, and the whole world more equal, just, and reduce the desperation and suffering. Reject the false and self-serving narratives that empathy doesn't matter, that altruism isn't "effective". You can change a person's whole life in a moment.

skrebbel 3mo ago
It's nuts how well "Superintelligence: The Idea That Eats Smart People"[0] aged. That talk is a decade old by now and still hits just as hard as it did back then, despite the incredible advances made in AI in the meantime. [0] https://idlewords.com/talks/superintelligence.htm https://idlewords.com/talks/superintelligence.htm
- taurath 3mo ago
  Yeah I think its possible that for many folks its the first time they're coming up on these concepts, and it troubles them in the same way that the concept of death troubles them (and me, to be clear!). For me its as simple as watching how people talk, and seeing how in every single case whatever the next thing is, if you believe It, there is only ever justification of doubling down, doing more, going deeper, reducing any doubt. These are not scientists, they're business people and salespeople, and a few optimists having recently on paper solved all their worldly financial needs. Even if one throws that aside, spending time exploring and building with the most state of the art LLMs is just as instructive. I'm watching the implementation - whats working is ML models trained on specific domains (not much different than 5+ years ago), and whats not working is a general model that humanity can let go to work on its own. Sit in front and observe ideas turn to the samey intellectual, high-syllable mush. Its productive, but not in any way that's promised.
  - TheOtherHobbes 3mo ago
    Did you actually read the text? OPs are calling that Plan D. They're proposing an alternative, which is a global brake on frontier AI research to keep the basilisk in its jar until we work out what we're dealing with and how to handle it.
    - layla5alive 3mo ago
      No, they're proposing a spying panopticon and state control of global resource distribution - specifically general purpose compute - including seizing and destroying GPUs. They're proposing a totalitarian global dictatorship controlling computing hardware and software. Lest you think I'm being hyperbolic: https://ai-2040.com/supplements/covert-ai-projects https://ai-2040.com/supplements/covert-ai-projects This is arsonists selling fire insurance.
      - esafak 2mo ago
        A nasty global inspections regime just like we have for nuclear weapons, which are less dangerous than this. Oh no ...
        layla5alive 2mo ago
        Nuclear weapons are less dangerous than compute? Are you listening to yourself? Humans employing this kind of thinking and desire to control other people is the genuine danger here.
  - YeGoblynQueenne 3mo ago
    >> Even if one throws that aside, spending time exploring and building with the most state of the art LLMs is just as instructive. I'm watching the implementation - whats working is ML models trained on specific domains (not much different than 5+ years ago), and whats not working is a general model that humanity can let go to work on its own. Sit in front and observe ideas turn to the samey intellectual, high-syllable mush. Its productive, but not in any way that's promised. Important point. LLMs were early on hailed as the first general-puprose AIs that can perform any task (remember "Sparks of AGI"?). Today they're increasingly promoted for specialised applications - coding, as a for instance.
- deleted 3mo ago
  [deleted]
- beshrkayali 3mo ago
  This is hilariously true > AI risk is string theory for computer programmers. It's fun to think about, interesting, and completely inaccessible to experiment given our current technology. You can build crystal palaces of thought, working from first principles, then climb up inside them and pull the ladder up behind you. People who can reach preposterous conclusions from a long chain of abstract reasoning, and feel confident in their truth, are the wrong people to be running a culture. I understand how people running in the same scene fall into the echo chamber effect and get gulped into the cult, but why does everybody want to be a prophet?
  - walrus01 3mo ago
    Being a prophet is probably great until you suddenly find yourself building a fortified compound in Waco, Texas and purchasing black market full auto machine guns.
    - saltwatercowboy 3mo ago
      Thank you for contacting Gunmetal Ranch, a legitimate 501(c). If your call is related to the class-action "most dangerous game" settlement, please hold for a cowpoke.
  - int3trap 3mo ago
    > but why does everybody want to be a prophet? Its not everyone building crystal palaces in their mind, they're all building fortresses. And they can't be wrong in their fortress or it breaks their world view which they cannot accept.
  - 0xDEAFBEAD 3mo ago
    Theorizing about nuclear winter is somewhat similar, in the sense of being inaccessible to experiment. Does that mean we should disregard the possibility of nuclear winter?
    - m4rtink 3mo ago
      As with using the Orion drive for launch and landing on Erth is possible, test nuclear winter is certainly possible. But as with the former, you won't be very popular among the survivors (if any).
- GCUMstlyHarmls 3mo ago
  Video link, as the link on the page is dead. https://www.youtube.com/watch?v=kErHiET5YPw https://www.youtube.com/watch?v=kErHiET5YPw
- casey2 3mo ago
  and upon rereading completely holds up technically we are still passing in massive data into simple networks giving no opportunity for introspection or recursive self improvement.
- 0xDEAFBEAD 3mo ago
  >despite the incredible advances made in AI in the meantime So the goalposts will be moved whenever necessary in that case? No amount of incredible advances in AI will ever get skeptical HN commenters to take AI's implications seriously? "The Gish gallops will continue until the nagging doubts have been silenced"
- tim333 3mo ago
  I always thought it a bit pessimistic on the upsides like robot to build us palaces, immortality, conquer the galaxy type stuff along the lines of the "Grandiosity" bit in the essay. It's a shame we have amazing tech possibilities and most people are either saying either it's rubbish and won't work or it'll bring doom. Aside from being boring they are probably factually wrong. Bit like saying early medical knowledge was rubbish or will doom us by summoning demons while really life expectancy at birth went from twenty something in Roman times to eighty odd now.
walrus01 3mo ago
[flagged]
fulafel 3mo ago
To interpret charitably, I guess we could solve this for religion = technological development so advanced as to be indistinguishable from magic. It's been done in some Star Trek episodes I think.
- taurath 3mo ago
  It has a lot of the properties of magic. You can put on quite a magic show with a trillion dollars.
trebaud 3mo ago
Or you deeply suffer from normalcy bias. Not believing that AGI is possible is irrational and unscientific imo, the human brain exists, it is not made of magic, it can reproduced. It is as simple as that. We can debate about timelines, architectures that may lead us there, etc... You can even talk all day long about definitions of AGI, and people waste their time doing that. But saying that the human brain cognitive capabilities cannot be reproduced on other types of substrates is stupid at this point.
- lins1909 3mo ago
  How can not believing something that hasn't been proven be unscientific? Do you know that words mean things?
  - ToValueFunfetti 3mo ago
    Science means the pursuit of knowledge. It doesn't mean "only believing proven things". If we're going to be rude, lets at least take the time to be right.
    - cindyllm 3mo ago
      [dead]
  - trebaud 3mo ago
    You dont seem to understand what proof means. Human brain is made of matter, matter can be arranged to make a thing that reproduces human brain properties. What's the confusion here? I say its unscientific because it places the human brain beyond the scope of what can be operated on. Not having the knowledge or tech yet to achieve that is irrelevant since we have an existence proof.
    - TheOtherHobbes 3mo ago
      Even if we accept your premise, and not everyone does, the confusion is whether we're capable of creating an equivalent arrangement, even in principle, using alternative materials. That's not "irrelevant", it's fundamental.
pibaker 3mo ago
When I learned that this website is by the same people who gave us AI 2027, I immediately thought about the Wikipedia page on doomsday predictions. https://en.wikipedia.org/wiki/List_of_dates_predicted_for_apocalyptic_events https://en.wikipedia.org/wiki/List_of_dates_predicted_for_ap... Notice the many times when a prediction failed and the so called prophet would come back in a few years and give you a new date. And they would tell you, well, this time it's real. I do find it ironic that many of the AI predictions are coming from the self titled "rationalists." It seems like building your identity around being rational and immune to psychological pitfalls is a good way to ensure that you don't even notice that you have walked straight into the one psychological trap every cult has employed since time immemorial.
- dheera 3mo ago
  ... and then there are the actual planet-threatening astronomical events that humanity should think about ways to mitigate but I'm worried that human lifespans and capitalism prevent us from working towards mitigations. Everyone will just say "meh, 1 million years is a long time, I won't be around" and a million years will go by. https://en.wikipedia.org/wiki/Timeline_of_the_far_future https://en.wikipedia.org/wiki/Timeline_of_the_far_future
  - thegrim33 3mo ago
    Capitalism, in particular NASA's commercial space development program, have already provided and demonstrated initial asteroid redirect capability via the DART mission launched by SpaceX, which impacted an asteroid and measured the changes to its trajectory.
- thegrim33 3mo ago
  They gave a ~2 year timeframe for their predictions in this one (tied to the next presidential election), so in 2 years when none of this has happened will they then switch over to a new AI 2044 Plan? Where's the accountability? Where's the followup or retrospective? What if there was some mechanism that branded these people so that when they made claims in the future people could clearly see the authors labeled as "made completely wrong fantastical claims in the past"?
reactordev 3mo ago
The end of the world is always just out of reach. It’s the perfect carrot on a stick for people who are conditioned to be afraid.
derektank 3mo ago
>Reject the false and self-serving narratives that empathy doesn't matter, that altruism isn't "effective". You can change a person's whole life in a moment. You can change more people’s lives, more substantially, if you donate effectively. Effective altruism started out as (and the majority of effective altruist financing is committed to) an effort to rationalize what has historically been a very emotionally driven activity by deploying insights from developmental economics. If you want to take longtermists to task, go right ahead, but please refrain from torching anti-malarial or child vaccination programs while doing so.
- gcr 3mo ago
  But in the public image, the EA community is synonymous with doubling down on AI / AGI to the exclusion of the other projects. OpenPhil changing its name to Coefficient Giving, 80000 hours and bluedot and (to a lesser extent) CFAR dropping other initiatives and switching to AGI promotion… to my knowledge GiveWell is the only other big name that continues to advance other initiatives. Then look at figureheads like SBF committing fraud and begging for a pardon from the architects of the USAID shutdown… We begin to paint a picture of a community that’s (by and large) abandoned its principles for power. I know the view from the inside is more nuanced, but I think it’s a reasonable association for random members of the public to make. My critique of the EA community is that it’s myopic and unregularized. If you really think AGI is make-or-break for civilization, it’s completely rational to deprioritize side bets.
  - derektank 3mo ago
    >My critique of the EA community is that it’s myopic and unregularized. If you really think AGI is make-or-break for civilization, it’s completely rational to deprioritize side bets. I’d be curious to hear you expand on this. What binds the EA community together, from the shrimp welfare enthusiasts and wild animal initiative, to the longtermist lightcone obsessive, to the people funding vitamin A supplementation, is simply a commitment to maximizing the number of quality adjusted life years saved each year and a belief that empirical observation can be used to improve that number. To my mind, this is a valuable insight on its own. Yes, if you come to such a heuristic with absurd prior beliefs, such as whether 100k neurons alone have QALYs in the first place or by placing equal value on people actually alive today and hypothetical people in the far flung future, you will get absurd results. Garbage in, garbage out. But that’s not an indictment of the fundamental insight, especially when you consider how poorly allocated the roughly $2 trillion in global charitable spending is.
MichaelDickens 3mo ago
I would rather see you engage with the substance of the article rather than skipping right to insulting the authors. I don't think this sort of comment is up to the standard I have come to expect from HN.
- romanhounds 3mo ago
  > I intend to donate (at least) 20% of my lifetime income to effective charities. I publish my donations on my Donations page. From your bio I suspect you're already in the cult.
  - HDThoreaun 3mo ago
    Imagine thinking someone donating money is evidence of something bad
    - nadir_ishiguro 3mo ago
      If the cause they donate to is bad, it is
      - jamilton 3mo ago
        What is bad about "effective charities"? It's probably mostly for malaria nets!
  - 0xDEAFBEAD 3mo ago
    "I see your objection to ad hominem arguments, and raise you a further ad hominem argument"
    - nadir_ishiguro 3mo ago
      His whole blog is exactly the same kind of nonsense as the article and reeks of effective altruism. That's not an ad hominem.
fwipsy 3mo ago
I remember thinking exactly the same thing around 5-7 years ago, in the GPT-2/GPT-3 era. "Oh sure they can produce semi-coherent output, but truly intelligent behavior is still far away. This isn't science fiction, they're just falling prey to Pascal's Mugging same as my religious friends did." Now I'm not so sure. I give the AI safety subculture as a whole a lot of credit for putting it on my radar back when it was otherwise still science fiction. I don't know if they're right about what comes next, but I think their case deserves to be evaluated on its merits, rather than assumed to be the result of psychological flaws. > Yall, with your incredible wealth and resources you could do real good in this world and make society better, healthier, better educated, and the whole world more equal, just, and reduce the desperation and suffering. Reject the false and self-serving narratives that empathy doesn't matter, that altruism isn't "effective". You can change a person's whole life in a moment. Confused at who this is directed towards. I'm fairly certain that the article was written by people who (at some point) identified as effective altruists, most of whom would enthusiastically agree with this. This community didn't start as AI researchers and later choose effective altruism; they were effective altruists who chose AI safety research as the most effective way to improve the world. Given that you apparently share their goals (a better world,) isn't it worth at least hearing them out on their methods?
- romanhounds 3mo ago
  >Given that you apparently share their goals (a better world,) isn't it worth at least hearing them out on their methods? Their methods are about convincing others that things that enrich and empower themselves at the expense of others is "improving the world". This isn't the stance of serious people who want to improve the world.
  - fwipsy 3mo ago
    Please don't confuse AI researchers and AI safety researchers (although there is some overlap.) That's like confusing the people from the Manhattan project with the people who protested 3 Mile Island, because they're both focused on nuclear technology. One effect of EA is that every AI researcher says they're trying to make the world better. Some of them are full of shit, but not all. I'm fairly certain the authors would be happy to see AI shut down indefinitely. They just don't believe that the coordination problem is solvable. This is their best attempt to come up with something workable in the real world, or at least get people started thinking about it.
kalkin 3mo ago
> Reject the false and self-serving narratives that empathy doesn't matter, that altruism isn't "effective" Who is this supposed to be arguing with? It sort of reads like it's trying to disparage "effective altruism", but I'm not sure. Setting aside any of the AI stuff, I've started to find it pretty grating when people seem to imply that transferring millions of dollars from wealthy people in California and the UK to impoverished Kenyans and Rwandans, or buying malaria bednets which can save a child's life for the cost of a fancy new gaming rig, is "self-serving" or something because weirdos are doing it, while true caring for other people involves [unspecified thing that doesn't appear to ask any material sacrifice comparable to donating a large percentage of income].
ozozozd 3mo ago
I completely agree. But not entirely sure that triggering the EA crowd is the effective way to deliver this message. They are the crowd who need to understand your framing the most, but they’re completely shutdown by your framing, as any religious follower would be. Your message gives me hope that not everyone’s drunk on the kool aid.
ls612 3mo ago
AI 2027 made substantive predictions about the near term future of the technology and the implications of it. I think the worst you can say about it is that the Agent-1 moment might come next year rather than this year. But being off by a year is far better than this 2040 slop, which is mostly disconnected from reality. The one thing I will say that they are correct about is that AI does have the potential to be highly destabilizing geopolitically, even if they get everything downstream of that wrong.
0xDEAFBEAD 3mo ago
Imagine observing powered flight for the first time and saying: "This is religious fervor folks. I remember the mythological story of Icarus from when I was younger. Key word, mythological." The existence of mythology describing Scenario X is not a valid argument against the plausibility of Scenario X. If we can acknowledge the possibility of nuclear doomsday without running an RCT of sample size 100 Earths, 50 of which undergo nuclear Armageddon, to verify that nuclear holocaust indeed a real phenomenon... then we can do the same for AI. Understand the arguments being made instead of engaging in these guilt-by-association arguments. Modern AI capabilities are already mind-boggling by the standards of 20 years ago. We should at least prepare for the possibility that trends continue on the current trajectory.
bryan0 3mo ago
From TFA > Plan A is primarily a recommendation, not a prediction. That sounds nothing like “religious fervor” The fact that technology can increase existential risk for civilization is not fantasy. It’s a risk that should be reasonably discussed.
bwanab 3mo ago
The other thing about this article is that it contains falsifiable claims that are, well, false. Consider the following direct quote: "The 2028 election cycle is heated, as usual. AI is the biggest topic." No poll of the American public has AI as one of the top concerns except in polls where the question is specifically about AI. The general polls show topics like healthcare costs, inflation/affordability, government dysfunction, and immigration as the top concerns. AI doesn't even show up in the top 20 in every list I looked at (Gallup, Pew, YouGov). It sounds more like "everybody I know thinks AI is the top concern" when everybody they know is people like them, i.e. an echo chamber. It gives me no confidence in any other claims.
- MaybiusStrip 3mo ago
  The current year is 2026. I think it's quite likely AI will make the top 3 by then. Putting this on my calendar.
  - bwanab 3mo ago
    Yes, I can't predict the future, but if it were a bet on Kalshi I'd be willing to put my money on the AI won't make the list side.
    - ToValueFunfetti 3mo ago
      This seems out of line with what you were saying above. Do you mean you would put money on a position less favorable than 50/50? What do you consider the likelihood such that you think you can announce it to be false today? 99%? I'd happily offer 30 to 1 odds on this today and add a 3x buffer for you up to $1000 of my dollars if you're serious.
      - bwanab 3mo ago
        I'm confused. In the original comment I pointed out that the article claims AI is a major topic when polls show that isn't the case. I still believe that in 2028 AI will not be a major topic as shown by polls and voting behavior. I'm not sure how I'd formulate the bet, but maybe it could be "as of election day 2028, AI will not be a top five concern mentioned in the most recent open-ended poll by Gallup polling the American public. This bet will not be decided by a poll in which AI is specifically mentioned in the polling question."
bogota 3mo ago
[dead]
huhkerrf 3mo ago
> I grew up in evangelical christianity, and to them the end of the world is just around the corner I grew up in evangelical Christianity, I'm still an evangelical Christian, and you're wrong to paint with a broad brush like that. Are there evangelicals that are millenarian? Yep, of course. But that is not my experience across multiple evangelical churches across decades.
aioproductos 3mo ago
[flagged]

by taurath 3mo ago

skrebbel 3mo ago

It's nuts how well "Superintelligence: The Idea That Eats Smart People"[0] aged. That talk is a decade old by now and still hits just as hard as it did back then, despite the incredible advances made in AI in the meantime. [0] https://idlewords.com/talks/superintelligence.htm https://idlewords.com/talks/superintelligence.htm

taurath 3mo ago
Yeah I think its possible that for many folks its the first time they're coming up on these concepts, and it troubles them in the same way that the concept of death troubles them (and me, to be clear!). For me its as simple as watching how people talk, and seeing how in every single case whatever the next thing is, if you believe It, there is only ever justification of doubling down, doing more, going deeper, reducing any doubt. These are not scientists, they're business people and salespeople, and a few optimists having recently on paper solved all their worldly financial needs. Even if one throws that aside, spending time exploring and building with the most state of the art LLMs is just as instructive. I'm watching the implementation - whats working is ML models trained on specific domains (not much different than 5+ years ago), and whats not working is a general model that humanity can let go to work on its own. Sit in front and observe ideas turn to the samey intellectual, high-syllable mush. Its productive, but not in any way that's promised.
- TheOtherHobbes 3mo ago
  Did you actually read the text? OPs are calling that Plan D. They're proposing an alternative, which is a global brake on frontier AI research to keep the basilisk in its jar until we work out what we're dealing with and how to handle it.
  - layla5alive 3mo ago
    No, they're proposing a spying panopticon and state control of global resource distribution - specifically general purpose compute - including seizing and destroying GPUs. They're proposing a totalitarian global dictatorship controlling computing hardware and software. Lest you think I'm being hyperbolic: https://ai-2040.com/supplements/covert-ai-projects https://ai-2040.com/supplements/covert-ai-projects This is arsonists selling fire insurance.
    - esafak 2mo ago
      A nasty global inspections regime just like we have for nuclear weapons, which are less dangerous than this. Oh no ...
      - layla5alive 2mo ago
        Nuclear weapons are less dangerous than compute? Are you listening to yourself? Humans employing this kind of thinking and desire to control other people is the genuine danger here.
- YeGoblynQueenne 3mo ago
  >> Even if one throws that aside, spending time exploring and building with the most state of the art LLMs is just as instructive. I'm watching the implementation - whats working is ML models trained on specific domains (not much different than 5+ years ago), and whats not working is a general model that humanity can let go to work on its own. Sit in front and observe ideas turn to the samey intellectual, high-syllable mush. Its productive, but not in any way that's promised. Important point. LLMs were early on hailed as the first general-puprose AIs that can perform any task (remember "Sparks of AGI"?). Today they're increasingly promoted for specialised applications - coding, as a for instance.
deleted 3mo ago
[deleted]
beshrkayali 3mo ago
This is hilariously true > AI risk is string theory for computer programmers. It's fun to think about, interesting, and completely inaccessible to experiment given our current technology. You can build crystal palaces of thought, working from first principles, then climb up inside them and pull the ladder up behind you. People who can reach preposterous conclusions from a long chain of abstract reasoning, and feel confident in their truth, are the wrong people to be running a culture. I understand how people running in the same scene fall into the echo chamber effect and get gulped into the cult, but why does everybody want to be a prophet?
- walrus01 3mo ago
  Being a prophet is probably great until you suddenly find yourself building a fortified compound in Waco, Texas and purchasing black market full auto machine guns.
  - saltwatercowboy 3mo ago
    Thank you for contacting Gunmetal Ranch, a legitimate 501(c). If your call is related to the class-action "most dangerous game" settlement, please hold for a cowpoke.
- int3trap 3mo ago
  > but why does everybody want to be a prophet? Its not everyone building crystal palaces in their mind, they're all building fortresses. And they can't be wrong in their fortress or it breaks their world view which they cannot accept.
- 0xDEAFBEAD 3mo ago
  Theorizing about nuclear winter is somewhat similar, in the sense of being inaccessible to experiment. Does that mean we should disregard the possibility of nuclear winter?
  - m4rtink 3mo ago
    As with using the Orion drive for launch and landing on Erth is possible, test nuclear winter is certainly possible. But as with the former, you won't be very popular among the survivors (if any).
GCUMstlyHarmls 3mo ago
Video link, as the link on the page is dead. https://www.youtube.com/watch?v=kErHiET5YPw https://www.youtube.com/watch?v=kErHiET5YPw
casey2 3mo ago
and upon rereading completely holds up technically we are still passing in massive data into simple networks giving no opportunity for introspection or recursive self improvement.
0xDEAFBEAD 3mo ago
>despite the incredible advances made in AI in the meantime So the goalposts will be moved whenever necessary in that case? No amount of incredible advances in AI will ever get skeptical HN commenters to take AI's implications seriously? "The Gish gallops will continue until the nagging doubts have been silenced"
tim333 3mo ago
I always thought it a bit pessimistic on the upsides like robot to build us palaces, immortality, conquer the galaxy type stuff along the lines of the "Grandiosity" bit in the essay. It's a shame we have amazing tech possibilities and most people are either saying either it's rubbish and won't work or it'll bring doom. Aside from being boring they are probably factually wrong. Bit like saying early medical knowledge was rubbish or will doom us by summoning demons while really life expectancy at birth went from twenty something in Roman times to eighty odd now.

walrus01 3mo ago

[flagged]

fulafel 3mo ago

To interpret charitably, I guess we could solve this for religion = technological development so advanced as to be indistinguishable from magic. It's been done in some Star Trek episodes I think.

taurath 3mo ago
It has a lot of the properties of magic. You can put on quite a magic show with a trillion dollars.

trebaud 3mo ago

Or you deeply suffer from normalcy bias. Not believing that AGI is possible is irrational and unscientific imo, the human brain exists, it is not made of magic, it can reproduced. It is as simple as that. We can debate about timelines, architectures that may lead us there, etc... You can even talk all day long about definitions of AGI, and people waste their time doing that. But saying that the human brain cognitive capabilities cannot be reproduced on other types of substrates is stupid at this point.

lins1909 3mo ago
How can not believing something that hasn't been proven be unscientific? Do you know that words mean things?
- ToValueFunfetti 3mo ago
  Science means the pursuit of knowledge. It doesn't mean "only believing proven things". If we're going to be rude, lets at least take the time to be right.
  - cindyllm 3mo ago
    [dead]
- trebaud 3mo ago
  You dont seem to understand what proof means. Human brain is made of matter, matter can be arranged to make a thing that reproduces human brain properties. What's the confusion here? I say its unscientific because it places the human brain beyond the scope of what can be operated on. Not having the knowledge or tech yet to achieve that is irrelevant since we have an existence proof.
  - TheOtherHobbes 3mo ago
    Even if we accept your premise, and not everyone does, the confusion is whether we're capable of creating an equivalent arrangement, even in principle, using alternative materials. That's not "irrelevant", it's fundamental.

pibaker 3mo ago

When I learned that this website is by the same people who gave us AI 2027, I immediately thought about the Wikipedia page on doomsday predictions. https://en.wikipedia.org/wiki/List_of_dates_predicted_for_apocalyptic_events https://en.wikipedia.org/wiki/List_of_dates_predicted_for_ap... Notice the many times when a prediction failed and the so called prophet would come back in a few years and give you a new date. And they would tell you, well, this time it's real. I do find it ironic that many of the AI predictions are coming from the self titled "rationalists." It seems like building your identity around being rational and immune to psychological pitfalls is a good way to ensure that you don't even notice that you have walked straight into the one psychological trap every cult has employed since time immemorial.

dheera 3mo ago
... and then there are the actual planet-threatening astronomical events that humanity should think about ways to mitigate but I'm worried that human lifespans and capitalism prevent us from working towards mitigations. Everyone will just say "meh, 1 million years is a long time, I won't be around" and a million years will go by. https://en.wikipedia.org/wiki/Timeline_of_the_far_future https://en.wikipedia.org/wiki/Timeline_of_the_far_future
- thegrim33 3mo ago
  Capitalism, in particular NASA's commercial space development program, have already provided and demonstrated initial asteroid redirect capability via the DART mission launched by SpaceX, which impacted an asteroid and measured the changes to its trajectory.
thegrim33 3mo ago
They gave a ~2 year timeframe for their predictions in this one (tied to the next presidential election), so in 2 years when none of this has happened will they then switch over to a new AI 2044 Plan? Where's the accountability? Where's the followup or retrospective? What if there was some mechanism that branded these people so that when they made claims in the future people could clearly see the authors labeled as "made completely wrong fantastical claims in the past"?

reactordev 3mo ago

The end of the world is always just out of reach. It’s the perfect carrot on a stick for people who are conditioned to be afraid.

derektank 3mo ago

>Reject the false and self-serving narratives that empathy doesn't matter, that altruism isn't "effective". You can change a person's whole life in a moment. You can change more people’s lives, more substantially, if you donate effectively. Effective altruism started out as (and the majority of effective altruist financing is committed to) an effort to rationalize what has historically been a very emotionally driven activity by deploying insights from developmental economics. If you want to take longtermists to task, go right ahead, but please refrain from torching anti-malarial or child vaccination programs while doing so.

gcr 3mo ago
But in the public image, the EA community is synonymous with doubling down on AI / AGI to the exclusion of the other projects. OpenPhil changing its name to Coefficient Giving, 80000 hours and bluedot and (to a lesser extent) CFAR dropping other initiatives and switching to AGI promotion… to my knowledge GiveWell is the only other big name that continues to advance other initiatives. Then look at figureheads like SBF committing fraud and begging for a pardon from the architects of the USAID shutdown… We begin to paint a picture of a community that’s (by and large) abandoned its principles for power. I know the view from the inside is more nuanced, but I think it’s a reasonable association for random members of the public to make. My critique of the EA community is that it’s myopic and unregularized. If you really think AGI is make-or-break for civilization, it’s completely rational to deprioritize side bets.
- derektank 3mo ago
  >My critique of the EA community is that it’s myopic and unregularized. If you really think AGI is make-or-break for civilization, it’s completely rational to deprioritize side bets. I’d be curious to hear you expand on this. What binds the EA community together, from the shrimp welfare enthusiasts and wild animal initiative, to the longtermist lightcone obsessive, to the people funding vitamin A supplementation, is simply a commitment to maximizing the number of quality adjusted life years saved each year and a belief that empirical observation can be used to improve that number. To my mind, this is a valuable insight on its own. Yes, if you come to such a heuristic with absurd prior beliefs, such as whether 100k neurons alone have QALYs in the first place or by placing equal value on people actually alive today and hypothetical people in the far flung future, you will get absurd results. Garbage in, garbage out. But that’s not an indictment of the fundamental insight, especially when you consider how poorly allocated the roughly $2 trillion in global charitable spending is.

MichaelDickens 3mo ago

I would rather see you engage with the substance of the article rather than skipping right to insulting the authors. I don't think this sort of comment is up to the standard I have come to expect from HN.

romanhounds 3mo ago
> I intend to donate (at least) 20% of my lifetime income to effective charities. I publish my donations on my Donations page. From your bio I suspect you're already in the cult.
- HDThoreaun 3mo ago
  Imagine thinking someone donating money is evidence of something bad
  - nadir_ishiguro 3mo ago
    If the cause they donate to is bad, it is
    - jamilton 3mo ago
      What is bad about "effective charities"? It's probably mostly for malaria nets!
- 0xDEAFBEAD 3mo ago
  "I see your objection to ad hominem arguments, and raise you a further ad hominem argument"
  - nadir_ishiguro 3mo ago
    His whole blog is exactly the same kind of nonsense as the article and reeks of effective altruism. That's not an ad hominem.

fwipsy 3mo ago

I remember thinking exactly the same thing around 5-7 years ago, in the GPT-2/GPT-3 era. "Oh sure they can produce semi-coherent output, but truly intelligent behavior is still far away. This isn't science fiction, they're just falling prey to Pascal's Mugging same as my religious friends did." Now I'm not so sure. I give the AI safety subculture as a whole a lot of credit for putting it on my radar back when it was otherwise still science fiction. I don't know if they're right about what comes next, but I think their case deserves to be evaluated on its merits, rather than assumed to be the result of psychological flaws. > Yall, with your incredible wealth and resources you could do real good in this world and make society better, healthier, better educated, and the whole world more equal, just, and reduce the desperation and suffering. Reject the false and self-serving narratives that empathy doesn't matter, that altruism isn't "effective". You can change a person's whole life in a moment. Confused at who this is directed towards. I'm fairly certain that the article was written by people who (at some point) identified as effective altruists, most of whom would enthusiastically agree with this. This community didn't start as AI researchers and later choose effective altruism; they were effective altruists who chose AI safety research as the most effective way to improve the world. Given that you apparently share their goals (a better world,) isn't it worth at least hearing them out on their methods?

romanhounds 3mo ago
>Given that you apparently share their goals (a better world,) isn't it worth at least hearing them out on their methods? Their methods are about convincing others that things that enrich and empower themselves at the expense of others is "improving the world". This isn't the stance of serious people who want to improve the world.
- fwipsy 3mo ago
  Please don't confuse AI researchers and AI safety researchers (although there is some overlap.) That's like confusing the people from the Manhattan project with the people who protested 3 Mile Island, because they're both focused on nuclear technology. One effect of EA is that every AI researcher says they're trying to make the world better. Some of them are full of shit, but not all. I'm fairly certain the authors would be happy to see AI shut down indefinitely. They just don't believe that the coordination problem is solvable. This is their best attempt to come up with something workable in the real world, or at least get people started thinking about it.

kalkin 3mo ago

> Reject the false and self-serving narratives that empathy doesn't matter, that altruism isn't "effective" Who is this supposed to be arguing with? It sort of reads like it's trying to disparage "effective altruism", but I'm not sure. Setting aside any of the AI stuff, I've started to find it pretty grating when people seem to imply that transferring millions of dollars from wealthy people in California and the UK to impoverished Kenyans and Rwandans, or buying malaria bednets which can save a child's life for the cost of a fancy new gaming rig, is "self-serving" or something because weirdos are doing it, while true caring for other people involves [unspecified thing that doesn't appear to ask any material sacrifice comparable to donating a large percentage of income].

ozozozd 3mo ago

I completely agree. But not entirely sure that triggering the EA crowd is the effective way to deliver this message. They are the crowd who need to understand your framing the most, but they’re completely shutdown by your framing, as any religious follower would be. Your message gives me hope that not everyone’s drunk on the kool aid.

ls612 3mo ago

AI 2027 made substantive predictions about the near term future of the technology and the implications of it. I think the worst you can say about it is that the Agent-1 moment might come next year rather than this year. But being off by a year is far better than this 2040 slop, which is mostly disconnected from reality. The one thing I will say that they are correct about is that AI does have the potential to be highly destabilizing geopolitically, even if they get everything downstream of that wrong.

0xDEAFBEAD 3mo ago

Imagine observing powered flight for the first time and saying: "This is religious fervor folks. I remember the mythological story of Icarus from when I was younger. Key word, mythological." The existence of mythology describing Scenario X is not a valid argument against the plausibility of Scenario X. If we can acknowledge the possibility of nuclear doomsday without running an RCT of sample size 100 Earths, 50 of which undergo nuclear Armageddon, to verify that nuclear holocaust indeed a real phenomenon... then we can do the same for AI. Understand the arguments being made instead of engaging in these guilt-by-association arguments. Modern AI capabilities are already mind-boggling by the standards of 20 years ago. We should at least prepare for the possibility that trends continue on the current trajectory.

bryan0 3mo ago

From TFA > Plan A is primarily a recommendation, not a prediction. That sounds nothing like “religious fervor” The fact that technology can increase existential risk for civilization is not fantasy. It’s a risk that should be reasonably discussed.

bwanab 3mo ago

The other thing about this article is that it contains falsifiable claims that are, well, false. Consider the following direct quote: "The 2028 election cycle is heated, as usual. AI is the biggest topic." No poll of the American public has AI as one of the top concerns except in polls where the question is specifically about AI. The general polls show topics like healthcare costs, inflation/affordability, government dysfunction, and immigration as the top concerns. AI doesn't even show up in the top 20 in every list I looked at (Gallup, Pew, YouGov). It sounds more like "everybody I know thinks AI is the top concern" when everybody they know is people like them, i.e. an echo chamber. It gives me no confidence in any other claims.

MaybiusStrip 3mo ago
The current year is 2026. I think it's quite likely AI will make the top 3 by then. Putting this on my calendar.
- bwanab 3mo ago
  Yes, I can't predict the future, but if it were a bet on Kalshi I'd be willing to put my money on the AI won't make the list side.
  - ToValueFunfetti 3mo ago
    This seems out of line with what you were saying above. Do you mean you would put money on a position less favorable than 50/50? What do you consider the likelihood such that you think you can announce it to be false today? 99%? I'd happily offer 30 to 1 odds on this today and add a 3x buffer for you up to $1000 of my dollars if you're serious.
    - bwanab 3mo ago
      I'm confused. In the original comment I pointed out that the article claims AI is a major topic when polls show that isn't the case. I still believe that in 2028 AI will not be a major topic as shown by polls and voting behavior. I'm not sure how I'd formulate the bet, but maybe it could be "as of election day 2028, AI will not be a top five concern mentioned in the most recent open-ended poll by Gallup polling the American public. This bet will not be decided by a poll in which AI is specifically mentioned in the polling question."

bogota 3mo ago

[dead]

huhkerrf 3mo ago

> I grew up in evangelical christianity, and to them the end of the world is just around the corner I grew up in evangelical Christianity, I'm still an evangelical Christian, and you're wrong to paint with a broad brush like that. Are there evangelicals that are millenarian? Yep, of course. But that is not my experience across multiple evangelical churches across decades.

aioproductos 3mo ago

[flagged]