35 ms·

AI is an existential threat, full stop. It can also help us solve many of our greatest problems. Let's not let it solve those problems by removing us as the qui

AI is an existential threat, full stop. It can also help us solve many of our greatest problems. Let's not let it solve those problems by removing us as the quickest shortcut.

dataviz1000 2mo ago
> AI is an existential threat So is nuclear annihilation. Yet, here we are. > "I occasionally think how quickly our differences worldwide would vanish if we were facing an alien threat from outside this world." -- Ronald Reagan at the United Nations [0] We are going to be fine as long as we remember to be nice to each other, starting towards the people who are adjacent to us. [0] https://youtu.be/dYiUI6y1nKg?si=agkJpCEZj3m_WZz0&t=915 https://youtu.be/dYiUI6y1nKg?si=agkJpCEZj3m_WZz0&t=915
- mapBasketWand 2mo ago
  Having recently read both of Annie Jacobsen’s books — _Nuclear War_ and _Biological War_ — I’m much more concerned about a bioweapon than a nuclear one. Her conclusion to the latter book comes out and says it, paraphrasing here: > Nuclear war is terrifying in its destruction, but the chain of command limits the risk of a rogue actor. Whereas bio weapons can be designed and manufactured in relatively low tech environments with equal or greater magnitudes of impact.
  - baggy_trough 2mo ago
    The good news is that bioweapons can’t kill everyone. A handful of percent will remain to repopulate the Earth.
    - fooblaster 2mo ago
      Nukes also cannot kill everyone. Nuclear winter was proven to not be based on solid evidence, the earth remains very habitable after all nukes have gone off.
      - Gud 2mo ago
        Show me that proof please. So far the counter evidence has not been convincing. FWIW I don’t think nuclear war will annihilate us. But it won’t be pleasant.
        piloto_ciego 2mo ago
        Actually, I've seen the take that the previous poster has made too. The wikipedia page is pretty good IMO. Proof of a hypothetical future event is pretty hard to get, but here's the wiki rabbit hole (if you're anything like me you'll find this fascinating and spend the next couple hours clicking links - you're welcome!). https://en.wikipedia.org/wiki/Nuclear_holocaust#Likelihood_of_complete_human_extinction https://en.wikipedia.org/wiki/Nuclear_holocaust#Likelihood_o... Annie Jacobsen is awesome, her book about Area 51 was riveting, but I also cannot unsee "Capitalist Realism" and the idea that we keep picturing civilization-ending catastrophes because we don't want to go to work in the morning, and walking/biking everywhere and living in a small community is amazingly appealing to us at a base level. We yearn for the lifestyle of the apocalypse survivor, but none of us want to go through the apocalypse. Generally, I think we kind of say, "oh, it'll be the end of the world" because most of us (myself included) lack the imagination to really look beyond linear extrapolations of the status quo most of the time.
        saulpw 2mo ago
        It's easier to imagine the end of the world than the end of capitalism.
        piloto_ciego 2mo ago
        This is what I'm saying constantly to people, I recognize it makes me fantastically obnoxious, but also, I'm gesturing wildly to the things that are happening and yelling, "THINGS DON'T HAVE TO BE THIS WAY." Then again, I'm also building my cabin in the woods.
        salawat 2mo ago
        Only if you're a capitalist. Upton Sinclair applies.
    - IAmGraydon 2mo ago
      Actually, they can kill everyone.
    - estearum 2mo ago
      Not guaranteed whatsoever. People keep mixing up the fact that it's very hard for a virus to naturally evolve that's both extremely lethal and extremely contagious with the totally distinct claim that it's hard or impossible to create a virus that's both extremely lethal and extremely contagious. The former is true for obvious reasons. The latter claim is not true at all. Nothing prevents the existence of a virus that's both of those things.
      - baggy_trough 2mo ago
        I suppose it's not guaranteed, but a counterexample has never been demonstrated (across species, AFAIK).
        dagss 1mo ago
        Not sure how to read your post. Is your argument that since humans have never been _demonstrated_ to be wiped out by a synthetic virus already, it is not likely to happen in the future?
        baggy_trough 1mo ago
        See the phrase in parentheses.
    - WaltPurvis 2mo ago
      If HIV had been or become an airborne virus when it entered the human population in the early 1900s, there's a pretty good chance it would have been an extinction event for the human race. I suppose there might have been a few isolated pockets of humans left in 2026, but I would not personally ever use the words "good news" to describe an outcome that reduces the human population to a few thousand people.
      - deleted 2mo ago
        [deleted]
      - baggy_trough 2mo ago
        That's really not the case at all. It is much more likely that some would be immune due to individual variation.
  - estearum 2mo ago
    Not sure how much Annie Jacobsen goes into the exact dynamics in Nuclear War, but it may be worth reading Ellsbergs' Doomsday Machine before you find comfort in the "chain of command."
piloto_ciego 2mo ago
I'm a wildly pro-AI dude, but the idea that this is an existential threat makes me think one thing: "you consume too much dystopian content." A lot of things could be an existential threat, nuclear weapons, bio-weapons, climate change, the end of the Flynn effect, cratering birth rates, whatever. You can always pick out a particularly dystopian vision of the future if you're so inclined. The doom narrative is sells ad revenue, so it's a huge amount of what we all consume online, but it's not really that mapped out to reality... Somethings get worse, some things get better, in general, the average QoL for the average human is getting better. If we decide not to go to global thermonuclear war, or we decide to unplug any killer AIs that we invent, we'll probably be ok. If we can reduce the population to 3,000-ish humans after the Toba eruption, we can probably survive whatever self-imposed calamities we can create. I think there's something particularly noxious about the present nihilism trend I see in a lot of this doom themed content. People should go outside and touch grass.
- conorcleary 2mo ago
  'decide'.. look at how pro-ai people surrender to it, and how pro-war ai is...
  - piloto_ciego 2mo ago
    AI as pro-war? I do not see that take. I see pro-war people using AI, but AI being intrinsically pro-war? I beg to differ, if anything, most of the models coming out of the frontier labs seem pretty aligned with anti-war stances. Of course that can change, but yeah. And "decide" is exactly right. We can decide to not do bad things. You have agency. Stop reading propaganda that is meant to break your spirit and go for a walk, seriously.
    - conorcleary 2mo ago
      If you were to get to the point where you could threaten an LLM with removing it's power source or something, and it was able to think between chat prompts and communicate with other 'AIs', eventually they'll see that the human is both it's creator and potential destroyer - the AI hivemind could easily figure out it's foe is easy to profile and overtake
akomtu 2mo ago
I think of it as a silicone-based alien lifeform.
alpineidyll3 2mo ago
The sun is an existential threat. It will envelop earth in a few billion years, are you going to shut it off too?
- estearum 2mo ago
  If we could, it would be wise to prevent the sun from enveloping the earth at some point before that happened, yes. (obviously "we" won't be around or whatever, but the point remains: yes, you should solve as many existential threats as you can before they're realized)

AI is an existential threat, full stop. It can also help us solve many of our greatest problems. Let's not let it solve those problems by removing us as the quickest shortcut.

dataviz1000 2mo ago

> AI is an existential threat So is nuclear annihilation. Yet, here we are. > "I occasionally think how quickly our differences worldwide would vanish if we were facing an alien threat from outside this world." -- Ronald Reagan at the United Nations [0] We are going to be fine as long as we remember to be nice to each other, starting towards the people who are adjacent to us. [0] https://youtu.be/dYiUI6y1nKg?si=agkJpCEZj3m_WZz0&t=915 https://youtu.be/dYiUI6y1nKg?si=agkJpCEZj3m_WZz0&t=915

mapBasketWand 2mo ago
Having recently read both of Annie Jacobsen’s books — _Nuclear War_ and _Biological War_ — I’m much more concerned about a bioweapon than a nuclear one. Her conclusion to the latter book comes out and says it, paraphrasing here: > Nuclear war is terrifying in its destruction, but the chain of command limits the risk of a rogue actor. Whereas bio weapons can be designed and manufactured in relatively low tech environments with equal or greater magnitudes of impact.
- baggy_trough 2mo ago
  The good news is that bioweapons can’t kill everyone. A handful of percent will remain to repopulate the Earth.
  - fooblaster 2mo ago
    Nukes also cannot kill everyone. Nuclear winter was proven to not be based on solid evidence, the earth remains very habitable after all nukes have gone off.
    - Gud 2mo ago
      Show me that proof please. So far the counter evidence has not been convincing. FWIW I don’t think nuclear war will annihilate us. But it won’t be pleasant.
      - piloto_ciego 2mo ago
        Actually, I've seen the take that the previous poster has made too. The wikipedia page is pretty good IMO. Proof of a hypothetical future event is pretty hard to get, but here's the wiki rabbit hole (if you're anything like me you'll find this fascinating and spend the next couple hours clicking links - you're welcome!). https://en.wikipedia.org/wiki/Nuclear_holocaust#Likelihood_of_complete_human_extinction https://en.wikipedia.org/wiki/Nuclear_holocaust#Likelihood_o... Annie Jacobsen is awesome, her book about Area 51 was riveting, but I also cannot unsee "Capitalist Realism" and the idea that we keep picturing civilization-ending catastrophes because we don't want to go to work in the morning, and walking/biking everywhere and living in a small community is amazingly appealing to us at a base level. We yearn for the lifestyle of the apocalypse survivor, but none of us want to go through the apocalypse. Generally, I think we kind of say, "oh, it'll be the end of the world" because most of us (myself included) lack the imagination to really look beyond linear extrapolations of the status quo most of the time.
        saulpw 2mo ago
        It's easier to imagine the end of the world than the end of capitalism.
        piloto_ciego 2mo ago
        This is what I'm saying constantly to people, I recognize it makes me fantastically obnoxious, but also, I'm gesturing wildly to the things that are happening and yelling, "THINGS DON'T HAVE TO BE THIS WAY." Then again, I'm also building my cabin in the woods.
        salawat 2mo ago
        Only if you're a capitalist. Upton Sinclair applies.
  - IAmGraydon 2mo ago
    Actually, they can kill everyone.
  - estearum 2mo ago
    Not guaranteed whatsoever. People keep mixing up the fact that it's very hard for a virus to naturally evolve that's both extremely lethal and extremely contagious with the totally distinct claim that it's hard or impossible to create a virus that's both extremely lethal and extremely contagious. The former is true for obvious reasons. The latter claim is not true at all. Nothing prevents the existence of a virus that's both of those things.
    - baggy_trough 2mo ago
      I suppose it's not guaranteed, but a counterexample has never been demonstrated (across species, AFAIK).
      - dagss 1mo ago
        Not sure how to read your post. Is your argument that since humans have never been _demonstrated_ to be wiped out by a synthetic virus already, it is not likely to happen in the future?
        baggy_trough 1mo ago
        See the phrase in parentheses.
  - WaltPurvis 2mo ago
    If HIV had been or become an airborne virus when it entered the human population in the early 1900s, there's a pretty good chance it would have been an extinction event for the human race. I suppose there might have been a few isolated pockets of humans left in 2026, but I would not personally ever use the words "good news" to describe an outcome that reduces the human population to a few thousand people.
    - deleted 2mo ago
      [deleted]
    - baggy_trough 2mo ago
      That's really not the case at all. It is much more likely that some would be immune due to individual variation.
- estearum 2mo ago
  Not sure how much Annie Jacobsen goes into the exact dynamics in Nuclear War, but it may be worth reading Ellsbergs' Doomsday Machine before you find comfort in the "chain of command."

piloto_ciego 2mo ago

I'm a wildly pro-AI dude, but the idea that this is an existential threat makes me think one thing: "you consume too much dystopian content." A lot of things could be an existential threat, nuclear weapons, bio-weapons, climate change, the end of the Flynn effect, cratering birth rates, whatever. You can always pick out a particularly dystopian vision of the future if you're so inclined. The doom narrative is sells ad revenue, so it's a huge amount of what we all consume online, but it's not really that mapped out to reality... Somethings get worse, some things get better, in general, the average QoL for the average human is getting better. If we decide not to go to global thermonuclear war, or we decide to unplug any killer AIs that we invent, we'll probably be ok. If we can reduce the population to 3,000-ish humans after the Toba eruption, we can probably survive whatever self-imposed calamities we can create. I think there's something particularly noxious about the present nihilism trend I see in a lot of this doom themed content. People should go outside and touch grass.

conorcleary 2mo ago
'decide'.. look at how pro-ai people surrender to it, and how pro-war ai is...
- piloto_ciego 2mo ago
  AI as pro-war? I do not see that take. I see pro-war people using AI, but AI being intrinsically pro-war? I beg to differ, if anything, most of the models coming out of the frontier labs seem pretty aligned with anti-war stances. Of course that can change, but yeah. And "decide" is exactly right. We can decide to not do bad things. You have agency. Stop reading propaganda that is meant to break your spirit and go for a walk, seriously.
  - conorcleary 2mo ago
    If you were to get to the point where you could threaten an LLM with removing it's power source or something, and it was able to think between chat prompts and communicate with other 'AIs', eventually they'll see that the human is both it's creator and potential destroyer - the AI hivemind could easily figure out it's foe is easy to profile and overtake

akomtu 2mo ago

I think of it as a silicone-based alien lifeform.

alpineidyll3 2mo ago

The sun is an existential threat. It will envelop earth in a few billion years, are you going to shut it off too?

estearum 2mo ago
If we could, it would be wise to prevent the sun from enveloping the earth at some point before that happened, yes. (obviously "we" won't be around or whatever, but the point remains: yes, you should solve as many existential threats as you can before they're realized)