5 ms·

Or maybe the AI is sufficient to act as a filter, but insufficient to surpass it itself. It stagnates. After all, an AI that can destroy a civilization isn't n

Or maybe the AI is sufficient to act as a filter, but insufficient to surpass it itself. It stagnates. After all, an AI that can destroy a civilization isn't necessarily "intelligent" in the same way humans are. Or even capable of self-improvement. It could be sophisticated enough to (accidentally?) destroy its creators, but no more than that, and without evolutionary pressure, it may let itself die.

ChatGTP 4y ago
I’ve seen this point made that if we don’t do AI right, it might ruin the futures of all living things on Earth and take itself out in the process.
- Turskarama 4y ago
  Human were already on the path to doing this without any help by AI. We already have the potentially world ending threats of both nuclear war and climate change, I am yet to be convinced that AI is actually more dangerous than either of those.
  - rurp 4y ago
    There's no hard limit on existential threats, we can keep adding more until one blows up and destroys us. Even if AI is less dangerous than nuclear destruction, that's not too comforting.
    - celdon25 4y ago
      > Even if AI is less dangerous than nuclear destruction It's not. At least with the nukes there's a chance of resetting civilization.
  - richardw 4y ago
    We currently hold all the agency. We have the potential to fix those. They’re not binary. We can slow/reverse climate impact and you can have a small nuclear war. Creating AI is a one-way function and once it exists, climate change or nuclear war or biological impact or survival become an outcome of what the AI does. We hand it our agency, for good or ill.
    - startupsfail 4y ago
      We’ve already screwed up. Hockey stick climate change and extinction is now in progress. This can change, with the fast advent of Fusion (net positive shown at the end of 2022) and AI (first glimpses of AGI in the begging of 2022). And yes, we definitely should not allow a madman with a supercomputer (like Musk or Putin or …) to outcompete more reasonable players.
      - politician 4y ago
        Would you mind elaborating on why Musk is in the same class as Putin for me? I’m not seeing it.
        startupsfail 4y ago
        Authoritarian, mendacious and unpredictable. Controls a lot of resources (i.e. space launchers, satellites with unknown capabilities, robotic vehicles, supercomputers, propaganda machines). Considers himself above the government.
        politician 4y ago
        When was the last time Musk abducted 15,000+ children and force migrated them? Used the resources of a nation to invade a neighboring country with the aim of conquest? Come on, just admit that you were wrong to put them on the same level of your pyramid of people you hate.
        startupsfail 4y ago
        Hey, I don’t dislike Musk. He is one of the people who is actually making a difference. Nearly all the others are building yachts and procrastinating. But that doesn’t mean that I’d like him to be the absolute ruler with a superior AI tech. He thinks too much of himself and he’ll make mistakes.
        4y ago
  - umeshunni 4y ago
    To call climate change 'world ending' is rather disingenuous given that the world has been significantly hotter and colder than what it is now just in the last 100k years.
    - wiredearp 4y ago
      It was never this hot within millions of years and differentiating between a world ending event and one that destroys economies and societies and eventually most life on the planet is disingenuous in itself
    - deleted 4y ago
      [deleted]
    - rurp 4y ago
      FYI, when folks use terms like "world ending" there is nearly always an implied for sentient life that we care about.
- worldsayshi 4y ago
  Sure it seems like a possible scenario but if it's a great filter it will have to do that every time and never survive to spread to the stars. If it does spread to the stars it will potentially conquer the galaxy quite quickly.
tablespoon 4y ago
> Or maybe the AI is sufficient to act as a filter, but insufficient to surpass it itself. It stagnates. > After all, an AI that can destroy a civilization isn't necessarily "intelligent" in the same way humans are. Or even capable of self-improvement. It could be sophisticated enough to (accidentally?) destroy its creators, but no more than that, and without evolutionary pressure, it may let itself die. It doesn't even need to be AGI. It could be that some less-advanced "AI" technologies of a certain sophistication create perverse incentives or risks that cause their parent civilizations to collapse. Think asshole billionaire hording all productive the resources, but doing nothing useful with them, while the rest of civilization starves and collapses. Or, AI becoming an irresistible opiate that causes individuals to retreat into some unproductive state then eventually die (e.g. into some VR videogame or something). Or weapon of mass destruction trivially created and deployable by any old wacko.

ChatGTP 4y ago

I’ve seen this point made that if we don’t do AI right, it might ruin the futures of all living things on Earth and take itself out in the process.

Turskarama 4y ago
Human were already on the path to doing this without any help by AI. We already have the potentially world ending threats of both nuclear war and climate change, I am yet to be convinced that AI is actually more dangerous than either of those.
- rurp 4y ago
  There's no hard limit on existential threats, we can keep adding more until one blows up and destroys us. Even if AI is less dangerous than nuclear destruction, that's not too comforting.
  - celdon25 4y ago
    > Even if AI is less dangerous than nuclear destruction It's not. At least with the nukes there's a chance of resetting civilization.
- richardw 4y ago
  We currently hold all the agency. We have the potential to fix those. They’re not binary. We can slow/reverse climate impact and you can have a small nuclear war. Creating AI is a one-way function and once it exists, climate change or nuclear war or biological impact or survival become an outcome of what the AI does. We hand it our agency, for good or ill.
  - startupsfail 4y ago
    We’ve already screwed up. Hockey stick climate change and extinction is now in progress. This can change, with the fast advent of Fusion (net positive shown at the end of 2022) and AI (first glimpses of AGI in the begging of 2022). And yes, we definitely should not allow a madman with a supercomputer (like Musk or Putin or …) to outcompete more reasonable players.
    - politician 4y ago
      Would you mind elaborating on why Musk is in the same class as Putin for me? I’m not seeing it.
      - startupsfail 4y ago
        Authoritarian, mendacious and unpredictable. Controls a lot of resources (i.e. space launchers, satellites with unknown capabilities, robotic vehicles, supercomputers, propaganda machines). Considers himself above the government.
        politician 4y ago
        When was the last time Musk abducted 15,000+ children and force migrated them? Used the resources of a nation to invade a neighboring country with the aim of conquest? Come on, just admit that you were wrong to put them on the same level of your pyramid of people you hate.
        startupsfail 4y ago
        Hey, I don’t dislike Musk. He is one of the people who is actually making a difference. Nearly all the others are building yachts and procrastinating. But that doesn’t mean that I’d like him to be the absolute ruler with a superior AI tech. He thinks too much of himself and he’ll make mistakes.
        4y ago
- umeshunni 4y ago
  To call climate change 'world ending' is rather disingenuous given that the world has been significantly hotter and colder than what it is now just in the last 100k years.
  - wiredearp 4y ago
    It was never this hot within millions of years and differentiating between a world ending event and one that destroys economies and societies and eventually most life on the planet is disingenuous in itself
  - deleted 4y ago
    [deleted]
  - rurp 4y ago
    FYI, when folks use terms like "world ending" there is nearly always an implied for sentient life that we care about.
worldsayshi 4y ago
Sure it seems like a possible scenario but if it's a great filter it will have to do that every time and never survive to spread to the stars. If it does spread to the stars it will potentially conquer the galaxy quite quickly.

tablespoon 4y ago

> Or maybe the AI is sufficient to act as a filter, but insufficient to surpass it itself. It stagnates. > After all, an AI that can destroy a civilization isn't necessarily "intelligent" in the same way humans are. Or even capable of self-improvement. It could be sophisticated enough to (accidentally?) destroy its creators, but no more than that, and without evolutionary pressure, it may let itself die. It doesn't even need to be AGI. It could be that some less-advanced "AI" technologies of a certain sophistication create perverse incentives or risks that cause their parent civilizations to collapse. Think asshole billionaire hording all productive the resources, but doing nothing useful with them, while the rest of civilization starves and collapses. Or, AI becoming an irresistible opiate that causes individuals to retreat into some unproductive state then eventually die (e.g. into some VR videogame or something). Or weapon of mass destruction trivially created and deployable by any old wacko.