5 ms·

Ugh, another one of those not-very-insightful (or very well-informed) think pieces trying to sound interesting by being "contrarian" (even though they're repeat

Ugh, another one of those not-very-insightful (or very well-informed) think pieces trying to sound interesting by being "contrarian" (even though they're repeating the same idea you can find under many HN or reddit threads). If you think it's dangerous when bad people control an AI, how dangerous do you think it will be when an AI that has no concept of good or bad controls itself? An AI whose values are more alien than that of any sociopath, completely orthogonal to human values? If you don't think that is going to happen, then make a convincing argument for your case, the way AI "doomers" provide well thought-out arguments for their case: https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no... https://astralcodexten.substack.com/p/why-i-am-not-as-much-of-a-doomer https://astralcodexten.substack.com/p/why-i-am-not-as-much-o... https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a... https://www.youtube.com/@RobertMilesAI/videos https://www.youtube.com/@RobertMilesAI/videos

freeone3000 3y ago
All of these arguments are predicated on some advancement or change that has yet to happen. Widespread adoption of the existing technology will cause the problems in the posted article.
- comp_throw7 3y ago
  Sure is a good thing AI hasn't ever gotten better in unpredictable ways before.
- lumenwrites 3y ago
  Honestly if feels like we're living in the "Don't Look Up" movie. The asteroid hasn't hit the Earth yet, so what are you worried about? The reason people talk about x-risk from AI before the world-ending AI is developed, is because after the world-ending AI is developed we won't be around to talk about it.
  - danaris 3y ago
    There is, as yet, no evidence that AGI of any kind, let alone the world-ending kind, is even possible. The proposition that when AGI has been proven possible, it will be too late, effectively relies on the quasi-religious notion of the Singularity—and not merely that the Singularity will happen, but that the very first AGI we build will without any doubt initiate it as soon as it exists. This is not a proposition that is based on any actual evidence or logic.
    - lumenwrites 3y ago
      Evidence: - Human brains exist and are generally intelligent, therefore general intelligence is possible. - If we observe the progress in the AI field, it's pretty clear that AI systems are getting smarter at a fast pace. - Many AI systems we have developed quickly become superhuman (Go, StarCraft, many aspects of ChatGPT and Midjourney). - When superhuman intelligence competes against human intelligence, the thing that superintelligence wants to happen happens (e.g. it winning at Go). - The AI capabilities progress is way faster than AI alignment progress, we have no idea how to control AI systems or make them want the same things we do. Logic: - What the fuck do you think will happen when you coexist in a world with a superhuman intelligence that wants something different from what you want?
      - danaris 3y ago
        > - Many AI systems we have developed quickly become superhuman (Go, StarCraft, many aspects of ChatGPT and Midjourney). This is a) cherrypicking (there are plenty of "AI systems" we have developed that have failed to become superhuman), and b) not accounting for massive differences in the types of "AI systems" being talked about in the different cases. > - The AI capabilities progress is way faster than AI alignment progress, we have no idea how to control AI systems or make them want the same things we do. We know perfectly well how to prevent an AI from doing things that can either harm us directly, or allow it to rewrite its own code. Just because we don't know how to make an LLM cite its sources doesn't mean we don't know how to sandbox our code. "Control over AI systems" and "AI alignment" are not binary things. Nowhere in your "evidence" have you presented anything that says AGI is even possible. Nowhere in your "evidence" have you presented anything that says that, if AGI is possible, the very first time an AGI is created, it will seek to make itself superintelligent regardless of its creators' intentions, or that it will see itself as being in opposition to humanity in any way—that it will "want" anything beyond what it was created to want. Nowhere in your "evidence" have you presented anything that says that, if AGI is possible, and wants to become superintelligent, and sees humans as being its enemies, it will have the capability to improve itself in those ways faster than we can detect it. All of these missing links are science fiction scenarios. That's at least three (possibly four, depending on how you want to count them) potentially-impossible gaps that have to be crossed in order to get from where we are today to the nightmare scenario you posit.

freeone3000 3y ago

All of these arguments are predicated on some advancement or change that has yet to happen. Widespread adoption of the existing technology will cause the problems in the posted article.

comp_throw7 3y ago
Sure is a good thing AI hasn't ever gotten better in unpredictable ways before.
lumenwrites 3y ago
Honestly if feels like we're living in the "Don't Look Up" movie. The asteroid hasn't hit the Earth yet, so what are you worried about? The reason people talk about x-risk from AI before the world-ending AI is developed, is because after the world-ending AI is developed we won't be around to talk about it.
- danaris 3y ago
  There is, as yet, no evidence that AGI of any kind, let alone the world-ending kind, is even possible. The proposition that when AGI has been proven possible, it will be too late, effectively relies on the quasi-religious notion of the Singularity—and not merely that the Singularity will happen, but that the very first AGI we build will without any doubt initiate it as soon as it exists. This is not a proposition that is based on any actual evidence or logic.
  - lumenwrites 3y ago
    Evidence: - Human brains exist and are generally intelligent, therefore general intelligence is possible. - If we observe the progress in the AI field, it's pretty clear that AI systems are getting smarter at a fast pace. - Many AI systems we have developed quickly become superhuman (Go, StarCraft, many aspects of ChatGPT and Midjourney). - When superhuman intelligence competes against human intelligence, the thing that superintelligence wants to happen happens (e.g. it winning at Go). - The AI capabilities progress is way faster than AI alignment progress, we have no idea how to control AI systems or make them want the same things we do. Logic: - What the fuck do you think will happen when you coexist in a world with a superhuman intelligence that wants something different from what you want?
    - danaris 3y ago
      > - Many AI systems we have developed quickly become superhuman (Go, StarCraft, many aspects of ChatGPT and Midjourney). This is a) cherrypicking (there are plenty of "AI systems" we have developed that have failed to become superhuman), and b) not accounting for massive differences in the types of "AI systems" being talked about in the different cases. > - The AI capabilities progress is way faster than AI alignment progress, we have no idea how to control AI systems or make them want the same things we do. We know perfectly well how to prevent an AI from doing things that can either harm us directly, or allow it to rewrite its own code. Just because we don't know how to make an LLM cite its sources doesn't mean we don't know how to sandbox our code. "Control over AI systems" and "AI alignment" are not binary things. Nowhere in your "evidence" have you presented anything that says AGI is even possible. Nowhere in your "evidence" have you presented anything that says that, if AGI is possible, the very first time an AGI is created, it will seek to make itself superintelligent regardless of its creators' intentions, or that it will see itself as being in opposition to humanity in any way—that it will "want" anything beyond what it was created to want. Nowhere in your "evidence" have you presented anything that says that, if AGI is possible, and wants to become superintelligent, and sees humans as being its enemies, it will have the capability to improve itself in those ways faster than we can detect it. All of these missing links are science fiction scenarios. That's at least three (possibly four, depending on how you want to count them) potentially-impossible gaps that have to be crossed in order to get from where we are today to the nightmare scenario you posit.