3 ms·

Honestly if feels like we're living in the "Don't Look Up" movie. The asteroid hasn't hit the Earth yet, so what are you worried about? The reason people talk

Honestly if feels like we're living in the "Don't Look Up" movie. The asteroid hasn't hit the Earth yet, so what are you worried about? The reason people talk about x-risk from AI before the world-ending AI is developed, is because after the world-ending AI is developed we won't be around to talk about it.

danaris 3y ago
There is, as yet, no evidence that AGI of any kind, let alone the world-ending kind, is even possible. The proposition that when AGI has been proven possible, it will be too late, effectively relies on the quasi-religious notion of the Singularity—and not merely that the Singularity will happen, but that the very first AGI we build will without any doubt initiate it as soon as it exists. This is not a proposition that is based on any actual evidence or logic.
- lumenwrites 3y ago
  Evidence: - Human brains exist and are generally intelligent, therefore general intelligence is possible. - If we observe the progress in the AI field, it's pretty clear that AI systems are getting smarter at a fast pace. - Many AI systems we have developed quickly become superhuman (Go, StarCraft, many aspects of ChatGPT and Midjourney). - When superhuman intelligence competes against human intelligence, the thing that superintelligence wants to happen happens (e.g. it winning at Go). - The AI capabilities progress is way faster than AI alignment progress, we have no idea how to control AI systems or make them want the same things we do. Logic: - What the fuck do you think will happen when you coexist in a world with a superhuman intelligence that wants something different from what you want?
  - danaris 3y ago
    > - Many AI systems we have developed quickly become superhuman (Go, StarCraft, many aspects of ChatGPT and Midjourney). This is a) cherrypicking (there are plenty of "AI systems" we have developed that have failed to become superhuman), and b) not accounting for massive differences in the types of "AI systems" being talked about in the different cases. > - The AI capabilities progress is way faster than AI alignment progress, we have no idea how to control AI systems or make them want the same things we do. We know perfectly well how to prevent an AI from doing things that can either harm us directly, or allow it to rewrite its own code. Just because we don't know how to make an LLM cite its sources doesn't mean we don't know how to sandbox our code. "Control over AI systems" and "AI alignment" are not binary things. Nowhere in your "evidence" have you presented anything that says AGI is even possible. Nowhere in your "evidence" have you presented anything that says that, if AGI is possible, the very first time an AGI is created, it will seek to make itself superintelligent regardless of its creators' intentions, or that it will see itself as being in opposition to humanity in any way—that it will "want" anything beyond what it was created to want. Nowhere in your "evidence" have you presented anything that says that, if AGI is possible, and wants to become superintelligent, and sees humans as being its enemies, it will have the capability to improve itself in those ways faster than we can detect it. All of these missing links are science fiction scenarios. That's at least three (possibly four, depending on how you want to count them) potentially-impossible gaps that have to be crossed in order to get from where we are today to the nightmare scenario you posit.
jakelazaroff 3y ago
Okay, but in that analogy it would be like if people were sounding the alarm at the very beginning of the movie before the comet was even detected. Sure, it’s possible! Maybe even likely if civilization goes on long enough! That doesn’t mean it’s a pressing concern right now!

danaris 3y ago

There is, as yet, no evidence that AGI of any kind, let alone the world-ending kind, is even possible. The proposition that when AGI has been proven possible, it will be too late, effectively relies on the quasi-religious notion of the Singularity—and not merely that the Singularity will happen, but that the very first AGI we build will without any doubt initiate it as soon as it exists. This is not a proposition that is based on any actual evidence or logic.

lumenwrites 3y ago
Evidence: - Human brains exist and are generally intelligent, therefore general intelligence is possible. - If we observe the progress in the AI field, it's pretty clear that AI systems are getting smarter at a fast pace. - Many AI systems we have developed quickly become superhuman (Go, StarCraft, many aspects of ChatGPT and Midjourney). - When superhuman intelligence competes against human intelligence, the thing that superintelligence wants to happen happens (e.g. it winning at Go). - The AI capabilities progress is way faster than AI alignment progress, we have no idea how to control AI systems or make them want the same things we do. Logic: - What the fuck do you think will happen when you coexist in a world with a superhuman intelligence that wants something different from what you want?
- danaris 3y ago
  > - Many AI systems we have developed quickly become superhuman (Go, StarCraft, many aspects of ChatGPT and Midjourney). This is a) cherrypicking (there are plenty of "AI systems" we have developed that have failed to become superhuman), and b) not accounting for massive differences in the types of "AI systems" being talked about in the different cases. > - The AI capabilities progress is way faster than AI alignment progress, we have no idea how to control AI systems or make them want the same things we do. We know perfectly well how to prevent an AI from doing things that can either harm us directly, or allow it to rewrite its own code. Just because we don't know how to make an LLM cite its sources doesn't mean we don't know how to sandbox our code. "Control over AI systems" and "AI alignment" are not binary things. Nowhere in your "evidence" have you presented anything that says AGI is even possible. Nowhere in your "evidence" have you presented anything that says that, if AGI is possible, the very first time an AGI is created, it will seek to make itself superintelligent regardless of its creators' intentions, or that it will see itself as being in opposition to humanity in any way—that it will "want" anything beyond what it was created to want. Nowhere in your "evidence" have you presented anything that says that, if AGI is possible, and wants to become superintelligent, and sees humans as being its enemies, it will have the capability to improve itself in those ways faster than we can detect it. All of these missing links are science fiction scenarios. That's at least three (possibly four, depending on how you want to count them) potentially-impossible gaps that have to be crossed in order to get from where we are today to the nightmare scenario you posit.

jakelazaroff 3y ago

Okay, but in that analogy it would be like if people were sounding the alarm at the very beginning of the movie before the comet was even detected. Sure, it’s possible! Maybe even likely if civilization goes on long enough! That doesn’t mean it’s a pressing concern right now!