4 ms·

I don't think you're looking honestly at yourself... Look at all the harm caused by humans! It's more or less all of it! You might not be uniquely dangerous, b

I don't think you're looking honestly at yourself... Look at all the harm caused by humans! It's more or less all of it! You might not be uniquely dangerous, but "general intelligence with an internet connection" describes Bill Gates, Rupert Murdoch, and Elon Musk just as well. These individuals seem at least as dangerous as our current generation of AI. (Or do they? I'm not sure I agree with myself here...) Worth considering what makes an intelligence dangerous, because I don't think scale or even influence are the real problems here.

homefree 3y ago
The best framing of this imo is all of the other animal extinctions caused by humans - not because humans hate these animals, but because humans were focused on other goals without really thinking of them. In the case of an unaligned AGI the goals don't even have to be worthwhile - they can be something that accidentally satisfies its reward function by mistake.
wonderwonder 3y ago
Not sure why you chose Bill Gates, Rupert Murdoch, and Elon Musk when you could have chosen people like Stalin or Hitler. People that used their influence to murder millions in much the same way that a AGI could likely use its influence on a naïve population.
- itishappy 3y ago
  I didn't give it much thought, tbh, just picked famous/controversial people. Goebbels was actually my first thought, but he didn't have internet.
godelski 3y ago
Yeah I'm really impressed about the lack of self awareness in these comments. People are incredibly dangerous. There's a reason we took control of the planet. Even any single given human is incredibly dangerous and can kill many others (human or non human, and we routinely kill non humans). You can go pick up a gun and just start shooting people. You could go grab a knife and stab dozens in the street. You could easily build explosives. Hell, you could throw fucking buckets of sharp metal objects on a freeway and kill hundreds directly and indirectly. Our intelligence and biology has provided us with many means to create or obtain force multipliers. I'm really glad the vast majority of humans have decided not to be particularly dangerous (at least to other humans). To think we aren't dangerous is just incredibly naive. A big reason we aren't is because ingrained into our biology is a literal mechanism to not want to harm others. There's definitely ways to bypass these mechanisms and there are definitely faulty or poorly functioning ones out there, but most humans have it. Oddly this might even be the most dangerous thing we do too, because the definition of a coalition is that the utility is higher than the utility of each individual component.

homefree 3y ago

The best framing of this imo is all of the other animal extinctions caused by humans - not because humans hate these animals, but because humans were focused on other goals without really thinking of them. In the case of an unaligned AGI the goals don't even have to be worthwhile - they can be something that accidentally satisfies its reward function by mistake.

wonderwonder 3y ago

Not sure why you chose Bill Gates, Rupert Murdoch, and Elon Musk when you could have chosen people like Stalin or Hitler. People that used their influence to murder millions in much the same way that a AGI could likely use its influence on a naïve population.

itishappy 3y ago
I didn't give it much thought, tbh, just picked famous/controversial people. Goebbels was actually my first thought, but he didn't have internet.

godelski 3y ago

Yeah I'm really impressed about the lack of self awareness in these comments. People are incredibly dangerous. There's a reason we took control of the planet. Even any single given human is incredibly dangerous and can kill many others (human or non human, and we routinely kill non humans). You can go pick up a gun and just start shooting people. You could go grab a knife and stab dozens in the street. You could easily build explosives. Hell, you could throw fucking buckets of sharp metal objects on a freeway and kill hundreds directly and indirectly. Our intelligence and biology has provided us with many means to create or obtain force multipliers. I'm really glad the vast majority of humans have decided not to be particularly dangerous (at least to other humans). To think we aren't dangerous is just incredibly naive. A big reason we aren't is because ingrained into our biology is a literal mechanism to not want to harm others. There's definitely ways to bypass these mechanisms and there are definitely faulty or poorly functioning ones out there, but most humans have it. Oddly this might even be the most dangerous thing we do too, because the definition of a coalition is that the utility is higher than the utility of each individual component.