3 ms·

I don't think that's the best defense, because even if 99.999% of programmers are perfect, ethical, decent people, all it takes is one psycho to go ahead and le

by NickM 9y ago

I don't think that's the best defense, because even if 99.999% of programmers are perfect, ethical, decent people, all it takes is one psycho to go ahead and let the genie out of the bottle anyway. Imagine there's a big red "end the world" button that anyone on the planet can push. No matter how idealistic and utopian your views are, do you really trust that there's not a single person out of 7 billion people that might push the button? That is the kind of situation I worry we may face once people figure out how to create strong AI.

bem94 9y ago
Turning what I think you are saying on its head: There is a 0.001% chance of something bad happening, so there is no point in the other 99.999% changing their behaviour in order to stop that bad thing happening. I don't think that follows either. Second guessing human nature to the point where you resign yourself to catastrophe is surely less worthwhile than striving to create a sense of collective responsibility before it is too late?
- NickM 9y ago
  I'm not saying there's an 0.0001% chance of it happening, I'm saying that if you give enough people a chance to create an AI, eventually someone will do it, even if 99.999% of people would choose not to. Right now, nobody knows how to do this, but my point is that eventually people will figure out how: once everyone knows how, then it's just a matter of time before someone does it. I'm not second guessing human nature, just saying people do crazy things all the time. The vast, vast majority of people are not suicide bombers, for example, but it still happens sometimes. It doesn't matter how much collective responsibility we have; eventually you may just end up with a brilliant programmer with schizophrenia or something who writes an AI in his basement and unleashes it on the world. I'm also certainly not resigning myself to catastrophe. There are probably ways to prevent or mitigate the negative impact of AI, like the one being suggested in the original article, for instance. I would hardly consider myself a cynic, but forgive me if I'm skeptical of the idea that every human in the world can somehow agree to never do anything that might create a dangerous AI. If it can happen, it eventually will happen.

by NickM 9y ago

bem94 9y ago

Turning what I think you are saying on its head: There is a 0.001% chance of something bad happening, so there is no point in the other 99.999% changing their behaviour in order to stop that bad thing happening. I don't think that follows either. Second guessing human nature to the point where you resign yourself to catastrophe is surely less worthwhile than striving to create a sense of collective responsibility before it is too late?

NickM 9y ago
I'm not saying there's an 0.0001% chance of it happening, I'm saying that if you give enough people a chance to create an AI, eventually someone will do it, even if 99.999% of people would choose not to. Right now, nobody knows how to do this, but my point is that eventually people will figure out how: once everyone knows how, then it's just a matter of time before someone does it. I'm not second guessing human nature, just saying people do crazy things all the time. The vast, vast majority of people are not suicide bombers, for example, but it still happens sometimes. It doesn't matter how much collective responsibility we have; eventually you may just end up with a brilliant programmer with schizophrenia or something who writes an AI in his basement and unleashes it on the world. I'm also certainly not resigning myself to catastrophe. There are probably ways to prevent or mitigate the negative impact of AI, like the one being suggested in the original article, for instance. I would hardly consider myself a cynic, but forgive me if I'm skeptical of the idea that every human in the world can somehow agree to never do anything that might create a dangerous AI. If it can happen, it eventually will happen.