3 ms·

That's fair. Can you point me to more reasoned AI doomers? Maybe someone that doesn't think there is a greater than 99% chance humanity will be wiped by AI out

by bko 3y ago

That's fair. Can you point me to more reasoned AI doomers? Maybe someone that doesn't think there is a greater than 99% chance humanity will be wiped by AI out very soon. I honestly want to understand the risk. If you believe something wild to that degree of certainty, there's no other word for it. It's religious. That's the certainty that makes it religious and hysterical. "I believe with certainty the world will end very soon because [reason]" Fill in UFO, revelations, etc

digging 3y ago
> Can you point me to more reasoned AI doomers? I will try to this afternoon... I don't have any links handy on this computer. But I am not sure I will meet your criteria. I think we need to first agree on a simple set of premises. Do you agree with: 1. It is possible for humans to create superhuman AI with agency (the ability to set goals and enact them, affecting things in the real world) 2. It is not currently possible for us to predict the motivations and reasoning of the above AI with high confidence 3. It is possible for the above AI to set goals for itself which will be damaging or destructive to human civilization With these assertions out of the way, it's an open question how likely these dangerous goals are. Some argue they're more likely than not, which I tend to agree with by way of analogy. That analogy is influence-seeking human individuals and organizations. Humans and human organizations tend to have negative effects on their surroundings when seeking influence. Individuals and organizations which seek to grow their power, no matter their intent, tend to harm people and environments. There are many, many reasons for this, but I think they can largely be boiled down to the following: it is easier to enact harmful change than to enact harmless change. (Think about how many regulations we have, for example, on creating a new building, and how many safety and health hazards still happen. Besides that, the footprint of the building obliterates a natural ecosystem: this is intrinsic harm.) > If you believe something wild to that degree of certainty, there's no other word for it. It's religious. I don't agree, but I accept that you believe this. We might just have to agree to disagree here. But that makes the belief wild? Is it religious to believe in quantum mechanics, which IMO is far less intuitive than AI apocalypse? (You might say no because QM is extremely rigorously tested. But as an idea, QM is insane.) Fundamentally, AI "doomerism" is a logical belief, not one of pure faith. It is derived from logical reasoning. It is actually more similar to string theory than QM, now that I think about it - it is reasonable, and there are logical pathways to reach the belief, but it can't be falsified, so it's not scientific. Perhaps it lives in the area between religion and theory.

by bko 3y ago

digging 3y ago

> Can you point me to more reasoned AI doomers? I will try to this afternoon... I don't have any links handy on this computer. But I am not sure I will meet your criteria. I think we need to first agree on a simple set of premises. Do you agree with: 1. It is possible for humans to create superhuman AI with agency (the ability to set goals and enact them, affecting things in the real world) 2. It is not currently possible for us to predict the motivations and reasoning of the above AI with high confidence 3. It is possible for the above AI to set goals for itself which will be damaging or destructive to human civilization With these assertions out of the way, it's an open question how likely these dangerous goals are. Some argue they're more likely than not, which I tend to agree with by way of analogy. That analogy is influence-seeking human individuals and organizations. Humans and human organizations tend to have negative effects on their surroundings when seeking influence. Individuals and organizations which seek to grow their power, no matter their intent, tend to harm people and environments. There are many, many reasons for this, but I think they can largely be boiled down to the following: it is easier to enact harmful change than to enact harmless change. (Think about how many regulations we have, for example, on creating a new building, and how many safety and health hazards still happen. Besides that, the footprint of the building obliterates a natural ecosystem: this is intrinsic harm.) > If you believe something wild to that degree of certainty, there's no other word for it. It's religious. I don't agree, but I accept that you believe this. We might just have to agree to disagree here. But that makes the belief wild? Is it religious to believe in quantum mechanics, which IMO is far less intuitive than AI apocalypse? (You might say no because QM is extremely rigorously tested. But as an idea, QM is insane.) Fundamentally, AI "doomerism" is a logical belief, not one of pure faith. It is derived from logical reasoning. It is actually more similar to string theory than QM, now that I think about it - it is reasonable, and there are logical pathways to reach the belief, but it can't be falsified, so it's not scientific. Perhaps it lives in the area between religion and theory.