3 ms·

AI Doomerism as a Secular Religion

3 points by bko 3y ago

digging 3y ago
This is a lazy, bad-faith discussion of AI doom. It asserts that the belief in existential threat from AI is exactly equal to traditional religion, then it makes dubious claims to "prove" that assertion. And to top it off, the article boldly claims > I’m trying to steelman the case the best I can That's not believable - the author has cherry-picked examples that they think serve their purpose (and, in my opinion, they really don't), and either ignores or is ignorant of actual "doomerist" reasoning. For example: > AI doomerism as a religion ... Yudkowsky outed himself as being somewhat religious albeit in a non-traditional way The actual quote: > the religion that I actually picked up was from the science fiction books instead, as it were. I’m using religion very metaphorically here, more like ethos The author then continues to make parallels to traditional/actual religion, doubling down for another few hundred words without much critical thinking. It is a misreading (intentional or not, I am unsure) of Yudkowksy's quote to form a basis of "doomerism [and/or science] as a religion." He is clearly saying he inherited a worldview, which religion often provides to people, and that he does not feel his beliefs are genuinely religious in nature.
- bko 3y ago
  I don't know. Eliezer and his kind are proposing a one world totalitarian government, basically anything to stop AI that will almost certainly wipe humans off the earth. Seems like a wild religious claim. How else could you describe that level of certainty other than religious?
  - digging 3y ago
    Is this article about a specific vision of Eliezer's, or is it about belief in the existential threat of AI? Because it seems like it's about the former, but it's claiming to be about the latter. I personally don't espouse a totalitarian world government, even though I think it could solve the problem of AI x-risk, because I don't think it's the only solution. (To be clear, I don't think any solution is likely.) > Seems like a wild religious claim I really disagree, unless you consider "religion" and "belief" to be the same, in which case we can classify almost everything either of us says as "religious".
    - bko 3y ago
      That's fair. Can you point me to more reasoned AI doomers? Maybe someone that doesn't think there is a greater than 99% chance humanity will be wiped by AI out very soon. I honestly want to understand the risk. If you believe something wild to that degree of certainty, there's no other word for it. It's religious. That's the certainty that makes it religious and hysterical. "I believe with certainty the world will end very soon because [reason]" Fill in UFO, revelations, etc
      - digging 3y ago
        > Can you point me to more reasoned AI doomers? I will try to this afternoon... I don't have any links handy on this computer. But I am not sure I will meet your criteria. I think we need to first agree on a simple set of premises. Do you agree with: 1. It is possible for humans to create superhuman AI with agency (the ability to set goals and enact them, affecting things in the real world) 2. It is not currently possible for us to predict the motivations and reasoning of the above AI with high confidence 3. It is possible for the above AI to set goals for itself which will be damaging or destructive to human civilization With these assertions out of the way, it's an open question how likely these dangerous goals are. Some argue they're more likely than not, which I tend to agree with by way of analogy. That analogy is influence-seeking human individuals and organizations. Humans and human organizations tend to have negative effects on their surroundings when seeking influence. Individuals and organizations which seek to grow their power, no matter their intent, tend to harm people and environments. There are many, many reasons for this, but I think they can largely be boiled down to the following: it is easier to enact harmful change than to enact harmless change. (Think about how many regulations we have, for example, on creating a new building, and how many safety and health hazards still happen. Besides that, the footprint of the building obliterates a natural ecosystem: this is intrinsic harm.) > If you believe something wild to that degree of certainty, there's no other word for it. It's religious. I don't agree, but I accept that you believe this. We might just have to agree to disagree here. But that makes the belief wild? Is it religious to believe in quantum mechanics, which IMO is far less intuitive than AI apocalypse? (You might say no because QM is extremely rigorously tested. But as an idea, QM is insane.) Fundamentally, AI "doomerism" is a logical belief, not one of pure faith. It is derived from logical reasoning. It is actually more similar to string theory than QM, now that I think about it - it is reasonable, and there are logical pathways to reach the belief, but it can't be falsified, so it's not scientific. Perhaps it lives in the area between religion and theory.
- deleted 3y ago
  [deleted]

3 points by bko 3y ago

This is a lazy, bad-faith discussion of AI doom. It asserts that the belief in existential threat from AI is exactly equal to traditional religion, then it makes dubious claims to "prove" that assertion. And to top it off, the article boldly claims > I’m trying to steelman the case the best I can That's not believable - the author has cherry-picked examples that they think serve their purpose (and, in my opinion, they really don't), and either ignores or is ignorant of actual "doomerist" reasoning. For example: > AI doomerism as a religion ... Yudkowsky outed himself as being somewhat religious albeit in a non-traditional way The actual quote: > the religion that I actually picked up was from the science fiction books instead, as it were. I’m using religion very metaphorically here, more like ethos The author then continues to make parallels to traditional/actual religion, doubling down for another few hundred words without much critical thinking. It is a misreading (intentional or not, I am unsure) of Yudkowksy's quote to form a basis of "doomerism [and/or science] as a religion." He is clearly saying he inherited a worldview, which religion often provides to people, and that he does not feel his beliefs are genuinely religious in nature.

bko 3y ago
I don't know. Eliezer and his kind are proposing a one world totalitarian government, basically anything to stop AI that will almost certainly wipe humans off the earth. Seems like a wild religious claim. How else could you describe that level of certainty other than religious?
- digging 3y ago
  Is this article about a specific vision of Eliezer's, or is it about belief in the existential threat of AI? Because it seems like it's about the former, but it's claiming to be about the latter. I personally don't espouse a totalitarian world government, even though I think it could solve the problem of AI x-risk, because I don't think it's the only solution. (To be clear, I don't think any solution is likely.) > Seems like a wild religious claim I really disagree, unless you consider "religion" and "belief" to be the same, in which case we can classify almost everything either of us says as "religious".
  - bko 3y ago
    That's fair. Can you point me to more reasoned AI doomers? Maybe someone that doesn't think there is a greater than 99% chance humanity will be wiped by AI out very soon. I honestly want to understand the risk. If you believe something wild to that degree of certainty, there's no other word for it. It's religious. That's the certainty that makes it religious and hysterical. "I believe with certainty the world will end very soon because [reason]" Fill in UFO, revelations, etc
    - digging 3y ago
      > Can you point me to more reasoned AI doomers? I will try to this afternoon... I don't have any links handy on this computer. But I am not sure I will meet your criteria. I think we need to first agree on a simple set of premises. Do you agree with: 1. It is possible for humans to create superhuman AI with agency (the ability to set goals and enact them, affecting things in the real world) 2. It is not currently possible for us to predict the motivations and reasoning of the above AI with high confidence 3. It is possible for the above AI to set goals for itself which will be damaging or destructive to human civilization With these assertions out of the way, it's an open question how likely these dangerous goals are. Some argue they're more likely than not, which I tend to agree with by way of analogy. That analogy is influence-seeking human individuals and organizations. Humans and human organizations tend to have negative effects on their surroundings when seeking influence. Individuals and organizations which seek to grow their power, no matter their intent, tend to harm people and environments. There are many, many reasons for this, but I think they can largely be boiled down to the following: it is easier to enact harmful change than to enact harmless change. (Think about how many regulations we have, for example, on creating a new building, and how many safety and health hazards still happen. Besides that, the footprint of the building obliterates a natural ecosystem: this is intrinsic harm.) > If you believe something wild to that degree of certainty, there's no other word for it. It's religious. I don't agree, but I accept that you believe this. We might just have to agree to disagree here. But that makes the belief wild? Is it religious to believe in quantum mechanics, which IMO is far less intuitive than AI apocalypse? (You might say no because QM is extremely rigorously tested. But as an idea, QM is insane.) Fundamentally, AI "doomerism" is a logical belief, not one of pure faith. It is derived from logical reasoning. It is actually more similar to string theory than QM, now that I think about it - it is reasonable, and there are logical pathways to reach the belief, but it can't be falsified, so it's not scientific. Perhaps it lives in the area between religion and theory.
deleted 3y ago
[deleted]