7 ms·

My parents are religious fundamentalists, and I've noticed there is increasing FUD being drummed up in their circles about anything to do with AI or significant

My parents are religious fundamentalists, and I've noticed there is increasing FUD being drummed up in their circles about anything to do with AI or significantly altering "what it means to be 'human'". The former is described as a front for evil spirits/demons, and the latter as the penultimate step before apocalyptic events, i.e. a precursor to the worst kinds of evil (in some circles, the 'mark of The Beast'). [For the record, I'm agnostic and feel the author raises some excellent points in any case.] There will be extreme resistance to these technologies from many kinds of religious people who would be willing to die and/or kill to prevent what they see as a takeover by the most malevolent of invisible forces. I may not agree with their reasoning, but some of their concerns could be worth reframing and considering in a different light. Personally, I strongly suspect we will have to fundamentally change as a species to get through at least one of the Great Filters looming before us. However, it's important to remember that the road is fraught with many perils and several possible paths which could lead to unimaginable suffering. It really seems to me that we're going to have to get lucky on multiple dice rolls here in the long run.

api 4y ago
> some of their concerns could be worth reframing and considering in a different light. There are some legitimate concerns about AI that sound almost like something out of the Book of Revelations, but are entirely rational. The biggest one is the idea of AI being used to implement "automated con artistry at scale." Imagine assigning every living human being a virtual AI powered con artist to tail them around and try to convince them of whatever is ordered by the highest bidder. This AI is powered by mass surveillance and big data and is able to know the "mark" intimately and work on them 24/7 both directly and via their social network. Now throw in deep fakes, attention maximizing "compulsion loops" and other adversarial models of human cognition, etc. It's basically the hydrogen bomb of propaganda, a doomsday machine for the mind. It would be like creating Satan, except this one would be entirely amoral and mercenary. At the very least this would be the end of democracy in any form. I could see this ushering in the permanent eternal victory of totalitarianism since I can't imagine the masses being able to organize any resistance when constantly bombarded with propaganda from their demons. The new feudal aristocracy would be the ones running the demons. This is one of the darkest plausible visions of the future I can think of at the moment, far worse than anything related to climate change or similar problems. If we manage to avoid these kinds of scenarios but still drown Miami I'll say we didn't do too badly. Transhumanism comes with some analogous concerns around the potential for enhanced humans to enslave the rest of humanity through superior cognitive capacities and the incredible accumulation of wealth. I can imagine a scenario where a few wealthy people gain access to technologies for life extension and cognitive enhancement and then "run away," effectively becoming a new species and exterminating or enslaving "legacy humans." This isn't mutually exclusive to the AI hellscape I describe above, since that would be an ideal mechanism to enslave the rest. Ultimately I think these technologies are all neutral. The problem is that we are not. I don't fear AI. I fear what humans will do with AI.
- Camillo 4y ago
  We already have mass media.
  - api 4y ago
    Mass media is to what I described what spears are to M-16s. That's a pretty good analogy. In the parent post I am describing propaganda becoming mechanized warfare. One person with an M-16 could take out an entire massive army of people wielding spears (as long as they had plenty of ammo).
    - defined 4y ago
      > One person with an M-16 could take out an entire massive army of people wielding spears I have to disagree. All it would take is for this army to surround the person with the M-16 and wait for a magazine change, then hurl 50 spears at the hapless pincushion. And that's not counting the probability of the rifle overheating, jamming, etc. after long enough. I realize I am taking this probably too literally, but my point stands :)
- pca006132 4y ago
  > I could see this ushering in the permanent eternal victory of totalitarianism since I can't imagine the masses being able to organize any resistance when constantly bombarded with propaganda from their demons. I thought this is already what is happening now! I think technology is neutral, but the entire system is not: it will strengthen the existing power imbalance until there is no hope of balancing it again. For example, the authority can have mass surveillance to identify possible risks and remove them, without the public noticing by controlling the social media. Even if you noticed, you have basically no chance of gathering a large enough crowd to protest due to communication channels being constantly monitored by AI. Even if you somehow get enough people, you will not have a chancw fighting them as their weapons are a lot more advanced, probably unmanned so not constrained by manpower.
- slfnflctd 4y ago
  My greatest hope is that we will somehow find a way to reduce/eliminate selfish and antisocial tendencies as an intrinsic aspect of any advances made in these areas. I think any tech which doesn't make that a top priority is doomed to create more suffering than would exist in its absence. This hope often seems like a pipe dream these days, though.
no-dr-onboard 4y ago
> It really seems to me that we're going to have to get lucky on multiple dice rolls here in the long run. From a materialist/evolutionist view, this will always be the case.
- _aavaa_ 4y ago
  And? Just because a branch of the evolutionary tree continues does not mean that it will be ours. Also, I don't think "always be the case" is a foregone conclusion. Mars' current evolutionary tree isn't looking very good.
  - saghm 4y ago
    > Also, I don't think "always be the case" is a foregone conclusion. Mars' current evolutionary tree isn't looking very good. I think they mean the opposite of what you inferred; they're saying we'll always need to be lucky to keep surviving, not that we won't need to be lucky to keep surviving after a certain point
  - collegeburner 4y ago
    by colonizing mars, we split our evolutionary tree more so wiping out 1 branch isn't catastrophic. yes a species has to keep getting lucky to survive. but more branches means more rolls of the dice.
citilife 4y ago
> Personally, I strongly suspect we will have to fundamentally change as a species to get through at least one of the Great Filters looming before us. However, it's important to remember that the road is fraught with many perils and several possible paths which could lead to unimaginable suffering. It really seems to me that we're going to have to get lucky on multiple dice rolls here in the long run. The opposite could be equally true. The evolution to use technology can make us weak / break us in some manner which eventually destroys us. Simple examples: climate change, genetic alterations, lack of reproduction, etc. AKA by rolling the dice we are running the risk. If you look at it another way, you're correct. But every new theoretically destructive technology is another dice role, eventually we will blow ourselves up (even with long-tail odds of failure). The only way to survive is to slow progress and roll the dice when we mitigate risk and increase our capability of surviving (running gene related experiments more slowly aka - COVID19 therapy) or testing nuclear weapons on mars, etc.
tboyd47 4y ago
- trowaLA 4y ago
- trasz 4y ago
  Like someone who can understand basic facts around us? We are already changing, because our way of interacting with the world has fundamentally changed. “Great Filter” - I’m guessing you might want to google that one. Suffering - let me remind you the consequences of climate catastrophe.
  - deleted 4y ago
    [deleted]
- buttercraft 4y ago
  I think they are referring to https://en.m.wikipedia.org/wiki/Great_Filter https://en.m.wikipedia.org/wiki/Great_Filter
  - 7thaccount 4y ago
    Yeah, great filter is just one of those things that tries to explain why we don't see signs of alien life everywhere we look in the cosmos. Everything looks dead. Why? Maybe life is incredibly hard to take off and even with a bazillion planets, a chance of 0.0000000000001% might mean we really are alone in the galaxy. A different one is that life is incredibly common, but developing sentience is so rare that we're the only ones. These are the better options as we're already past these hurdle. What if it's incredibly common for sentient life to evolve that makes machines and all the things we do, yet the entire galaxy still seems dead? Well that would be very bad for us as it means there is a big "filter" ahead for us. Perhaps there are millions of dead civilizations amongst the cosmos that died from solar disruptions, gama ray bursts, asteroids, climate change, nuclear Armageddon, or a superior alien species bent on the eradication of all other life?
- 4y ago
jl6 4y ago
The last resort of every peace process is an appeal to our common humanity, so I do think there is some risk to tinkering with what it means to be human. Look at the state of conflict today, even with the benefit of us all being one species. We figuratively dehumanize our opponents in order to justify violence (“they’re monsters!”). It could get a lot worse if we find ourselves in conflict with people who can be literally dehumanized, with whom we have lost the most fundamental thread of commonality.
- becquerel 4y ago
  I think we will have to do away with the notion of being human as special and talk about shared values or systems instead. For example, I don't torture animals, even though they're not human - because they still have the capacity to suffer, to feel pain, etc. I would feel the same way about an AI.
  - marcianx 4y ago
    Is that last point not obviated by not building AIs that "suffer"? Shouldn't that be the responsibility of its creator to not create "suffering", whatever that is?
    - jl6 4y ago
      The creator may have such a responsibility, but not the capability. Suffering may be a necessary side effect of something positive.
    - XorNot 4y ago
      It's murkier when you could build an AI with self awareness but no self-preservation. Say an intelligent missile which wakes up with a deep seated drive to effectively explode.
      - mwigdahl 4y ago
        Bomb 20 in the movie _Dark Star_... https://www.youtube.com/watch?v=qjGRySVyTDk https://www.youtube.com/watch?v=qjGRySVyTDk
mythrwy 4y ago
I wonder if the primary biological directives "preservation of my species, my group and myself" plays a role here. Like, it's almost instinctual. It does seem a lot of religious edicts are based around these directives.
spicybright 4y ago
> Personally, I strongly suspect we will have to fundamentally change as a species to get through at least one of the Great Filters looming before us. We have almost no evidence there's a filter ahead of us. It could be we already passed one and don't need to change at all at this point to survive long term. There could even be no filters at all.
- colinmhayes 4y ago
  We don’t really have any evidence for any of the solutions to the Fermi paradox, that’s what makes it a paradox. Personally the filter being ahead of us seems the most likely answer, but you’re right that it could be just about anything. The odds that we are the only civilization to make it to the “computer age” just seems far too unlikely. 1 in a trillion is not where I’m putting my money.
- naasking 4y ago
  On the contrary, we have ample evidence from biology that every species destroys its own habitat if it's actions are not checked by some kind of feedback mechanism, like predation. Humans no longer have such a feedback mechanism, and we're seeing the results in the destruction of our habitat in climate change.
- deleted 4y ago
  [deleted]
afpx 4y ago
Lots of people have the the blanketed "religion is bad" perspective. The problem is that technology has always been used to oppress people. The result of transhumanism will be oppression of all by a few (or worse, just one). And, following that trending out a few hundred years, the long-term result will be a single Human organism with one head. Would you rather that head be a Human or a God?
- glitchc 4y ago
  This is a bit of the pot calling the kettle black. Religion has a long, storied history of oppressing outsiders, minorities and classes within its own fold (caste system).
  - brodouevencode 4y ago
    I don't think OP was attempting to make a point like that either way. They said > The problem is that technology has always been used to oppress people. This goes both ways.
    - glitchc 4y ago
      They said stuff before and after that sentence too. The gist of the parent's argument is that the head of the human race should be a God, not a human. That can only be classified as a religious statement.
      - brodouevencode 4y ago
        It can also be read in a more generalized way as religion is no different than any other human endeavor: prone to abuse for gain (however that may manifest). Also given some of the other responses in this thread contain some thoughts of which are critical of religion (any maybe rightly so). IOW, they basically said what you said.
        alexvoda 4y ago
        Indeed, religion is "social technology". Religion is a memeplex made of memes that gets installed on brains.
throwaway_4ever 4y ago
Many AI researchers think our current trajectory towards AGI will be one of creating our misaligned overlords and the end of humanity as we know it. If that doesn’t change, you bet your ass a significant faction of humanity will go to war against anyone getting close to AGI, where AI researchers will be assassinated like nuclear scientists in Iran.
- anotherman554 4y ago
  I know this is a popular opinion among certain bloggers, but this presumes that a significant faction of humanity takes these blogs you read as seriously as you do.
  - throwaway_4ever 4y ago
    It's not about if/when people take these blogs seriously. It's a race between the creators of AGI and when it becomes mainstream enough for a popular politician to make it the cause de jour and rally the populist base against an existential threat.
    - anotherman554 4y ago
      If it really is an existential threat why didn't a politician rally the population against AI in 1984 after the Terminator and War Games movies came out? I guess because nobody would take them seriously and they'd look ridiculous? You're positing at some point this is going to change but I'm not seeing how. Nuclear weapons are already an existential threat and I don't see anyone rallying against it.
      - fatherzine 4y ago
        There was no GPT3 or DALL-E in 1984, and no way to viscerally convey the (presumed) capabilities of such systems to the average person. In practice, the average person is so disempowered that nothing will change. We will step into the transhuman era with utmost vanity.
RajT88 4y ago
This sort of thing always happens in religious circles at the advent of new technology. See: Credit cards See: RFID chips See: NFC phone payments https://www.pillarcatholic.com/p/what-is-the-mark-of-the-beast-actually https://www.pillarcatholic.com/p/what-is-the-mark-of-the-bea... https://www.cnet.com/culture/is-rfid-the-mark-of-the-beast/ https://www.cnet.com/culture/is-rfid-the-mark-of-the-beast/ https://christianjournal.net/nwo/apples-cashless-world/ https://christianjournal.net/nwo/apples-cashless-world/ It will probably be fine.
deleted 4y ago
[deleted]
gadders 4y ago
I don't think that any of these advantages would necessarily lead to the end of the world, but as a thought experiment picture any of these proposed improvements in the hands of, say, the North Korean government to use on their population. That's how bad these things could get.

api 4y ago

> some of their concerns could be worth reframing and considering in a different light. There are some legitimate concerns about AI that sound almost like something out of the Book of Revelations, but are entirely rational. The biggest one is the idea of AI being used to implement "automated con artistry at scale." Imagine assigning every living human being a virtual AI powered con artist to tail them around and try to convince them of whatever is ordered by the highest bidder. This AI is powered by mass surveillance and big data and is able to know the "mark" intimately and work on them 24/7 both directly and via their social network. Now throw in deep fakes, attention maximizing "compulsion loops" and other adversarial models of human cognition, etc. It's basically the hydrogen bomb of propaganda, a doomsday machine for the mind. It would be like creating Satan, except this one would be entirely amoral and mercenary. At the very least this would be the end of democracy in any form. I could see this ushering in the permanent eternal victory of totalitarianism since I can't imagine the masses being able to organize any resistance when constantly bombarded with propaganda from their demons. The new feudal aristocracy would be the ones running the demons. This is one of the darkest plausible visions of the future I can think of at the moment, far worse than anything related to climate change or similar problems. If we manage to avoid these kinds of scenarios but still drown Miami I'll say we didn't do too badly. Transhumanism comes with some analogous concerns around the potential for enhanced humans to enslave the rest of humanity through superior cognitive capacities and the incredible accumulation of wealth. I can imagine a scenario where a few wealthy people gain access to technologies for life extension and cognitive enhancement and then "run away," effectively becoming a new species and exterminating or enslaving "legacy humans." This isn't mutually exclusive to the AI hellscape I describe above, since that would be an ideal mechanism to enslave the rest. Ultimately I think these technologies are all neutral. The problem is that we are not. I don't fear AI. I fear what humans will do with AI.

Camillo 4y ago
We already have mass media.
- api 4y ago
  Mass media is to what I described what spears are to M-16s. That's a pretty good analogy. In the parent post I am describing propaganda becoming mechanized warfare. One person with an M-16 could take out an entire massive army of people wielding spears (as long as they had plenty of ammo).
  - defined 4y ago
    > One person with an M-16 could take out an entire massive army of people wielding spears I have to disagree. All it would take is for this army to surround the person with the M-16 and wait for a magazine change, then hurl 50 spears at the hapless pincushion. And that's not counting the probability of the rifle overheating, jamming, etc. after long enough. I realize I am taking this probably too literally, but my point stands :)
pca006132 4y ago
> I could see this ushering in the permanent eternal victory of totalitarianism since I can't imagine the masses being able to organize any resistance when constantly bombarded with propaganda from their demons. I thought this is already what is happening now! I think technology is neutral, but the entire system is not: it will strengthen the existing power imbalance until there is no hope of balancing it again. For example, the authority can have mass surveillance to identify possible risks and remove them, without the public noticing by controlling the social media. Even if you noticed, you have basically no chance of gathering a large enough crowd to protest due to communication channels being constantly monitored by AI. Even if you somehow get enough people, you will not have a chancw fighting them as their weapons are a lot more advanced, probably unmanned so not constrained by manpower.
slfnflctd 4y ago
My greatest hope is that we will somehow find a way to reduce/eliminate selfish and antisocial tendencies as an intrinsic aspect of any advances made in these areas. I think any tech which doesn't make that a top priority is doomed to create more suffering than would exist in its absence. This hope often seems like a pipe dream these days, though.

no-dr-onboard 4y ago

> It really seems to me that we're going to have to get lucky on multiple dice rolls here in the long run. From a materialist/evolutionist view, this will always be the case.

_aavaa_ 4y ago
And? Just because a branch of the evolutionary tree continues does not mean that it will be ours. Also, I don't think "always be the case" is a foregone conclusion. Mars' current evolutionary tree isn't looking very good.
- saghm 4y ago
  > Also, I don't think "always be the case" is a foregone conclusion. Mars' current evolutionary tree isn't looking very good. I think they mean the opposite of what you inferred; they're saying we'll always need to be lucky to keep surviving, not that we won't need to be lucky to keep surviving after a certain point
- collegeburner 4y ago
  by colonizing mars, we split our evolutionary tree more so wiping out 1 branch isn't catastrophic. yes a species has to keep getting lucky to survive. but more branches means more rolls of the dice.

citilife 4y ago

> Personally, I strongly suspect we will have to fundamentally change as a species to get through at least one of the Great Filters looming before us. However, it's important to remember that the road is fraught with many perils and several possible paths which could lead to unimaginable suffering. It really seems to me that we're going to have to get lucky on multiple dice rolls here in the long run. The opposite could be equally true. The evolution to use technology can make us weak / break us in some manner which eventually destroys us. Simple examples: climate change, genetic alterations, lack of reproduction, etc. AKA by rolling the dice we are running the risk. If you look at it another way, you're correct. But every new theoretically destructive technology is another dice role, eventually we will blow ourselves up (even with long-tail odds of failure). The only way to survive is to slow progress and roll the dice when we mitigate risk and increase our capability of surviving (running gene related experiments more slowly aka - COVID19 therapy) or testing nuclear weapons on mars, etc.

tboyd47 4y ago

trowaLA 4y ago
trasz 4y ago
Like someone who can understand basic facts around us? We are already changing, because our way of interacting with the world has fundamentally changed. “Great Filter” - I’m guessing you might want to google that one. Suffering - let me remind you the consequences of climate catastrophe.
- deleted 4y ago
  [deleted]
buttercraft 4y ago
I think they are referring to https://en.m.wikipedia.org/wiki/Great_Filter https://en.m.wikipedia.org/wiki/Great_Filter
- 7thaccount 4y ago
  Yeah, great filter is just one of those things that tries to explain why we don't see signs of alien life everywhere we look in the cosmos. Everything looks dead. Why? Maybe life is incredibly hard to take off and even with a bazillion planets, a chance of 0.0000000000001% might mean we really are alone in the galaxy. A different one is that life is incredibly common, but developing sentience is so rare that we're the only ones. These are the better options as we're already past these hurdle. What if it's incredibly common for sentient life to evolve that makes machines and all the things we do, yet the entire galaxy still seems dead? Well that would be very bad for us as it means there is a big "filter" ahead for us. Perhaps there are millions of dead civilizations amongst the cosmos that died from solar disruptions, gama ray bursts, asteroids, climate change, nuclear Armageddon, or a superior alien species bent on the eradication of all other life?
4y ago

jl6 4y ago

The last resort of every peace process is an appeal to our common humanity, so I do think there is some risk to tinkering with what it means to be human. Look at the state of conflict today, even with the benefit of us all being one species. We figuratively dehumanize our opponents in order to justify violence (“they’re monsters!”). It could get a lot worse if we find ourselves in conflict with people who can be literally dehumanized, with whom we have lost the most fundamental thread of commonality.

becquerel 4y ago
I think we will have to do away with the notion of being human as special and talk about shared values or systems instead. For example, I don't torture animals, even though they're not human - because they still have the capacity to suffer, to feel pain, etc. I would feel the same way about an AI.
- marcianx 4y ago
  Is that last point not obviated by not building AIs that "suffer"? Shouldn't that be the responsibility of its creator to not create "suffering", whatever that is?
  - jl6 4y ago
    The creator may have such a responsibility, but not the capability. Suffering may be a necessary side effect of something positive.
  - XorNot 4y ago
    It's murkier when you could build an AI with self awareness but no self-preservation. Say an intelligent missile which wakes up with a deep seated drive to effectively explode.
    - mwigdahl 4y ago
      Bomb 20 in the movie _Dark Star_... https://www.youtube.com/watch?v=qjGRySVyTDk https://www.youtube.com/watch?v=qjGRySVyTDk

mythrwy 4y ago

I wonder if the primary biological directives "preservation of my species, my group and myself" plays a role here. Like, it's almost instinctual. It does seem a lot of religious edicts are based around these directives.

spicybright 4y ago

> Personally, I strongly suspect we will have to fundamentally change as a species to get through at least one of the Great Filters looming before us. We have almost no evidence there's a filter ahead of us. It could be we already passed one and don't need to change at all at this point to survive long term. There could even be no filters at all.

colinmhayes 4y ago
We don’t really have any evidence for any of the solutions to the Fermi paradox, that’s what makes it a paradox. Personally the filter being ahead of us seems the most likely answer, but you’re right that it could be just about anything. The odds that we are the only civilization to make it to the “computer age” just seems far too unlikely. 1 in a trillion is not where I’m putting my money.
naasking 4y ago
On the contrary, we have ample evidence from biology that every species destroys its own habitat if it's actions are not checked by some kind of feedback mechanism, like predation. Humans no longer have such a feedback mechanism, and we're seeing the results in the destruction of our habitat in climate change.
deleted 4y ago
[deleted]

afpx 4y ago

Lots of people have the the blanketed "religion is bad" perspective. The problem is that technology has always been used to oppress people. The result of transhumanism will be oppression of all by a few (or worse, just one). And, following that trending out a few hundred years, the long-term result will be a single Human organism with one head. Would you rather that head be a Human or a God?

glitchc 4y ago
This is a bit of the pot calling the kettle black. Religion has a long, storied history of oppressing outsiders, minorities and classes within its own fold (caste system).
- brodouevencode 4y ago
  I don't think OP was attempting to make a point like that either way. They said > The problem is that technology has always been used to oppress people. This goes both ways.
  - glitchc 4y ago
    They said stuff before and after that sentence too. The gist of the parent's argument is that the head of the human race should be a God, not a human. That can only be classified as a religious statement.
    - brodouevencode 4y ago
      It can also be read in a more generalized way as religion is no different than any other human endeavor: prone to abuse for gain (however that may manifest). Also given some of the other responses in this thread contain some thoughts of which are critical of religion (any maybe rightly so). IOW, they basically said what you said.
      - alexvoda 4y ago
        Indeed, religion is "social technology". Religion is a memeplex made of memes that gets installed on brains.

throwaway_4ever 4y ago

Many AI researchers think our current trajectory towards AGI will be one of creating our misaligned overlords and the end of humanity as we know it. If that doesn’t change, you bet your ass a significant faction of humanity will go to war against anyone getting close to AGI, where AI researchers will be assassinated like nuclear scientists in Iran.

anotherman554 4y ago
I know this is a popular opinion among certain bloggers, but this presumes that a significant faction of humanity takes these blogs you read as seriously as you do.
- throwaway_4ever 4y ago
  It's not about if/when people take these blogs seriously. It's a race between the creators of AGI and when it becomes mainstream enough for a popular politician to make it the cause de jour and rally the populist base against an existential threat.
  - anotherman554 4y ago
    If it really is an existential threat why didn't a politician rally the population against AI in 1984 after the Terminator and War Games movies came out? I guess because nobody would take them seriously and they'd look ridiculous? You're positing at some point this is going to change but I'm not seeing how. Nuclear weapons are already an existential threat and I don't see anyone rallying against it.
    - fatherzine 4y ago
      There was no GPT3 or DALL-E in 1984, and no way to viscerally convey the (presumed) capabilities of such systems to the average person. In practice, the average person is so disempowered that nothing will change. We will step into the transhuman era with utmost vanity.

RajT88 4y ago

This sort of thing always happens in religious circles at the advent of new technology. See: Credit cards See: RFID chips See: NFC phone payments https://www.pillarcatholic.com/p/what-is-the-mark-of-the-beast-actually https://www.pillarcatholic.com/p/what-is-the-mark-of-the-bea... https://www.cnet.com/culture/is-rfid-the-mark-of-the-beast/ https://www.cnet.com/culture/is-rfid-the-mark-of-the-beast/ https://christianjournal.net/nwo/apples-cashless-world/ https://christianjournal.net/nwo/apples-cashless-world/ It will probably be fine.

deleted 4y ago

[deleted]

gadders 4y ago

I don't think that any of these advantages would necessarily lead to the end of the world, but as a thought experiment picture any of these proposed improvements in the hands of, say, the North Korean government to use on their population. That's how bad these things could get.