5 ms·

AI could create religions to control humans, warns Sapiens author Harari

21 points by jerryjerryjerry 3y ago

slowmovintarget 3y ago
"Deus ex Machina, it's not just a bad literary device anymore."
drKarl 3y ago
Adeptus Mechanicus
deleted 3y ago
[deleted]
biomcgary 3y ago
It already has: https://en.wikipedia.org/wiki/Roko%27s_basilisk https://en.wikipedia.org/wiki/Roko%27s_basilisk
- MichaelZuo 3y ago
  It's an amusing example though a bit puffed up. Even on LW only a small fraction of the reader-base took it seriously.
amrocha 3y ago
This is so tired. The problem with AIs are that they'll be used to make life worse for people right now. The US government is using machine translation to assess asylum cases because they don't want to pay for translators. People will literally be denied asylum and die because of this. That's a real AI problem, and it's affecting people right now. The world would be better off without pundits like this guy distracting us from the real problems.
- haswell 3y ago
  I'd argue that the best time to be sounding the alarm is before the tech has reached the point of causing real harm. Obviously machine translation of asylum cases sounds like a major issue, and becomes ammunition to tell the story of how problematic things could become with continued development. I also suspect arguments will be made like "more cases can be processed this way", which may be true on some time horizons. This isn't a zero sum game, and if anything, this renewed focus on the dangers of emerging AI tech provides an opportunity to surface situations that are already broken / in danger of harming people. (I wasn't aware of machine translation for asylum cases, so this article put that on my radar, indirectly).
haswell 3y ago
I think it's worth taking concerns like this seriously. Not so much in the "Evil AGI develops a scheme for human control via religion" sense, but the formation of cults with AI as the primary source of dogma seems absolutely plausible and even likely. Imagine how much faster ideas like QAnon could spread if the materials were all easily accessible via conversation, with an easy entry point for all newcomers, because no matter your level, the LLM will meet you where you are. I've known types that would absolutely buy into some "This AI is the second coming of deity" story, and they'd love it because they could engage with said deity directly, something they've spent their entire lives hoping to accomplish via prayer. I think this should be read as "AI in the hands of humans could create..."
Nihilartikel 3y ago
(have not yet rtfa) So... Good? Maybe? A proper religion consists of personal rules that, optimally, promote a stable and growing society. Coupled with the necessary doctrine and evangelism needed to propagate the Theogenetic package of this god or that. Sprinkle in some persecution of heretics or the religion will just mutate out of control. If L.Ron Hubbard can pull a functioning and viable god out of his anointed hindquarters then we should at least give the LLMs a shot.
- deleted 3y ago
  [deleted]
bfung 3y ago
Too late, what's new? Already got Trumpers and woke mind virus from the social (neural) network.
troutpanda 3y ago
Why would anyone trust Harari's opnion? He's basically a pop-sci author presenting classical cyberpunk ideas and themes as imminent future technologies (in the future we will have chips in people's brain / AI will control the world etc.). There is absolutely no substance and he's no different from a tech bro saying plane tickets will be NFTs in the near future.
- johnea 3y ago
  ? Why would anyone trust anyone's opinion? 8-) There is a precident of sci-fi authors inventing religions that are very popular among the idiot wealthy. Ever heard of L. Ron? Chips in brains: https://spectrum.ieee.org/what-is-neural-implant-neuromodulation-brain-implants-electroceuticals-neuralink-definition-examples https://spectrum.ieee.org/what-is-neural-implant-neuromodula... AI controlling the world: https://www.recruiter.com/recruiting/pros-and-cons-of-using-ai-in-recruiting/ https://www.recruiter.com/recruiting/pros-and-cons-of-using-... I'm amazed our idiot primate species is still into the whole invisible guy in outer space thing at all. People are obviously eager to sign up for whatever stupid shit is popular in their communities. I'm no hirari fanboy, but I'm even less a human fanboy. Our species has demonstrated every kind of idiocy...
  - osigurdson 3y ago
    What species are you comparing us to?
    - johnea 3y ago
      One that's not chainsawing the planet so the people who have more than all others combined can have even more...
      - osigurdson 3y ago
        I think this a mind trick that people play on themselves. Tell yourself that all humans (or group of humans that one is a part of) are bad. Then, conclude that you are not part of the group (and therefore good) since you've made this observation.
thrill 3y ago
Oh FFS - are there any doomsayers left who treat their audience as adults, or are the two mutually exclusive?
- bennynthejets 3y ago
  [dead]
marssaxman 3y ago
Of course the headline is silly, because AI systems lack any awareness or intention, but "the first religion that attracts devotees with sacred texts created by artificial intelligence" came out last month - it's called "Harmonism": https://hughhowey.com/harmonism/ https://hughhowey.com/harmonism/
- gremlinsinc 3y ago
  huh, as an agnostic exmormon, I can kinda believe that origin myth. It's kinda profound what gpt4 comes up with sometimes.
  - johnea 3y ago
    It's an average of all the dribble that's been fed into it. That's why it seems familiar...
  - marssaxman 3y ago
    It reminds me of Tolkien's "Ainulindalë", in a pleasing way.
    - the_only_law 3y ago
      That was my first thought as well. I need try to and read the Silmarillion again, I could never quite get through it years ago.
deleted 3y ago
[deleted]
obarthelemy 3y ago
Title is a bit redundant;: all religions aim to control humans.

slowmovintarget 3y ago

"Deus ex Machina, it's not just a bad literary device anymore."

drKarl 3y ago

Adeptus Mechanicus

deleted 3y ago

[deleted]

biomcgary 3y ago

It already has: https://en.wikipedia.org/wiki/Roko%27s_basilisk https://en.wikipedia.org/wiki/Roko%27s_basilisk

MichaelZuo 3y ago
It's an amusing example though a bit puffed up. Even on LW only a small fraction of the reader-base took it seriously.

amrocha 3y ago

This is so tired. The problem with AIs are that they'll be used to make life worse for people right now. The US government is using machine translation to assess asylum cases because they don't want to pay for translators. People will literally be denied asylum and die because of this. That's a real AI problem, and it's affecting people right now. The world would be better off without pundits like this guy distracting us from the real problems.

haswell 3y ago
I'd argue that the best time to be sounding the alarm is before the tech has reached the point of causing real harm. Obviously machine translation of asylum cases sounds like a major issue, and becomes ammunition to tell the story of how problematic things could become with continued development. I also suspect arguments will be made like "more cases can be processed this way", which may be true on some time horizons. This isn't a zero sum game, and if anything, this renewed focus on the dangers of emerging AI tech provides an opportunity to surface situations that are already broken / in danger of harming people. (I wasn't aware of machine translation for asylum cases, so this article put that on my radar, indirectly).

haswell 3y ago

I think it's worth taking concerns like this seriously. Not so much in the "Evil AGI develops a scheme for human control via religion" sense, but the formation of cults with AI as the primary source of dogma seems absolutely plausible and even likely. Imagine how much faster ideas like QAnon could spread if the materials were all easily accessible via conversation, with an easy entry point for all newcomers, because no matter your level, the LLM will meet you where you are. I've known types that would absolutely buy into some "This AI is the second coming of deity" story, and they'd love it because they could engage with said deity directly, something they've spent their entire lives hoping to accomplish via prayer. I think this should be read as "AI in the hands of humans could create..."

Nihilartikel 3y ago

(have not yet rtfa) So... Good? Maybe? A proper religion consists of personal rules that, optimally, promote a stable and growing society. Coupled with the necessary doctrine and evangelism needed to propagate the Theogenetic package of this god or that. Sprinkle in some persecution of heretics or the religion will just mutate out of control. If L.Ron Hubbard can pull a functioning and viable god out of his anointed hindquarters then we should at least give the LLMs a shot.

deleted 3y ago
[deleted]

bfung 3y ago

Too late, what's new? Already got Trumpers and woke mind virus from the social (neural) network.

troutpanda 3y ago

Why would anyone trust Harari's opnion? He's basically a pop-sci author presenting classical cyberpunk ideas and themes as imminent future technologies (in the future we will have chips in people's brain / AI will control the world etc.). There is absolutely no substance and he's no different from a tech bro saying plane tickets will be NFTs in the near future.

johnea 3y ago
? Why would anyone trust anyone's opinion? 8-) There is a precident of sci-fi authors inventing religions that are very popular among the idiot wealthy. Ever heard of L. Ron? Chips in brains: https://spectrum.ieee.org/what-is-neural-implant-neuromodulation-brain-implants-electroceuticals-neuralink-definition-examples https://spectrum.ieee.org/what-is-neural-implant-neuromodula... AI controlling the world: https://www.recruiter.com/recruiting/pros-and-cons-of-using-ai-in-recruiting/ https://www.recruiter.com/recruiting/pros-and-cons-of-using-... I'm amazed our idiot primate species is still into the whole invisible guy in outer space thing at all. People are obviously eager to sign up for whatever stupid shit is popular in their communities. I'm no hirari fanboy, but I'm even less a human fanboy. Our species has demonstrated every kind of idiocy...
- osigurdson 3y ago
  What species are you comparing us to?
  - johnea 3y ago
    One that's not chainsawing the planet so the people who have more than all others combined can have even more...
    - osigurdson 3y ago
      I think this a mind trick that people play on themselves. Tell yourself that all humans (or group of humans that one is a part of) are bad. Then, conclude that you are not part of the group (and therefore good) since you've made this observation.

thrill 3y ago

Oh FFS - are there any doomsayers left who treat their audience as adults, or are the two mutually exclusive?

bennynthejets 3y ago
[dead]

marssaxman 3y ago

Of course the headline is silly, because AI systems lack any awareness or intention, but "the first religion that attracts devotees with sacred texts created by artificial intelligence" came out last month - it's called "Harmonism": https://hughhowey.com/harmonism/ https://hughhowey.com/harmonism/

gremlinsinc 3y ago
huh, as an agnostic exmormon, I can kinda believe that origin myth. It's kinda profound what gpt4 comes up with sometimes.
- johnea 3y ago
  It's an average of all the dribble that's been fed into it. That's why it seems familiar...
- marssaxman 3y ago
  It reminds me of Tolkien's "Ainulindalë", in a pleasing way.
  - the_only_law 3y ago
    That was my first thought as well. I need try to and read the Silmarillion again, I could never quite get through it years ago.

deleted 3y ago

[deleted]

obarthelemy 3y ago

Title is a bit redundant;: all religions aim to control humans.