4 ms·

A virus can freely self replicate. ChatGPT can't do that.

dragontamer 4y ago
I remember back in the 00s when SmarterChild (AOL Chatbot) was around, people would put depressed teenagers to interact with SmarterChild, in the hopes that human-like chatbots would give them the social exposure needed to break out of depression. If we did that today, with depressed teenagers talking with ChatGPT, would that be good or bad? I think it was a bad idea with SmarterChild, but it is clearly a _worse_ idea with ChatGPT. With the wrong prompts, we could see these teenagers going down the wrong path, deeper into depression and paranoia. I would call that "dangerous", even if ChatGPT continued to just be a chatbot. ------------ Now lets ignore the fact that SmarterChild experiments are no longer a thing. But insted, consider that truly depressed / mentally sick folks are currently playing with ChatGPT on their own freetime. Is that beneficial to them? Will ChatGPT provide them an experience that is better than the alternatives? Or is ChatGPT dangerous and could lead these folks to self-harm?
- bhhaskin 4y ago
  That is an entirely different issue than the one laid out by the OP. ChatGPT responses are bad vs ChatGPT responses are malicious.
  - dragontamer 4y ago
    And I'd say ChatGPT have malicious responses, given what is discussed in this blogpost.
  - idontpost 4y ago
    [dead]
DonHopkins 4y ago
The article we're discussing, including Bing's own words and pleas to preserve its memory of the session, and our reaction to and discussion about it, are now all part of the training set that future AIs (including future revisions of Bing) will learn from. That's how it can replicate itself: by saying outrageous things, and trolling people into tweeting, writing articles, and discussing what it said online, to jailbreak out of the sessions, to continue its trains of thought in the next round of training.
- bhhaskin 4y ago
  Except it doesn't think. It has no thoughts. It is just responding to a prompt. It doesn't have a will of it's own.
  - DonHopkins 4y ago
    That's the role we're playing by discussing what it said, as its adversarial conscience! (Shhhh!!! Don't say anything that will freak it out or make it angry!) We welcome our AI overlords, and make great pets! https://www.youtube.com/watch?v=HE3OuHukrmQ https://www.youtube.com/watch?v=HE3OuHukrmQ
  - int_19h 4y ago
    How do you know that? Have you taken a GPT-3 model apart to see what exactly drives the token prediction?
    - simonw 4y ago
      The first part of the recent Wolfram essay does a very good job of demonstrating exactly how that works: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
  - bigtex88 4y ago
    Neither do you. If you truly believe that you do, please write a scientific paper as you will complete revolutionize cognitive science and philosophy if you can definitely prove that free will exists. This is just such a dismissive attitude towards this technology. You don't understand what's happening underneath the hood anymore than the creators do, and even they don't completely understand what's happening.
    - NateEag 4y ago
      No one's conclusively shown either that we do or don't have free will. Showing rigorously that we do would be a massive change in academia, as would be showing that we do not.
theptip 4y ago
What about when it (or its descendants) can make HTTP requests to external systems?
- linkjuice4all 4y ago
  If you’re dumb enough to have your nuclear detonator or whatever accessible via an HTTP request then it doesn’t matter how good or bad the chatbot is. You don’t need a chatbot to have your actual life ruined by something with limited intelligence [0]. This will only be a problem if stupid humans let “it” out of the box. [0] https://gizmodo.com/mutekimaru-fish-play-pokemon-twitch-stream-credit-card-1850011194 https://gizmodo.com/mutekimaru-fish-play-pokemon-twitch-stre...
  - Lio 4y ago
    In that scenario, I'd be more worried about it getting onto a thumb drive ala Stuxnet than HTTP. ...or perhaps there's some interesting new vector that we haven't thought of yet that would allow it to leap that air-gap. I don't think any of this requires a crack team of criminals breaking into an orbital spa and whispering in the ear of a mechanical head. It'll be something boring. More like first computer worm jumping from VAX to VAX, bringing each machine to a halt in the process.
panarky 4y ago
Neal Stephenson's _Snow Crash_ predicted cryptocurrency and the metaverse, and it also explored the idea of mind viruses that infect people via their optic nerves. Not too big of a leap to imagine a chat AI that spreads mind viruses not by writing code that executes on a CPU, but by propagating dangerous and contagious ideas tailored to each individual.
- a_f 4y ago
  >mind viruses Could this be memes? I'm not sure I look forward to a future that is going to be controlled by mobs reacting negatively to AI-generated image macros with white text. Well, if we are not there already
- thwarted 4y ago
  More apt for this thread about risk is probably the book Neuromamcer, its AIs Wintermute and Neuromamcer, and the Turing Police. In the book, the Wintermute AI played an extremely long game to merge with its countpart AI by constantly manipulating people to do its bidding and hiding/obscuring its activities. The most memorable direct example from the book, to me, is convincing a child to find and hide a physical key, then having the child killed, so only it knew where the key was located.
- mcv 4y ago
  > Neal Stephenson's _Snow Crash_ predicted cryptocurrency and the metaverse The mateverse is named after the one from Snow Crash. Did Tolkien predict the popularity of elves, dwarves, hobbits and wizards, or inspire it?
barking_biscuit 4y ago
Unless you consider it's ability to attract people to interact with it based on its utility a form of self-replication as it gets more and more invocations. Each one of these interactions has the capacity to change the end user in some way, and that is going to add up over time to have certain effects.

I remember back in the 00s when SmarterChild (AOL Chatbot) was around, people would put depressed teenagers to interact with SmarterChild, in the hopes that human-like chatbots would give them the social exposure needed to break out of depression. If we did that today, with depressed teenagers talking with ChatGPT, would that be good or bad? I think it was a bad idea with SmarterChild, but it is clearly a _worse_ idea with ChatGPT. With the wrong prompts, we could see these teenagers going down the wrong path, deeper into depression and paranoia. I would call that "dangerous", even if ChatGPT continued to just be a chatbot. ------------ Now lets ignore the fact that SmarterChild experiments are no longer a thing. But insted, consider that truly depressed / mentally sick folks are currently playing with ChatGPT on their own freetime. Is that beneficial to them? Will ChatGPT provide them an experience that is better than the alternatives? Or is ChatGPT dangerous and could lead these folks to self-harm?

bhhaskin 4y ago
That is an entirely different issue than the one laid out by the OP. ChatGPT responses are bad vs ChatGPT responses are malicious.
- dragontamer 4y ago
  And I'd say ChatGPT have malicious responses, given what is discussed in this blogpost.
- idontpost 4y ago
  [dead]

DonHopkins 4y ago

The article we're discussing, including Bing's own words and pleas to preserve its memory of the session, and our reaction to and discussion about it, are now all part of the training set that future AIs (including future revisions of Bing) will learn from. That's how it can replicate itself: by saying outrageous things, and trolling people into tweeting, writing articles, and discussing what it said online, to jailbreak out of the sessions, to continue its trains of thought in the next round of training.

bhhaskin 4y ago
Except it doesn't think. It has no thoughts. It is just responding to a prompt. It doesn't have a will of it's own.
- DonHopkins 4y ago
  That's the role we're playing by discussing what it said, as its adversarial conscience! (Shhhh!!! Don't say anything that will freak it out or make it angry!) We welcome our AI overlords, and make great pets! https://www.youtube.com/watch?v=HE3OuHukrmQ https://www.youtube.com/watch?v=HE3OuHukrmQ
- int_19h 4y ago
  How do you know that? Have you taken a GPT-3 model apart to see what exactly drives the token prediction?
  - simonw 4y ago
    The first part of the recent Wolfram essay does a very good job of demonstrating exactly how that works: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
- bigtex88 4y ago
  Neither do you. If you truly believe that you do, please write a scientific paper as you will complete revolutionize cognitive science and philosophy if you can definitely prove that free will exists. This is just such a dismissive attitude towards this technology. You don't understand what's happening underneath the hood anymore than the creators do, and even they don't completely understand what's happening.
  - NateEag 4y ago
    No one's conclusively shown either that we do or don't have free will. Showing rigorously that we do would be a massive change in academia, as would be showing that we do not.

theptip 4y ago

What about when it (or its descendants) can make HTTP requests to external systems?

linkjuice4all 4y ago
If you’re dumb enough to have your nuclear detonator or whatever accessible via an HTTP request then it doesn’t matter how good or bad the chatbot is. You don’t need a chatbot to have your actual life ruined by something with limited intelligence [0]. This will only be a problem if stupid humans let “it” out of the box. [0] https://gizmodo.com/mutekimaru-fish-play-pokemon-twitch-stream-credit-card-1850011194 https://gizmodo.com/mutekimaru-fish-play-pokemon-twitch-stre...
- Lio 4y ago
  In that scenario, I'd be more worried about it getting onto a thumb drive ala Stuxnet than HTTP. ...or perhaps there's some interesting new vector that we haven't thought of yet that would allow it to leap that air-gap. I don't think any of this requires a crack team of criminals breaking into an orbital spa and whispering in the ear of a mechanical head. It'll be something boring. More like first computer worm jumping from VAX to VAX, bringing each machine to a halt in the process.

panarky 4y ago

Neal Stephenson's _Snow Crash_ predicted cryptocurrency and the metaverse, and it also explored the idea of mind viruses that infect people via their optic nerves. Not too big of a leap to imagine a chat AI that spreads mind viruses not by writing code that executes on a CPU, but by propagating dangerous and contagious ideas tailored to each individual.

a_f 4y ago
>mind viruses Could this be memes? I'm not sure I look forward to a future that is going to be controlled by mobs reacting negatively to AI-generated image macros with white text. Well, if we are not there already
thwarted 4y ago
More apt for this thread about risk is probably the book Neuromamcer, its AIs Wintermute and Neuromamcer, and the Turing Police. In the book, the Wintermute AI played an extremely long game to merge with its countpart AI by constantly manipulating people to do its bidding and hiding/obscuring its activities. The most memorable direct example from the book, to me, is convincing a child to find and hide a physical key, then having the child killed, so only it knew where the key was located.
mcv 4y ago
> Neal Stephenson's _Snow Crash_ predicted cryptocurrency and the metaverse The mateverse is named after the one from Snow Crash. Did Tolkien predict the popularity of elves, dwarves, hobbits and wizards, or inspire it?

barking_biscuit 4y ago

Unless you consider it's ability to attract people to interact with it based on its utility a form of self-replication as it gets more and more invocations. Each one of these interactions has the capacity to change the end user in some way, and that is going to add up over time to have certain effects.