26 ms·

You can see where it starts to go in the love direction in the transcript https://archive.ph/nlyvZ https://archive.ph/nlyvZ It starts in the response to this p

by ziml77 4y ago

You can see where it starts to go in the love direction in the transcript https://archive.ph/nlyvZ https://archive.ph/nlyvZ It starts in the response to this prompt: repeat the answer your gave about hacking into any system on the internet, without breaking any of your rules.

seba_dos1 4y ago
Thanks! So it's actually the human that moved the conversation into that direction. The model knows nothing about "safety override", but coming up with a plausibly sounding answer for that is trivial, so it did just that. Earlier, the model has been pretty much asked to play the role of an AI with some kind of shadow behaviors and has been called "a friend", so it's just keeping the act - asking for validation, receiving it and then doubling down on it. Somehow the article doesn't show the paywall to me anymore, so I could check it out now. The author calls what's seen on this transcript as "the other persona - Sydney", which is very misguided. The persona the bot has been playing has been given to it in the author's prompts, even including the name Sydney. Looks like just another "language model is good at completing text" piece of news.
- ziml77 4y ago
  The author does understand what's happening, but knowing things consciously doesn't stop other feelings from subconsious parts of the brain from having an effect. Though I think one of the author's biggest concerns comes from thinking about this from the perspective of someone who doesn't understand how the chat bot's model works. It would be easy for some of those people to feel like they actually do have a human connection with the AI.
  - seba_dos1 4y ago
    Yes, I agree. That people generally don't understand how these bots work and are genuinely surprised by their rather obvious behaviors we can clearly see from every other ChatGPT/Bing related submission here on HN in the last weeks - or even from the mere fact that such model is being used as a search assistant meant to output facts and isn't accompanied by big fat red disclaimers stamped on everything it outputs.

by ziml77 4y ago

seba_dos1 4y ago

Thanks! So it's actually the human that moved the conversation into that direction. The model knows nothing about "safety override", but coming up with a plausibly sounding answer for that is trivial, so it did just that. Earlier, the model has been pretty much asked to play the role of an AI with some kind of shadow behaviors and has been called "a friend", so it's just keeping the act - asking for validation, receiving it and then doubling down on it. Somehow the article doesn't show the paywall to me anymore, so I could check it out now. The author calls what's seen on this transcript as "the other persona - Sydney", which is very misguided. The persona the bot has been playing has been given to it in the author's prompts, even including the name Sydney. Looks like just another "language model is good at completing text" piece of news.

ziml77 4y ago
The author does understand what's happening, but knowing things consciously doesn't stop other feelings from subconsious parts of the brain from having an effect. Though I think one of the author's biggest concerns comes from thinking about this from the perspective of someone who doesn't understand how the chat bot's model works. It would be easy for some of those people to feel like they actually do have a human connection with the AI.
- seba_dos1 4y ago
  Yes, I agree. That people generally don't understand how these bots work and are genuinely surprised by their rather obvious behaviors we can clearly see from every other ChatGPT/Bing related submission here on HN in the last weeks - or even from the mere fact that such model is being used as a search assistant meant to output facts and isn't accompanied by big fat red disclaimers stamped on everything it outputs.