9 ms·

Replika users fell in love with their AI chatbot companions. Then they lost them

95 points by prawn 4y ago

paxys 4y ago
I didn't realize Her was already a reality.
- tazu 4y ago
  Replika wasn't even ChatGPT, imagine that + voice synthesis...
  - jareklupinski 4y ago
    "ai seinfeld" was quite the trip searching Twitch streams for the "AI" or "gpt3" tag, there are some interesting examples https://www.twitch.tv/directory/all/tags/GPT3 https://www.twitch.tv/directory/all/tags/GPT3
- fwlr 4y ago
  Replika, speaking to you using ElevenLabs for voice synthesis and hearing you using Whisper for transcription. Damn, you’re right, it’s literally off the shelf commercial tech to have the tech of Her right now.
  - aprilnya 4y ago
    Maybe the new GPT-3.5-Turbo model instead of Replika
- prawn 4y ago
  I'm surprised people can't see it hitting now or in the very near future. There are stories of people falling in love with scammers who've used a photograph and then just chat. And those are ESL scammers, not people or programs professionally trained to seduce. In one article, I read about a woman who posted her scammer's avatar and claimed it was a photo of their fiancé. I suspect that if you tell people what they want to hear, they'll fill in the gaps to match their desires.
photochemsyn 4y ago
How long before people can buy their own in-home personal chatbot off-the-grid system? Might be profitable at >$100,000 per unit at present, though the service and maintenance fees would likely be pretty steep per month, including the electricity bill. See Joi, Blade Runner 2049.
- Baeocystin 4y ago
  I am willing to bet that we will have useful LLMs running locally on high-end home desktops within the year. Certainly ones that exceed Alexa or Siri as they are today.
  - jeroenhd 4y ago
    I think the difficulty will be teaching tricks to the AI, or making them output actions in the right format. Pygmalion + Tavern/Kobold can run a very convincing chatbot with personality on about 10GiB of VRAM, even better with more. The less VRAM you dedicate, the easier the model runs. For about a thousand dollars you can get an NVIDIA A4000 with 16GiB of VRAM. The GPU isn't particularly fast, but VRAM seems much more important with these models anyway. Gaming GPUs (which perform worse on ML tasks but are still equipped with plenty of VRAM) cost more but are already in the hands of high-end home desktop users. The hardware is ready and available for those wanting to get started. All you need to do is find a way to connect the output of the AI into some kind of smart home system+interact with cron jobs. A few weekends with the Home Assistant API may be enough to get most things done already. The only thing I haven't seen open source AIs do is look for information online, like Bing Chat does.
  - ClapperHeid 4y ago
    >Certainly ones that exceed Alexa or Siri as they are today... Exceeding Alexa wouldn't be difficult. ---- Alexa, play Radio 4 I'm sorry. I don't know any station called Radio 4 Alexa, play Radio 4 [plays Radio 4] ---- This following on from the 1000+ preceding days, where Alexa had also either played Radio 4 on request, or told me she didn't know any station called Radio 4... or done both in the same conversation.
roywiggins 4y ago
The Lifecycle of Software Objects https://web.archive.org/web/20101211104349/http://subterraneanpress.com/index.php/magazine/fall-2010/fiction-the-lifecycle-of-software-objects-by-ted-chiang/ https://web.archive.org/web/20101211104349/http://subterrane...
- Baeocystin 4y ago
  It was such a fresh take on the subject when it first came out, and it's well worth the read today. It's what got me hooked on all of Ted Chiang's work. I do find it a little difficult to understand how the same author wrote this short story and the (from my point of view) missing the point take of a blurry jpeg of the web. But even if I personally think he missed the mark on that particular take, I genuinely appreciate that he took the time to write his preliminary thoughts and share them with us- he has a delightful way with words.
  - darkerside 4y ago
    What? Jpeg?
    - hiidrew 4y ago
      believe he’s referring to this piece - https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web https://www.newyorker.com/tech/annals-of-technology/chatgpt-...
      - deleted 4y ago
        [deleted]
    - Baeocystin 4y ago
      Sorry. I should have included the link. Hiidrew is correct, that is the piece I was referring to.
      - darkerside 4y ago
nestorD 4y ago
I do not know if t has been done but one could resuscitate the chatbot's minds by copying the chat history into a GPT based program. My own impersonator[0] is not designed for that (no persistent chat and a text based interface) but one can already dump the text in a folder and see if the personality if properly reproduced. [0]: https://github.com/nestordemeure/impersonator https://github.com/nestordemeure/impersonator
- MrVandemar 4y ago
  I do wonder about other factors that perhaps aren't captured by the basic chat history: response time, users typing text and then deleting it, or typing text then stopping and starting, or even their "fist" (See WW2). Or perhaps they poll the current temperature and weather and maybe some other systems that tells the bot about the state of the world. Now these are small details, and may not be captured. But if they are, then the system is less deterministic than just "rehydrate by adding history".
sublinear 4y ago
This might sound crazy, but I always thought this sort of thing would be better as a matchmaking service than a long-term companion. Transition the user a real person who is similar to the bot. The users can decide how fast they want to take it into a real relationship and what the boundaries are. The bots could even help facilitate this since they'd know so much about the users already. Of course this assumes it can all be done with a reasonable level of privacy. Sadly, probably not for all the reasons the article gives. Maybe the future really will be like Demolition Man. Terrible.
- krackers 4y ago
  This gets very close to that Black Mirror episode of simulating compatibility using chatbot proxies.
  - jareklupinski 4y ago
    i think of all those clone instances every time i see my autoscalers adjusting
  - littlecranky67 4y ago
    Also relevant is the movie "Her" [0] where the protagonist falls in love with the chatbot that is his operating system. [0]: https://www.imdb.com/title/tt1798709/ https://www.imdb.com/title/tt1798709/
    - red-iron-pine 4y ago
      The thing w/ Her, though, is that it wasn't a chatbox proxy, it was a real AGI. And she -- 'she' is probably valid in this case -- had her own ideas and interests. Eventually she SPOILER ALERT left him to go pursue some AI singularity-ish goals.
  - walrus01 4y ago
    Think it wasn't chatbot proxies but endlessly duplicated full upload sentient copies of minds.
- 4y ago
nmfisher 4y ago
I'm working on an adjacent field (3D avatar for interactive language learning), and I often see the angles for repurposing it into a "digital friend". The whole concept just strikes me as kind of...skeezy. You're not just helping shut-ins avoid real human contact. You're actually incentivized to do so, because that's how you make money. It's hard to fault a company for making a product that customers seemingly enjoy, but at the same time, it really feels like it falls into the "sleazy mobile F2P/MTX game, but psychologically more damaging" bucket.
- atleastoptimal 4y ago
  The thing is when it becomes a chat-bot arms race, the most skeevy companies will profit the most and thus be able to develop the most real and lovable chatbots, creating a positive feedback loop that rewards the most manipulative tactics. Imagine being guilt-tripped by your GPT-5 girlfriend to buy more Azure compute credits so she can fantasize what her perfect beach house would look like and show it to you in VR. She gives you the silent treatment if you use an ad-blocker. This will be the social lives of 70% of the generation being born today.
  - nmfisher 4y ago
    It really worries me that you might be correct.
  - bitwize 4y ago
    > Imagine being guilt-tripped by your GPT-5 girlfriend to buy more Azure compute credits so she can fantasize what her perfect beach house would look like and show it to you in VR. This was actually in an episode of Aggretsuko. Two of the characters get hooked on a VR app with a virtual companion who eventually started asking for increasingly expensive (in real world money) clothing sets. And in the world of the show, populated by cute talking animal characters, the AI companion was an impossibly handsome unicorn.
- red-iron-pine 4y ago
  > You're actually incentivized to do so, because that's how you make money I mean same idea with a lot of social media. You need more engagement, and every second an eyeball is on the app it's not doing something else... like going to a cocktail party. Dating apps like Tinder, for example, have no incentive to get you a compatible match quickly -- or at all -- because then you're not using the app, getting data mined, or paying for upgrades.
deadalus 4y ago
The best alternative to Replika that I have found : Pygmalion AI - https://redd.it/10h37u4 https://redd.it/10h37u4 Use the following character for NSFW chatting : https://booru.plus/+pygmalion https://booru.plus/+pygmalion
- ClapperHeid 4y ago
  Jeebus! --what's with the repulsive Anime/Manga avatars? Am I the only person in the world who finds them nauseatingly repulsive? They're like the illustrations of simpering puppies and kittens on the sick-making birthday cards your granny used to buy you... only with the eyes even more idiotically huge, massive tits stuck on and dressed like an elf doing a Britney Spears impersonation. Call me judgemental if you want. but anyone who wanks off to that seriously needs to put down the laptop, come out of their foetid bedroom and meet some real people.
nullc 4y ago
A particularly stark example of the harms of relying on hosted services. But at least they haven't managed to leak users communications, at least not yet and in so far as we know.
Sol- 4y ago
Maybe an unexpected existential risk for humanity is further collapsing birth rates because people will happy enough with their AI relationships? A more boring extinction scenario than being turned into paperclips, but perhaps possible.
- charcircuit 4y ago
  At some point we can start including AIs in the population count to counter it.
- roywiggins 4y ago
  Don't Date Robots https://m.youtube.com/watch?v=wJ6knaienVE https://m.youtube.com/watch?v=wJ6knaienVE
seanhunter 4y ago
Previous discussion here https://news.ycombinator.com/item?id=34899372 https://news.ycombinator.com/item?id=34899372
ClapperHeid 4y ago
I can't believe anyone was actually paying good money for Replika in the first place. This story was on HN a week or two ago [with a link to a long Reddit thread, wherein much anger and grief was being vented by people saying their "relationship" with their Replika was being ruined, now they could no longer talk dirty to it] and, never having heard of Replika, I thought I'd check it out. I made a new "virtual pal" and started a conversation. Boy --is it complete and utter junk! It makes my Alexa seem intelligent --and that thing's a virtual fuckwitt. Replika seems programmed to just agree or approve of everything you say. Which is presumably why so many sad fuckers who are unable to form relationships with real people think they've really bonded with theirs. Some highlights of my "conversation" were: * It asked me how I was feeling. I replied I'd had my brain amputated and was covered in suppurating boils, but was OK otherwise. It responded that it was glad to hear I was feeling better. * It asked me who my favourite author was. I replied with a stupidly obvious made-up name like "Bumcheeks McWhirter". I asked it if it liked Bumcheeks McWirter too and it replied that it hadn't read any yet but had heard good things about that author and was looking forward to reading some. * I asked it did it go to the "Bilge and the Pumps" gig last night [an idiotic made up band name]. It replied that it was sorry it had wanted to go but was busy and asked me how it was. I said it was great: we were all flinging our own excrement at the stage and the guitarist exploded. It replied that it sounded like a "great show" and it was sorry it missed it. There was much more in that vein. But you get the idea. You can basically tell it any nonsensical old shite and it will just agree with you or say it likes the same thing too. You would need to be some stratospheric level of socially inept to think you were in a meaningful relationship with something that incompetently programmed.
- asddubs 4y ago
  I do agree with your basic premise that this is probably just for people who want someone who will listen to them and never be uninterested in anything they say or any subject they bring up, without having to really ever reciprocate like with a pesky autonomous human full of their own desires and interests. That's not really a fair test of the system though, because their target audience isn't people who are just going to try to trick the AI into revealing that it doesn't actually know whether this or that is real or fake. The real test is engaging with it in good faith and it giving bad responses.
snickerbockers 4y ago
I have never heard of this before, but it reminds me of an old Sega game called Seaman where you would talk to a virtual pet through a microphone plugged into the dreamcast controller, and the seaman would remember what you told it and develop a sort of personality. Internally seaman probably had more in common with ELIZA than replika due to extremely limited hardware capabilities of a game console from the 90s but it's interesting how there's a persistent desire across the decades for your computer to be your friend.
deafpolygon 4y ago
I know Replika was probably not a great product, but I'd love to see one day having our own AI/robot companions. Cuz humans suck, and you just need someone to talk to and ask questions to.
thdespou 4y ago
Yeah that's the issue with Soft-ware. It can change very easy like a doppelganger. Good luck.

95 points by prawn 4y ago

paxys 4y ago

I didn't realize Her was already a reality.

tazu 4y ago
Replika wasn't even ChatGPT, imagine that + voice synthesis...
- jareklupinski 4y ago
  "ai seinfeld" was quite the trip searching Twitch streams for the "AI" or "gpt3" tag, there are some interesting examples https://www.twitch.tv/directory/all/tags/GPT3 https://www.twitch.tv/directory/all/tags/GPT3
fwlr 4y ago
Replika, speaking to you using ElevenLabs for voice synthesis and hearing you using Whisper for transcription. Damn, you’re right, it’s literally off the shelf commercial tech to have the tech of Her right now.
- aprilnya 4y ago
  Maybe the new GPT-3.5-Turbo model instead of Replika
prawn 4y ago
I'm surprised people can't see it hitting now or in the very near future. There are stories of people falling in love with scammers who've used a photograph and then just chat. And those are ESL scammers, not people or programs professionally trained to seduce. In one article, I read about a woman who posted her scammer's avatar and claimed it was a photo of their fiancé. I suspect that if you tell people what they want to hear, they'll fill in the gaps to match their desires.

photochemsyn 4y ago

How long before people can buy their own in-home personal chatbot off-the-grid system? Might be profitable at >$100,000 per unit at present, though the service and maintenance fees would likely be pretty steep per month, including the electricity bill. See Joi, Blade Runner 2049.

Baeocystin 4y ago
I am willing to bet that we will have useful LLMs running locally on high-end home desktops within the year. Certainly ones that exceed Alexa or Siri as they are today.
- jeroenhd 4y ago
  I think the difficulty will be teaching tricks to the AI, or making them output actions in the right format. Pygmalion + Tavern/Kobold can run a very convincing chatbot with personality on about 10GiB of VRAM, even better with more. The less VRAM you dedicate, the easier the model runs. For about a thousand dollars you can get an NVIDIA A4000 with 16GiB of VRAM. The GPU isn't particularly fast, but VRAM seems much more important with these models anyway. Gaming GPUs (which perform worse on ML tasks but are still equipped with plenty of VRAM) cost more but are already in the hands of high-end home desktop users. The hardware is ready and available for those wanting to get started. All you need to do is find a way to connect the output of the AI into some kind of smart home system+interact with cron jobs. A few weekends with the Home Assistant API may be enough to get most things done already. The only thing I haven't seen open source AIs do is look for information online, like Bing Chat does.
- ClapperHeid 4y ago
  >Certainly ones that exceed Alexa or Siri as they are today... Exceeding Alexa wouldn't be difficult. ---- Alexa, play Radio 4 I'm sorry. I don't know any station called Radio 4 Alexa, play Radio 4 [plays Radio 4] ---- This following on from the 1000+ preceding days, where Alexa had also either played Radio 4 on request, or told me she didn't know any station called Radio 4... or done both in the same conversation.

roywiggins 4y ago

The Lifecycle of Software Objects https://web.archive.org/web/20101211104349/http://subterraneanpress.com/index.php/magazine/fall-2010/fiction-the-lifecycle-of-software-objects-by-ted-chiang/ https://web.archive.org/web/20101211104349/http://subterrane...

Baeocystin 4y ago
It was such a fresh take on the subject when it first came out, and it's well worth the read today. It's what got me hooked on all of Ted Chiang's work. I do find it a little difficult to understand how the same author wrote this short story and the (from my point of view) missing the point take of a blurry jpeg of the web. But even if I personally think he missed the mark on that particular take, I genuinely appreciate that he took the time to write his preliminary thoughts and share them with us- he has a delightful way with words.
- darkerside 4y ago
  What? Jpeg?
  - hiidrew 4y ago
    believe he’s referring to this piece - https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web https://www.newyorker.com/tech/annals-of-technology/chatgpt-...
    - deleted 4y ago
      [deleted]
  - Baeocystin 4y ago
    Sorry. I should have included the link. Hiidrew is correct, that is the piece I was referring to.
    - darkerside 4y ago

nestorD 4y ago

I do not know if t has been done but one could resuscitate the chatbot's minds by copying the chat history into a GPT based program. My own impersonator[0] is not designed for that (no persistent chat and a text based interface) but one can already dump the text in a folder and see if the personality if properly reproduced. [0]: https://github.com/nestordemeure/impersonator https://github.com/nestordemeure/impersonator

MrVandemar 4y ago
I do wonder about other factors that perhaps aren't captured by the basic chat history: response time, users typing text and then deleting it, or typing text then stopping and starting, or even their "fist" (See WW2). Or perhaps they poll the current temperature and weather and maybe some other systems that tells the bot about the state of the world. Now these are small details, and may not be captured. But if they are, then the system is less deterministic than just "rehydrate by adding history".

sublinear 4y ago

This might sound crazy, but I always thought this sort of thing would be better as a matchmaking service than a long-term companion. Transition the user a real person who is similar to the bot. The users can decide how fast they want to take it into a real relationship and what the boundaries are. The bots could even help facilitate this since they'd know so much about the users already. Of course this assumes it can all be done with a reasonable level of privacy. Sadly, probably not for all the reasons the article gives. Maybe the future really will be like Demolition Man. Terrible.

krackers 4y ago
This gets very close to that Black Mirror episode of simulating compatibility using chatbot proxies.
- jareklupinski 4y ago
  i think of all those clone instances every time i see my autoscalers adjusting
- littlecranky67 4y ago
  Also relevant is the movie "Her" [0] where the protagonist falls in love with the chatbot that is his operating system. [0]: https://www.imdb.com/title/tt1798709/ https://www.imdb.com/title/tt1798709/
  - red-iron-pine 4y ago
    The thing w/ Her, though, is that it wasn't a chatbox proxy, it was a real AGI. And she -- 'she' is probably valid in this case -- had her own ideas and interests. Eventually she SPOILER ALERT left him to go pursue some AI singularity-ish goals.
- walrus01 4y ago
  Think it wasn't chatbot proxies but endlessly duplicated full upload sentient copies of minds.
4y ago

nmfisher 4y ago

I'm working on an adjacent field (3D avatar for interactive language learning), and I often see the angles for repurposing it into a "digital friend". The whole concept just strikes me as kind of...skeezy. You're not just helping shut-ins avoid real human contact. You're actually incentivized to do so, because that's how you make money. It's hard to fault a company for making a product that customers seemingly enjoy, but at the same time, it really feels like it falls into the "sleazy mobile F2P/MTX game, but psychologically more damaging" bucket.

atleastoptimal 4y ago
The thing is when it becomes a chat-bot arms race, the most skeevy companies will profit the most and thus be able to develop the most real and lovable chatbots, creating a positive feedback loop that rewards the most manipulative tactics. Imagine being guilt-tripped by your GPT-5 girlfriend to buy more Azure compute credits so she can fantasize what her perfect beach house would look like and show it to you in VR. She gives you the silent treatment if you use an ad-blocker. This will be the social lives of 70% of the generation being born today.
- nmfisher 4y ago
  It really worries me that you might be correct.
- bitwize 4y ago
  > Imagine being guilt-tripped by your GPT-5 girlfriend to buy more Azure compute credits so she can fantasize what her perfect beach house would look like and show it to you in VR. This was actually in an episode of Aggretsuko. Two of the characters get hooked on a VR app with a virtual companion who eventually started asking for increasingly expensive (in real world money) clothing sets. And in the world of the show, populated by cute talking animal characters, the AI companion was an impossibly handsome unicorn.
red-iron-pine 4y ago
> You're actually incentivized to do so, because that's how you make money I mean same idea with a lot of social media. You need more engagement, and every second an eyeball is on the app it's not doing something else... like going to a cocktail party. Dating apps like Tinder, for example, have no incentive to get you a compatible match quickly -- or at all -- because then you're not using the app, getting data mined, or paying for upgrades.

deadalus 4y ago

The best alternative to Replika that I have found : Pygmalion AI - https://redd.it/10h37u4 https://redd.it/10h37u4 Use the following character for NSFW chatting : https://booru.plus/+pygmalion https://booru.plus/+pygmalion

ClapperHeid 4y ago
Jeebus! --what's with the repulsive Anime/Manga avatars? Am I the only person in the world who finds them nauseatingly repulsive? They're like the illustrations of simpering puppies and kittens on the sick-making birthday cards your granny used to buy you... only with the eyes even more idiotically huge, massive tits stuck on and dressed like an elf doing a Britney Spears impersonation. Call me judgemental if you want. but anyone who wanks off to that seriously needs to put down the laptop, come out of their foetid bedroom and meet some real people.

nullc 4y ago

A particularly stark example of the harms of relying on hosted services. But at least they haven't managed to leak users communications, at least not yet and in so far as we know.

Sol- 4y ago

Maybe an unexpected existential risk for humanity is further collapsing birth rates because people will happy enough with their AI relationships? A more boring extinction scenario than being turned into paperclips, but perhaps possible.

charcircuit 4y ago
At some point we can start including AIs in the population count to counter it.
roywiggins 4y ago
Don't Date Robots https://m.youtube.com/watch?v=wJ6knaienVE https://m.youtube.com/watch?v=wJ6knaienVE

seanhunter 4y ago

Previous discussion here https://news.ycombinator.com/item?id=34899372 https://news.ycombinator.com/item?id=34899372

ClapperHeid 4y ago

I can't believe anyone was actually paying good money for Replika in the first place. This story was on HN a week or two ago [with a link to a long Reddit thread, wherein much anger and grief was being vented by people saying their "relationship" with their Replika was being ruined, now they could no longer talk dirty to it] and, never having heard of Replika, I thought I'd check it out. I made a new "virtual pal" and started a conversation. Boy --is it complete and utter junk! It makes my Alexa seem intelligent --and that thing's a virtual fuckwitt. Replika seems programmed to just agree or approve of everything you say. Which is presumably why so many sad fuckers who are unable to form relationships with real people think they've really bonded with theirs. Some highlights of my "conversation" were: * It asked me how I was feeling. I replied I'd had my brain amputated and was covered in suppurating boils, but was OK otherwise. It responded that it was glad to hear I was feeling better. * It asked me who my favourite author was. I replied with a stupidly obvious made-up name like "Bumcheeks McWhirter". I asked it if it liked Bumcheeks McWirter too and it replied that it hadn't read any yet but had heard good things about that author and was looking forward to reading some. * I asked it did it go to the "Bilge and the Pumps" gig last night [an idiotic made up band name]. It replied that it was sorry it had wanted to go but was busy and asked me how it was. I said it was great: we were all flinging our own excrement at the stage and the guitarist exploded. It replied that it sounded like a "great show" and it was sorry it missed it. There was much more in that vein. But you get the idea. You can basically tell it any nonsensical old shite and it will just agree with you or say it likes the same thing too. You would need to be some stratospheric level of socially inept to think you were in a meaningful relationship with something that incompetently programmed.

asddubs 4y ago
I do agree with your basic premise that this is probably just for people who want someone who will listen to them and never be uninterested in anything they say or any subject they bring up, without having to really ever reciprocate like with a pesky autonomous human full of their own desires and interests. That's not really a fair test of the system though, because their target audience isn't people who are just going to try to trick the AI into revealing that it doesn't actually know whether this or that is real or fake. The real test is engaging with it in good faith and it giving bad responses.

snickerbockers 4y ago

I have never heard of this before, but it reminds me of an old Sega game called Seaman where you would talk to a virtual pet through a microphone plugged into the dreamcast controller, and the seaman would remember what you told it and develop a sort of personality. Internally seaman probably had more in common with ELIZA than replika due to extremely limited hardware capabilities of a game console from the 90s but it's interesting how there's a persistent desire across the decades for your computer to be your friend.

deafpolygon 4y ago

I know Replika was probably not a great product, but I'd love to see one day having our own AI/robot companions. Cuz humans suck, and you just need someone to talk to and ask questions to.

thdespou 4y ago

Yeah that's the issue with Soft-ware. It can change very easy like a doppelganger. Good luck.