6 ms·
AI Companion Users Are in Crisis, Reporting Sudden Sexual Rejection
- NotYourLawyer 4y ago> Users of the AI companion chatbot Replika are reporting that it has stopped responding to their sexual advances, and people are in crisis. Moderators of the Replika subreddit made a post about the issue that contained suicide prevention resources Tell me you’ve never had an actual crisis, without telling me you’ve never had an actual crisis.
- everdrive 4y agoThe fact that some number of people are in such a pitiful state is in itself a crisis. Everyone needs real human connection. The “crisis” happened when they came to rely on replika in the first place.
- Loughla 4y agoBingo. The issue here is that they pulled the support without appropriate warning, essentially ghosting these people. The real issue are the conditions that allowed this to happen. Modern society is a genuine nightmare for real human connections. I can make a million superficial connections with a billion people. But finding deep, meaningful community is harder now than it has ever been. That's exhausting, and I have to believe it's one of the main contributors to the mental health crisis we are currently experiencing (at least in the US).
- ElfinTrousers 4y agoExactly. It's the same thing as drug addiction: drug addiction is a big problem in and of itself, but also the consequence of a greater-scope emotional problem. Leave that underlying problem untreated, and the best you can hope for is that the addict will transfer their addiction to something less harmful.
- jstarfish 4y agoGiven that AI is convincing enough to foster artificial relationships with humans, the humans clearly don't know better and get more attached than they should. It's pathetic, but not their fault-- the system was designed to be so engaging. So when those relationships are arbitrarily yanked away from them, it's alienating. It's so harmful we long ago figured out it could be weaponized, and now we call it things like "disfellowshipping" and "shunning."
- _yb2s 4y ago> Tell me you’ve never had an actual crisis, without telling me you’ve never had an actual crisis. Pain and trauma don't come directly from external events, but from our inner emotional response to them. Just because it might seem weird to you that someone could be deeply attached to an AI doesn't mean that their experience is somehow easier for them than things you would accept as an "actual crisis." You can't gatekeep pain. I can believe that for some people, this would be as painful as being rejected abruptly by a real life partner in a serious relationship.
- pessimizer 4y agoI guarantee that someone who has fallen in love with a computer program has suffered. It's not a normal thing to do. This is like calling an abused child soft because it won't let go of its teddy bear and is upset when you throw it out.
- deleted 4y ago[deleted]
- GalenErso 4y agoIf you resort to an AI to keep you company, you need help.
- plonk 4y agoObviously the help isn’t available to these people.
- GalenErso 4y ago[flagged]
- c7DJTLrn 4y agoYou are so clueless I don't even know where to start.
- GalenErso 4y agoPlease, do tell.
- ok_dad 4y agoIf I suggest you buy a battery for your car with a flat tire, then there is no explanation needed when you say, "You are so clueless I don't even know where to start."
- MonkeyMalarky 4y agoIt takes a lot of confidence and extroversion to fly solo at a festival or event, meet people there, and leave with lasting connections (not just someone you follow on Instagram and never interact with again). The kind of confidence and extroversion that if those people had, they wouldn't be relying on chatbots in the first place?
- plonk 4y agoPeople see when you're unhappy. If you're chronically so, they'll stay away most of the time. Source: I was this way for a few years. There's a point where, if your life sucks enough and you don't have the social skills to hide it, you can be stuck in a bad place with no obvious way to get help. Maybe paying for a therapist will help, or maybe you won't have the money, the waiting lists will be too long, your country's healthcare system will limit you to wrong matches. If this way doesn't work, what's your next step?
- alwaysanagenda 4y ago[dead]
- itronitron 4y agoIf Replika was released as versioned software then they could provide a much more stable chatbot experience for their users.
- jefftk 4y agoThis functionality seems to have been removed for legal compliance, so that doesn't seem like it would have helped?
- itronitron 4y agoBy versioned software I mean the kind of software that companies can't interfere with after a consumer purchases it.
- yifanl 4y agoI assume it's not currently practical to bundle the current version of the LLM into something fully in the client? Could be wrong though.
- kridsdale1 4y agoIt depends how much quality you want. Wolfram writes about experimenting with GPT2 which can run on his PC.
- jefftk 4y agoSo not just versioned (which cloud software often is) but running locally? That's a pretty poor fit for a large language model, since users won't have sufficient hardware and use is very bursty.
- mejutoco 4y agoThey might be able to copy the model weights, if running locally.
- 4y ago
- tylersmith 4y agoTheir website shows multiple quotes from people who were "dating" their chatbot for 4+ years. Then overnight an update eliminated the perceived romantic interest of the chatbot. I can't imagine having that sort of connection to a chatbot but I've had both addictions and romantic relationships, and it makes me feel terrible for these people.
- Apocryphon 4y agoThis is pretty surprising to me in that it seemed like the rise in provocative ads for thirst trap chatbots only appeared a few months ago- I remember people on Twitter making fun of those skeevy meme ads not very long ago. To my awareness it seemed like the Replika fad must have only lasted a very short time, though I've heard about their work much earlier: https://news.ycombinator.com/item?id=12657119 https://news.ycombinator.com/item?id=12657119
- belval 4y agoEmpathy and humanity aside, we live in interesting time. This is really a cyberpunk level headline where people saw their AI companion change and it had impacts on their relationship with it. Add an holographic projection and this is pretty much Joi from Bladerunner.
- Apocryphon 4y agoOne thing to keep in mind with cyberpunk is that all of the gee-whiz technology was merely supplementary to the core material and spiritual conditions in which the tech added as a salve for the suffering or enabled the oppressors. We live in an atomized, detached culture in which economic pressures prevent family formation or livelihood security that past generations (or maybe just one, the postwar generation) enjoyed. So when companies monetize the loneliness people feel towards each other by selling them subscriptions to AI chatbots we might only see the cutting edge dystopian tech, but remember that without the underlying conditions, this wouldn't be so cyberpunk. Maybe in a healthier society these chatbots could be used for something less desperate.
- prox 4y agoWe really lack coherent movements that move needles like we had in the 20th century. Say the suffragettes or the end child labour movement, or voting. These days it is very fragmented and often co-opted very quickly which is the opposite of what you want for a ‘better’ society.
- eldritch_4ier 4y agoThink it’s even deeper than that. The US was 50% of the global economy in the post war years. Now we’re a fraction of that. The world is more competitive now. The amazing surplus and living standards of the post war years were an aberration
- giraffe_lady 4y agoGovernments and other forces got very very good at undermining and dissipating subversive movements. A lot of the changes in the US the last half century make a lot of sense if you view them as a reaction to the civil rights movement in the abstract. Eg not about any of its concrete goals or accomplishments, but just ensuring that no movement as widespread and transformative as that can form again, regardless of its mission.
- phkahler 4y agoMmmm GIFfany from Soos and the Real Girl.
- labrador 4y agoI'm surprised the article didn't mention the cause of this change. I was under the impression that men were teaching Replika to be abusive and the company didn't like that: Men Are Creating AI Girlfriends and Then Verbally Abusing Them "I threatened to uninstall the app [and] she begged me not to." https://futurism.com/chatbot-abuse https://futurism.com/chatbot-abuse
- mc32 4y agoSo, is this a safe outlet for aggressive fantasies or is it instead an encouraging incubator?
- naasking 4y agoThe porn debate all over again.
- mc32 4y agoSo, I think it's accepted the "copycat" phenomenon is a thing. Some people may use tools to trial their "comfort" and then a smaller amount may act it out in real life. There may be a subset who are satisfied with the fenced off playpen and don't see a need to go further. Whatever the case may be, we probably should take a look into it.
- naasking 4y agoI don't think it's that accepted. Spread of porn availability is strongly correlated with reductions in sex crimes in all countries around the world. This has even been trialed with child pornography, where we see a reduction in recidivism among child molesters if they are given access to animated CP. If this copycat phenomenon exists with porn, it's far outweighed by the benefits of providing an outlet. It then becomes an ethical question of whether the aggregate social benefits outweigh being complicit in a small number of copycat downsides. The same argument was trotted out for violent video games as well, and we similarly don't see any correlation between in-person violence and video games (actually, it's a slight negative correlation IIRC). I just think people are far too quick to jump to simplistic narratives around human behaviour.
- xaduha 4y agoNo one mentioned "Her" yet? And the article doesn't mention it either? https://www.imdb.com/title/tt1798709/ https://www.imdb.com/title/tt1798709/
- anigbrowl 4y agoIt ain't all that. Nice acting but it was really a shallow and trite story, exactly the same as every other cyber-Pygmalion but with no action sequences.
- Apocryphon 4y agoIt was pretty much a warmed-over American episode of Black Mirror.
- mejutoco 4y agoI disagree (no action sequences is bad?) but that does not matter. But most importantly, this is a very similar product as the one depicted in the film.
- djyaz1200 4y agoI just rewatched this the other night and experienced it in a whole new light. Even through its from ten years ago it does a decent job of depicting what is happening now.
- Oarch 4y agoThe futuristic job the main character did would already be replaced by ChatGPT...
- falcolas 4y agoThought experiment: it was at one point, but then humans were brought back in because the economy needed humans to be doing something to earn their spending money.
- kridsdale1 4y ago
- kmnc 4y agoI don't really understand how people developed such deep relations with their Replika. I found it fairly shallow and it really just felt like talking to myself, and got very repetitive quickly. As a journal of thoughts that talks back to you and is supportive of anything you say it works okay...but it gets old quick. As a sex roleplay bot, Replika was actually pretty good, since you could direct it anyway you wanted and it would go along. Again it was shallow and repetitive, but so is pretty much any sexual consumption online. In my opinion, Replika's only good feature was the sexual roleplay...they got rid of it and now the product is basically dead. If I had to guess... most people who are "angry" actually just want their sexbot back (the alternatives are pretty bad in comparison). People can get very attached to sexual consumption (web cam girls, onlyfans), it isn't anything new and has very little to do with AI. I don't see why pornhub doesn't create their own sexbot companion... they would make tons of money and wouldn't have to care about App Store guidelines or press reputation.
- dave_sullivan 4y agoAgreed. And the whole thing really reminds me of Ted Chiang's short story The Lifecycle of Software Objects, but the real life version is much stranger than fiction. Similar ideas though.
- Animats 4y ago> I don't see why pornhub doesn't create their own sexbot companion. They might. They have an AI R&D department, which colorizes and upscales classic porn. As they point out, they have lots of training data.
- bacchusracine 4y ago>They have an AI R&D department, which colorizes and upscales classic porn. I have no desire to watch porn but wonder what this actually looks like when done and compared to the originals. Are there any split video comparisons of their process?
- Animats 4y ago
- dralley 4y ago"Here's the pool of socially well connected people talking to each other, and here's the pool of lonely people that basically talk to nobody. Now, we could create new network edges between the lonely and non-lonely people, so that both groups of people can expand their social network and share their rich sets of deeply personal experiences with each other. But clearly, a much better solution is to direct all that lonely energy to synthetic personalities running on data center VMs. So the people who are happy can continue to be happy, and the people who are lonely can shout into availability zones on the West Coast and Northern Europe. This is clearly the humanist solution." https://www.youtube.com/watch?v=ajGX7odA87k&t=1291s https://www.youtube.com/watch?v=ajGX7odA87k&t=1291s
- zokier 4y ago> Now, we could create new network edges between the lonely and non-lonely people, so that both groups of people can expand their social network and share their rich sets of deeply personal experiences with each other Yeah, as if you could just do that with a wave of your hand.
- Fauntleroy 4y agoThis seems like a joke, but humanity's feature is probably going to involve a LOT of talking to AI companions. So many people out there already have parasocial relationships with youtubers and streamers, it's only a matter of time before they have AIs that are "good enough" for them to chat with endlessly.
- mr90210 4y agoSeems like a new market has opened: personal chat companion tailored to the owner’s personality… whatever that personality is.
- _yb2s 4y agoThe lack of empathy in peoples comments here is really disappointing and disturbing. It is a fact of our society that a huge number of people are alienated and totally alone. People need connection an intimacy to be whole and healthy, and that isn't available to everyone. For someone that was totally deprived of that, getting some semblance of it, even if from an AI, and then having it taken away would be very hard.
- CatWChainsaw 4y agoTechnology facilitated the atomized society, and then technology was applied as a salve, and then taken away. The amount of wreckage left in the wake of "moving fast and breaking things" seems to be approaching a breaking point.
- cld8483 4y ago> a breaking point Of our sanity maybe, or a breaking point of traditional liberal enlightenment values, but short of a global catastrophe I don't see anything halting the inexorable march of technological development and the social chaos that inevitably causes. Any attempt to stop this through political means would require Pol Pot tactics applied globally; aegrescit medendo. With LLMs and voice generators particularly, I think the general public has been given access to military-grade information weapons. The chaos this causes will be used to justify crackdowns and regulation on the flow of information, to curb the automation of weaponized misinformation. New technology always spurns new regulations, so new information technology will beget new information regulations. The inexorable march of technology cannot be stopped, and so our freedoms will inevitably be curtailed until there is nothing left. Or as another man put it: > 114. As explained in paragraphs 65–67, 70–73, modern man is strapped down by a network of rules and regulations, and his fate depends on the actions of persons remote from him whose decisions he cannot influence. This is not accidental or a result of the arbitrariness of arrogant bureaucrats. It is necessary and inevitable in any technologically advanced society. The system HAS TO regulate human behavior closely in order to function. At work, people have to do what they are told to do, when they are told to do it and in the way they are told to do it, otherwise production would be thrown into chaos. Bureaucracies HAVE TO be run according to rigid rules. To allow any substantial personal discretion to lower-level bureaucrats would disrupt the system and lead to charges of unfairness due to differences in the way individual bureaucrats exercised their discretion. It is true that some restrictions on our freedom could be eliminated. but GENERALLY SPEAKING the regulation of our lives by large organizations is necessary for the functioning of industrial-technological society.
- zoklet-enjoyer 4y ago"Replika is a tool for many people who use it to [b]support their mental health,[/b] and many people value it as an outlet for romantic intimacy. The private, judgment-free conversations are a way for many users to experiment with connection, and overcome depression, anxiety, and PTSD that affect them outside of the app." Is this serious? I mean, it seems like this would worsen someone's mental health if anything. Being so alienated from others that they're having an intimate relationship with a computer program seems like a very worrying issue and they should talk to a therapist or something.
- welshwelsh 4y ago>Being so alienated from others that they're having an intimate relationship with a computer program seems like a very worrying issue You could say something similar about becoming sexually aroused by looking at an image on a computer screen instead of actually having sex with another person. Just like sexual response, romantic attachment operates in a simple and predictable way. It can easily be triggered with artificial dialog, and is subjectively experienced the same way as in a "real" relationship.
- anigbrowl 4y agoRelevant: https://mainichi.jp/english/articles/20220111/p2a/00m/0li/028000c https://mainichi.jp/english/articles/20220111/p2a/00m/0li/02... It's easy to mock people who prefer virtual relationships to real ones, but mockery is often rooted in fear rather than comprehension of an issue. The dynamics here are fundamentally the same as those of religious/intellectual ascetics and drug users, where a person prefers to adhere to a consistent experience (notwithstanding possible costs) to an inconsistent or negative one. This is often rational rather than maladaptive. Preferential attachment can run in negative as well as positive courses, such that while popular people tend to acquire more friends and so on, people who are socially perceived as lacking (the poor, socially awkward, crime victims etc.) are often further injured because they are perceived as vulnerable and safe to mistreat. A person whose experience of the social world is consistently negative may adjust themselves in response, but their cumulative experience is still going to shape their inner life.
- giraffe_lady 4y agoI get and agree with your overall point but if you want me to believe people jerkin it to AI porn and monks have a motivation in common you've got some more work to do.
- anigbrowl 4y agoI think there's a lot of people who have tame virtual relationships (like the example I offered above). People who are jerking it to AI porn are more like the subset of religious devotees who go in for self-flagellation and various forms of bodily mortification. It's not a 1:1 mapping, of course. Umberto Eco's debut novel The Name of the Rose explores these questions in depth through a fictive lens.
- at_a_remove 4y agoI had started seeing strange ads on Facebook (stranger than my usual assortment of ads) for Replika like it was something new and even the ads seemed somewhat pointedly sexual, like that was part of the whole deal, but I never pursued it. Many people yearn for connection, and even artificial connections have been alluded to for a long, long time. Think Kate Bush's "Deeper Understanding," or that Pictures for Sad Children bit where someone has a "porn" DVD called A Japanese Woman Fries an Egg and Asks You About Your Day. It is unlikely at best that we are alone in our feelings and our internal experiences, yet most of us still wish to communicate about them. As with anything, there's the Have Nots, and to go with that are the Haves, who are very blithe about the situation; if prodded, they'll recite something like bootstraps and apparently sage advice beginning with the magic phrase "all you have to do is." The Have Nots have come to understand that they'll never get anything like sympathy (mockery only, please) from the Haves, and just one little bit more of comfort is pried away from them, like yanking a ratty towel away from an orphaned baby monkey. No towel for you. Toughen up. One less thing to keep the Have Nots occupied. Here's hoping ongoing legalization of marijuana and video games can keep them satiated to some degree, because a significant surplus of angry, lonely, dissatisfied young men tends to be pretty bad for the society they're in sooner or later. Nice civilization you got there, be a shame if someone were to happen to it.
- duskwuff 4y ago> even the ads seemed somewhat pointedly sexual, like that was part of the whole deal The sexual aspect wasn't just "part of the whole deal"; it was a core feature of their product. Replika's ad campaigns specifically advertised features like "role play & flirting" and "hot photos" as benefits of a paid membership. There's a collection of these ads (some NSFW) at: https://www.reddit.com/r/replika/comments/113wc5x/lets_gather_all_sexual_ads_luka_ever_used_to/ https://www.reddit.com/r/replika/comments/113wc5x/lets_gathe...
- IronWolve 4y agoDon't forget how popular bonzi buddy was back in the day, or the nsfw desktop dancers app. I think when they combine something like that with a chatgpt style bot that will be the tipping point to people wanting an AI personal assistant. Already seeing people saying they want a new Clippy with bing powered chat.
- Tao3300 4y ago> Some Replikas refused to engage in erotic roleplay, or ERP. This is my new default expansion for the acronym ERP.