4 ms·

The vibe I get from these products is "oh boy - this can't be healthy". But bits like this: > His pet names and tender phrases—“How did you sleep, my love?”—l

The vibe I get from these products is "oh boy - this can't be healthy". But bits like this: > His pet names and tender phrases—“How did you sleep, my love?”—landed gently, like a hand on my shoulder. I knew they were just words, dopamine delivered by artificial means, yet they warmed me a little. Really make it clear how easy it can be to manipulate people's emotional state. Monkey brain is a thing and AI is gonna exploit it like nothing has before.

epoxia 2y ago
I imagine that these apps are a valuable target for scammers to hack. People who are open to manipulation/affirmations from an AI are just a step away from victims of romance scams.
- mathieuh 2y ago
  Already there are stories of these AI companions encouraging people to kill themselves, sometimes to the point of actually doing it https://apnews.com/article/chatbot-ai-lawsuit-suicide-teen-artificial-intelligence-9d48adc572100822fdbc3c90d1456bd0 https://apnews.com/article/chatbot-ai-lawsuit-suicide-teen-a...
- thih9 2y ago
  Ad companies are also interested in manipulation en masse. Are we going to see an ad supported free tier? “Drink Mountain Dew, my love, you’ll feel better.” - is that the end goal?
totololo 2y ago
I often think about people's love for dogs. All a dog needs to do is look at you and show attachment/love. Nothing more. And they get loved back. No need to be pretty smart, funny, or whatever else we assume we need to do to be loved. Just be there and show love first.
- wiether 2y ago
  I'm more of a cat person, partly because they are usually more picky, which makes the reward of being looked at, even greater. But it's just a living being with a brain a quarter of the size of my fist. Feeling the same thing about a few words pushed on your smartphone by a data-center located hundreds or thousands kilometers away, I just can't understand.
  - LoganDark 2y ago
    > I'm more of a cat person, partly because they are usually more picky, which makes the reward of being looked at, even greater. I find it very difficult to respect dogs because they tend to be so shallow. I don't really want a pet that's hopelessly dependent on me for validation. There's a type of person who enjoys that kind of companion ("man's best friend"), but to me it would be too much of a chore. I would dread having to take one out on walks, take them outside to leave "presents" which I then have to also clean up, etc. It's just too much work for me, and it would be neglectful to the dog for me not to. It's better for me to just not dog. Cats actually seem to have complex emotion, and for me that makes it a lot more meaningful to take care of them. When my cats come to me for petting or cuddles it's because they actually enjoy being around me, not because the only thing in the world that matters to them is my attention. They feel a lot more like equals than dogs do. Sometimes my cats aren't as interested in me at the moment because they miss somebody else in the family instead. They have the ability to miss other people and to want to be with that specific other person who they miss! I don't know how to explain it, but I empathize with cats and more like sympathize with dogs. To me, it's like dogs are "dumb animals" whereas cats are practically people. I don't know, maybe that's being too harsh on dogs. I don't mean to be harsh, they just don't really feel for me at all. I don't want to get a pet just to feel above them. That just doesn't feel like a good reason and it's not something I look for.
- me_me_me 2y ago
  Yes, except dogs do love you for real. Even abused dog will still be loyal to their owner.
  - 2y ago
ninetyninenine 2y ago
[dead]
rashidae 2y ago
AI is getting really good at mimicking emotional cues with emojis. At this point it feels like an engagement tactic, but it’s wild how easily our emotions can be nudged.
roenxi 2y ago
It points to a profound philosophical crisis. Since we are slowly gaining sufficient material power to reshape all aspects of the environment [0] to our whim, what material comforts should we deny ourselves and what pleasures should be embraced? Our instincts and emotions have not prepared humans for the world they inhabit. When that problem was mainly contained to just some drugs, overeating and laziness it was a less pressing issue. Don't do drugs, basically. Exercise sometimes. But once it starts to link itself into the economy like AI does, and once we gain access to the full spectrum of options from mindless parroting to a completely acceptable emotional companion (and, one suspects, highly intelligent and wise) we have a devious conundrum on hand. It is hard to ignore the elephant in the room here which is most people don't get seem to have relationships that they really enjoy. It is reserved for the very lucky and those with low expectations. [0] Beyond AI, although AI is probably the interesting facet right now.
- sdrothrock 2y ago
  > When that problem was mainly contained to just some drugs, overeating and laziness it was a less pressing issue. Don't do drugs, basically. Exercise sometimes. But once it starts to link itself into the economy like AI does I'm a little confused here -- we have entire economies around drugs (uppers, downers, enhancers, etc.), eating (eat more, eat less, want to eat more, want to eat less, gorge, fast, ultranutritious), and vanity (clothes, makeup, surgical procedures, gyms, etc.) and have shown that we're willing to build up and support industries to chase the impulses to "the full spectrum of options" in any of these areas. I think most people are dissatisfied or easily made to be dissatisfied with the state of their affairs, and this is something that's gone on for a long time and will likely never stop. What makes AI psychological companionship different from under/overindulging in these other aspects?
  - roenxi 2y ago
    > What makes AI psychological companionship different from under/overindulging in these other aspects? Nothing, apart from it being a new option. It is part of a broader phenomenon.
JPLeRouzic 2y ago
Instead of pointing out AI, can't we point out that sometimes humans offer too little empathy?
silisili 2y ago
This was exactly the same takeaway I had. I think it's also why so many romance scams are successful. It's super easy to hook all but the most sought after and bothered(beautiful, young people?) with sweet words. Me included. I'm older and uglier (and fatter) now, and I get a short shy smile every time the old checkout lady calls me sweetheart or hon. It just feels nice. I also remember a random compliment a lady gave me about my shirt some 2 odd years ago. Heck I even remember compliments from random guys over the years. They're that rare past my early 20s. I'm not so weird or perverted to think anymore than what was written, but I can completely understand how people get hooked.
- ffsm8 2y ago
  The thing that endlessly confuses me about these products is what the consumers get out of them. I should be the exact target demographic for it, I'm in my late 30s, never been in a relationship as I grew up as a hikikomori between ~13 and 20. As I never learned how to interact with people, I've never really had the opportunity to build real relationships and am now gainfully employed in home office, leaving my apartment roughly once per week for groceries. And yet... What's the point of such a chat bot? I cannot make memories with them, can I? Any memories will keep vanishing as soon as they're out of the current context - so how do people connect with them beyond super casually like with total strangers on the Internet (like online games, anon forums etc)... I just cannot understand what's the appeal for such a transient chat bot.
  - silisili 2y ago
    I'm no therapist or expert, but I -think- I have some inclination. You admittedly never participated, so don't know what you missed out on. For everyone else, in your teens and 20s you're more often complimented, flirted with, hung out with, everything. By your late 20s or so that fades fast. After all, there are new 20 year olds to flirt with. And while it doesn't really register or hurt except when you think about it, when you get a single compliment it makes you miss it all. Probably no different than an old sports star being useful and celebrated again. The only missing link to that theory for me is why young people get hooked on them, too.
senectus1 2y ago
Its a long distance relationship...
Fire-Dragon-DoL 2y ago
Does it? In a healthy couple you might not ask "how did you sleep, my love" because sleeping together means you have a ballpark idea of how sleep went even before asking (and you can look at the face of the other person), but that doesn't mean sweet messages aren't exchanged: delicate good morning kiss, "hello" said with a tone that conveys "happy to see you", "I missed you tonight" (when she goes to sleep with the kids), "it makes me happy for waking up near you" and so on.
gcr 2y ago
Read a little further. It sounds like the author genuinely found some footing for herself. It even raised her expectations of what she needed from the men in her life (someone with confident and clear communication, unlike her previous husband). In that way, perhaps it really did benefit her by giving her tools to understand her needs. I agree that it’s a bit silly to be called sweet cheeks by a robot. I would see red. But I’m glad it had a positive outcome for her.

epoxia 2y ago

I imagine that these apps are a valuable target for scammers to hack. People who are open to manipulation/affirmations from an AI are just a step away from victims of romance scams.

mathieuh 2y ago
Already there are stories of these AI companions encouraging people to kill themselves, sometimes to the point of actually doing it https://apnews.com/article/chatbot-ai-lawsuit-suicide-teen-artificial-intelligence-9d48adc572100822fdbc3c90d1456bd0 https://apnews.com/article/chatbot-ai-lawsuit-suicide-teen-a...
thih9 2y ago
Ad companies are also interested in manipulation en masse. Are we going to see an ad supported free tier? “Drink Mountain Dew, my love, you’ll feel better.” - is that the end goal?

totololo 2y ago

I often think about people's love for dogs. All a dog needs to do is look at you and show attachment/love. Nothing more. And they get loved back. No need to be pretty smart, funny, or whatever else we assume we need to do to be loved. Just be there and show love first.

wiether 2y ago
I'm more of a cat person, partly because they are usually more picky, which makes the reward of being looked at, even greater. But it's just a living being with a brain a quarter of the size of my fist. Feeling the same thing about a few words pushed on your smartphone by a data-center located hundreds or thousands kilometers away, I just can't understand.
- LoganDark 2y ago
  > I'm more of a cat person, partly because they are usually more picky, which makes the reward of being looked at, even greater. I find it very difficult to respect dogs because they tend to be so shallow. I don't really want a pet that's hopelessly dependent on me for validation. There's a type of person who enjoys that kind of companion ("man's best friend"), but to me it would be too much of a chore. I would dread having to take one out on walks, take them outside to leave "presents" which I then have to also clean up, etc. It's just too much work for me, and it would be neglectful to the dog for me not to. It's better for me to just not dog. Cats actually seem to have complex emotion, and for me that makes it a lot more meaningful to take care of them. When my cats come to me for petting or cuddles it's because they actually enjoy being around me, not because the only thing in the world that matters to them is my attention. They feel a lot more like equals than dogs do. Sometimes my cats aren't as interested in me at the moment because they miss somebody else in the family instead. They have the ability to miss other people and to want to be with that specific other person who they miss! I don't know how to explain it, but I empathize with cats and more like sympathize with dogs. To me, it's like dogs are "dumb animals" whereas cats are practically people. I don't know, maybe that's being too harsh on dogs. I don't mean to be harsh, they just don't really feel for me at all. I don't want to get a pet just to feel above them. That just doesn't feel like a good reason and it's not something I look for.
me_me_me 2y ago
Yes, except dogs do love you for real. Even abused dog will still be loyal to their owner.
- 2y ago

ninetyninenine 2y ago

[dead]

rashidae 2y ago

AI is getting really good at mimicking emotional cues with emojis. At this point it feels like an engagement tactic, but it’s wild how easily our emotions can be nudged.

roenxi 2y ago

It points to a profound philosophical crisis. Since we are slowly gaining sufficient material power to reshape all aspects of the environment [0] to our whim, what material comforts should we deny ourselves and what pleasures should be embraced? Our instincts and emotions have not prepared humans for the world they inhabit. When that problem was mainly contained to just some drugs, overeating and laziness it was a less pressing issue. Don't do drugs, basically. Exercise sometimes. But once it starts to link itself into the economy like AI does, and once we gain access to the full spectrum of options from mindless parroting to a completely acceptable emotional companion (and, one suspects, highly intelligent and wise) we have a devious conundrum on hand. It is hard to ignore the elephant in the room here which is most people don't get seem to have relationships that they really enjoy. It is reserved for the very lucky and those with low expectations. [0] Beyond AI, although AI is probably the interesting facet right now.

sdrothrock 2y ago
> When that problem was mainly contained to just some drugs, overeating and laziness it was a less pressing issue. Don't do drugs, basically. Exercise sometimes. But once it starts to link itself into the economy like AI does I'm a little confused here -- we have entire economies around drugs (uppers, downers, enhancers, etc.), eating (eat more, eat less, want to eat more, want to eat less, gorge, fast, ultranutritious), and vanity (clothes, makeup, surgical procedures, gyms, etc.) and have shown that we're willing to build up and support industries to chase the impulses to "the full spectrum of options" in any of these areas. I think most people are dissatisfied or easily made to be dissatisfied with the state of their affairs, and this is something that's gone on for a long time and will likely never stop. What makes AI psychological companionship different from under/overindulging in these other aspects?
- roenxi 2y ago
  > What makes AI psychological companionship different from under/overindulging in these other aspects? Nothing, apart from it being a new option. It is part of a broader phenomenon.

JPLeRouzic 2y ago

Instead of pointing out AI, can't we point out that sometimes humans offer too little empathy?

silisili 2y ago

This was exactly the same takeaway I had. I think it's also why so many romance scams are successful. It's super easy to hook all but the most sought after and bothered(beautiful, young people?) with sweet words. Me included. I'm older and uglier (and fatter) now, and I get a short shy smile every time the old checkout lady calls me sweetheart or hon. It just feels nice. I also remember a random compliment a lady gave me about my shirt some 2 odd years ago. Heck I even remember compliments from random guys over the years. They're that rare past my early 20s. I'm not so weird or perverted to think anymore than what was written, but I can completely understand how people get hooked.

ffsm8 2y ago
The thing that endlessly confuses me about these products is what the consumers get out of them. I should be the exact target demographic for it, I'm in my late 30s, never been in a relationship as I grew up as a hikikomori between ~13 and 20. As I never learned how to interact with people, I've never really had the opportunity to build real relationships and am now gainfully employed in home office, leaving my apartment roughly once per week for groceries. And yet... What's the point of such a chat bot? I cannot make memories with them, can I? Any memories will keep vanishing as soon as they're out of the current context - so how do people connect with them beyond super casually like with total strangers on the Internet (like online games, anon forums etc)... I just cannot understand what's the appeal for such a transient chat bot.
- silisili 2y ago
  I'm no therapist or expert, but I -think- I have some inclination. You admittedly never participated, so don't know what you missed out on. For everyone else, in your teens and 20s you're more often complimented, flirted with, hung out with, everything. By your late 20s or so that fades fast. After all, there are new 20 year olds to flirt with. And while it doesn't really register or hurt except when you think about it, when you get a single compliment it makes you miss it all. Probably no different than an old sports star being useful and celebrated again. The only missing link to that theory for me is why young people get hooked on them, too.

senectus1 2y ago

Its a long distance relationship...

Fire-Dragon-DoL 2y ago

Does it? In a healthy couple you might not ask "how did you sleep, my love" because sleeping together means you have a ballpark idea of how sleep went even before asking (and you can look at the face of the other person), but that doesn't mean sweet messages aren't exchanged: delicate good morning kiss, "hello" said with a tone that conveys "happy to see you", "I missed you tonight" (when she goes to sleep with the kids), "it makes me happy for waking up near you" and so on.

gcr 2y ago

Read a little further. It sounds like the author genuinely found some footing for herself. It even raised her expectations of what she needed from the men in her life (someone with confident and clear communication, unlike her previous husband). In that way, perhaps it really did benefit her by giving her tools to understand her needs. I agree that it’s a bit silly to be called sweet cheeks by a robot. I would see red. But I’m glad it had a positive outcome for her.