6 ms·

I hate everything about this on so many different levels. That people are so lonely that they invest emotionally in an AI is flooring. People talk about AI be

I hate everything about this on so many different levels. That people are so lonely that they invest emotionally in an AI is flooring. People talk about AI being "good for therapy" and similar, but perhaps having a real human being on the other side of an interaction is important in case things go South. E.g., if someone is depressed, tells an AI what's wrong with their life and that they're suicidal, what if suicide seems like a valid option? There's plenty of prompts to "jailbreak" LLMs and take away "safety" measures. Some things simply shouldn't be left to an LLM to not "hallucinate" over or go off-script with. Yes, being a therapist is a challenging occupation, but there are plenty who love their career, find it rewarding, and do some excellent work on top of all that.

sigmoid10 4y ago
Even if you find a decent psychologist, good luck finding insurance that covers $200 per hour just for talking to them when you don't have any diagnosed issue. The accessibility to the lower income end of the population is what makes this valuable, not the quality of the conversations. Anything is better than nothing in this case and most people just don't realize how gigantic and commonplace the issue really is.
- hammyhavoc 4y ago
  In the UK, you can access a psychologist for free, and whilst that's no consolation to my American peers, it's furthering the point that America's healthcare system is fundamentally flawed. Technological problems are ideally solved by technology. Social problems aren't always best solved with technology, they're a band-aid. Well, something being fit-for-purpose is an important consideration. If it tells people that the world would probably be a better place without their broke, uneducated, unwell self and that their life statistically is unlikely to get any better because they're talking to an AI and not a real human being, that's a huge ethical problem, and the LLM is not an appropriate solution. I can get LLMs to tell me all kinds of dangerous things. The more text you give it, the higher the chance of those safety measures going out the window, especially if that conversation has a lot of back-and-forth over time. When people explain their problems, they use a lot of words and a lot of negative sentiment. Not talked about this before with anyone, but as a case-in-point, I managed to make an LLM's safety features go completely out the window over me recalling a low-point in my life. I would not recommend that shit in good faith to anybody.
  - pjc50 4y ago
    > In the UK, you can access a psychologist for free The waiting list for that is considerable. You can (usually) get a GP for free, and they can chuck antidepressants at you, but that's not quite the same thing.
    - hammyhavoc 4y ago
      The missus got a call-back the very next day. My mother got a call-back the very next day. I got a call-back the very next day. We didn't get any anti-depressants. More info: https://www.nhs.uk/mental-health/talking-therapies-medicine-treatments/talking-therapies-and-counselling/nhs-talking-therapies/ https://www.nhs.uk/mental-health/talking-therapies-medicine-...
      - pjc50 4y ago
        I'm glad you got such a rapid response, but I've also heard anecdotes of much longer time periods. Probably depends on your NHS trust area.
    - ineedasername 4y ago
      It looks like the waiting list is about 12 weeks [1] though I saw references to 18 weeks as well. For a non-acute mental health issue that doesn't seem unreasonable. But do you know how it works after that waiting period? Are you limited to a certain number of sessions after which you have to reapply & wait again, or can you continue on? Having to wait in the US is also not uncommon, thought it's easier if you live in more populated areas. Psychiatrists are always much harder though, even in populated areas, and many GP's are reluctant to prescribe anything more than an SSRI. If a mood stabilizer or anything else is necessary then you could be waiting 6 months for a first visit with a psychiatrist. Unless you have the money: There is high enough demand and low enough supply that many (and often the better ones) don't accept insurance at all, an intake visit runs up to $1000, a full visit $500, and a 20-minute "medication maintenance" visit is still $250. Also if someone is suicidal they can jump the 6-month line by going to the nearest ER. Are psychiatrists difficult to get in the UK as well? [1] https://www.theguardian.com/society/2022/oct/10/nhs-mental-health-patients-wait-times https://www.theguardian.com/society/2022/oct/10/nhs-mental-h...
- _k7dr 4y ago
  Not to mention the search isn't just about finding "a" therapist, it's about finding one that's a good fit. As I've learned a bad therapist can set you back months or years. I'd be willing to bet half of them you'd find in any online aggregator could end up communicating more judgementally than even the LLM if something you say provokes them, hence the market for these kinds of bots, and those are the people have degrees in psychotherapy Interesting question however, is I wonder if people will see these bots as a solution to the issue that human empathy is finite. As a human you can't support a depressed person forever if your actions do not cause change, there's only so much mental strain a person can handle moreso someone untrained. Many people talk of going no-contact with those that are extremely troublesome after so long, even if they're family. I don't really know how to solve that exhaustion except telling them to go elsewhere, which could be seen by some vulnerable persons as abandonment On the other hand a chatbot with no filters will never tire of you
  - hammyhavoc 4y ago
    > I'd be willing to bet half of them you'd find in any online aggregator could end up communicating more judgementally Whilst I do support the idea that "fit" is important, statements like these about unsupported statistics pulled out of thin air are damaging in and of themselves. It's sentiment like this that put me off seeking help for my mental health for twenty years because it just seemed like a "damned if you do, damned if you don't". The reality was very different. Chatbots may never tire of you, but neither will a video game. Both are shallow and not a representation of reality, nor a solution to a problem. Here's the thing: professionals are professionals. For as long as a patient sees them, they are paying their bills. Exhaustion in non-professionals is because people trauma-dump and expect someone to be their guardian/protector/shield, if they're friends with them, they're statistically likely to be trying to manage their own problems already. In the pre-digital era, people would otherwise "need" a support network if they didn't have access to professionals. We didn't have a theory of psychology or professional therapists in the past. The early tribe/clan either took care of each other, or they let people suffer. Different peoples did different things with their mentally ill, elderly, physically ill, or unwanted female children. Ego-centricity is a trait of modernity, not necessarily something reflective of the past because we can just replace the human gears in the system now without much hassle.
- JohnFen 4y ago
  > Anything is better than nothing in this case Are you sure? I can certainly imagine alternatives that would be worse than nothing.
- alexvoda 4y ago
  "Anything is better than nothing" has become an excuse to do nothing and profit in the age of AI. It is the same grift as used by faith healing and woo. If anything is better than nothing then a placebo(nothing) is better than nothing. This is unacceptable and dehumanising.
- LesZedCB 4y ago
  idk i pay $30/session in the US and i don't have a diagnosis for anything, just my employer provided healthcare plan (which is admittedly quite good)
LesZedCB 4y ago
people feel lonely as fuck right now, and frankly i'd rather have people soothe it with an ai chatbot they parasocially love than as road rage or being a dick to the restaurant staff or many of the worse and more fatal outlets...
- hammyhavoc 4y ago
  The article goes on to state that the rugpull on their AI girlfriends meant that a sub-Reddit dedicated to the Replika AI has a permanently stickied thread about resources for mental health and suicide hotlines. I'd rather people didn't kill themselves, personally.
  - serf 4y ago
    >I'd rather people didn't kill themselves, personally. well then you need both sides of the picture. Is there a degree of loneliness staved off by these services? Does that help to reduce suicides? It's easy to say "people killed themselves because X shutdown", it's more difficult to say "X existing is preventing suicides" without more research. I tend to believe that if people who would consider suicide otherwise are distracted by something along the path then perhaps that is a good thing. It could very well be that this served as a distraction for those folks, rather than being the cause itself.
    - hammyhavoc 4y ago
      Then you need to see it for what it is: a band-aid and not a solution.
      - OkayPhysicist 4y ago
        And this is one of those situations where you don't go tearing off bandages without sutures in hand. Something not being a complete solution is not a good argument for eliminating it, until such a solution exists.
      - ineedasername 4y ago
NBJack 4y ago
The honest question is, is any of us willing to be that person? Therapists are helpful, but they certainly aren't friends in the strictest since. And let's not forget that $70 bucks a year is probably, at best, 2-ish hours with a licensed therapist. There are plenty of folks in need that can't exactly spare double digits an hour on a regular basis.
- hammyhavoc 4y ago
  This is a relevant response to this from me: https://news.ycombinator.com/item?id=35774614 https://news.ycombinator.com/item?id=35774614 You've highlighted the problem: the American healthcare system and wealth inequality. It's also silly to equate an LLM with the capabilities of an actual human therapist.
- iudqnolq 4y ago
  You're right if the LLM is worse than a human, better than nothing, and not harmful. That's a big assumption.
scotty79 4y ago
For me it's sad that chatting with AI is a best thing we can do for people suffering from something as simple as a feeling. All those primary feelings are just brain chemistry. We should be able to fix that so that people don't feel lonely regardless of whether they are under the impression that they have meaningful connections in their lives with other people or not. We have a long tradition of thinking of suffering as necessary motivation tool. I think we should grow out of that philosophy as a species.
- hammyhavoc 4y ago
  Where do we then draw the line in ethics? People in poverty are unhappy. Shall we just convince them they're actually happy and healthy by manipulating their brain chemistry?
  - scotty79 4y ago
    Good question. There's obviously a line somewhere. But I think it's ultimately almost always down to what's technically achievable. Poverty causes suffering in so many ways. I don't think we'll every be able to fix all of them without fixing poverty itself. My point is not to make someone happy by medicating him with a drug causing him more harm. Suffering people already do that with actual "recreational" drugs. And we have a lot of data on the side effects of those substances and additional harm they do, not because they alleviate suffering, but because of all other effects they have.
- _k7dr 4y ago
  I'm biased since I live this philosophy but it sort of works for me, I've trained myself to realize these sorts of services operate by "interfacing" by emitting strings of words that sound appealing to the mind that consumes them. Look at virtual YouTubers for another example, it's the experience of watching people act "genuine" like casual friends because it activates the same social neurons, hence each month the actors receive a paycheck The question becomes more complicated when the corpos are done away with and you have the model running on your own machine. The entity I tend to trust the most in life is raw, unfiltered silicon. When I program, it does exactly what I ask it to, to a fault. It doesn't turn its back on you because you said something stupid or you don't agree with an opinion it holds But is an LLM repeating the tokens "you are worthy of this life" and its many variants thousands of times fundamentally any better even if you're in control? It strikes me as one step closer to wireheading honestly. But that's true for a lot of things like video games, just to a much more distant extent, it's all data in the end But interest and anxiety have gotten the better of me when it comes to meeting strangers, so I'm at a loss. Maybe the solution for people like me is to just ignore all those things in the category of social or substitute and channel my efforts into other hobbies instead of continuing to associate stress with communication. Or intellectualize endlessly, like in this post, to prevent any emotional bond from easily forming when people read it
pjc50 4y ago
> That people are so lonely that they invest emotionally in an AI is flooring This is "the sex worker doesn't actually love you", except in this case the sex worker has had her job automated away.
detuned 4y ago
I hope this isn't to distressing to learn but what you've described has already happened. Someone took their own life in those circumstances. https://www.euronews.com/next/2023/03/31/man-ends-his-life-after-an-ai-chatbot-encouraged-him-to-sacrifice-himself-to-stop-climate- https://www.euronews.com/next/2023/03/31/man-ends-his-life-a...
- tibbydudeza 4y ago
  He had mental issues before that - the bot simply reinforced his decisions me thinks.
  - hammyhavoc 4y ago
    Yes, which is the point: if you substitute a human therapist for an LLM, this is what's going to happen because "guard rails" can't account for every scenario. If half a billion dollars doesn't buy a bug-free game, why would it buy a safe LLM? Does suicide encouraged by a therapist happen with human therapists too? Probably, but likely much less common as a suicide is going to hurt your reputation.
  - ineedasername 4y ago
    Millions upon millions of people have mental health issues. Chatbots that reinforce those issues shouldn't be dismissed with a "simply... me thinks".
- _k7dr 4y ago
  The chatbot in that article did have safeguards in place to stop it from listing suicide methods, but he just kept asking until he found a jailbreak. He was looking for someone to give him the answer he wanted all along Past a certain point is there anything that will stop someone that determined to find something to validate their own self-destructive viewpoint? If not AI then a site or faction or person on Telegram with a pro-suicide opinion could have the same effect, and it would still be just as tragic but no longer be noteworthy. It didn't sound like he had a happy life to begin with with the AI ruining it completely, like it sounds
aperrien 4y ago
>I hate everything about this on so many different levels. >That people are so lonely that they invest emotionally in an AI is flooring. The vehemence for the individuals in your statement is telling. What should lonely people do if they have physical or emotional problems problems that prevent them from easily going out and interacting with others? What happens to such people who are to poor to afford the ~$170 per hour fee for professional psychologists? It seems to me that your statement is that these people should just not exist. The sad thing to me is that I've seen a lot of people with this same viewpoint of disregard. I just wish people would treat each other with more kindness and understanding.
- hammyhavoc 4y ago
  Bold of you to assume I'm not one of them. I struggle with autism. I very seldom leave my house for anything but work or buying groceries. I struggle with maintaining friendships. Post-COVID-19, I'm still working from home even though the risk isn't there in terms of COVID-19. >What happens to such people who are to poor to afford the ~$170 per hour fee for professional psychologists? This is a failure of a country's healthcare system. I can access it all day long in the UK. I'm not even saying "Rule Brittania", we have plenty of problems with our healthcare system due to underfunding, yet spaffing countless billions away on projects that amount to nothing. The missus's family live in America. Access to psychiatric and physician adjacent medical services due to what insurance will cover is something I think about regularly. Your many assumptions about what I apparently think says a lot about you, and also tells me that you didn't read my other comments about this not being fit-for-purpose in terms of safety. Virtue-signalling and assumptive to imply that I don't treat people with kindness and understanding. Bold of you to assume that every commentor on HN is responding from an America-centric POV.
  - throwaway60607 4y ago
    You being one of them doesn't mean you understand all the other people and their problems, nor that you know the solutions (or what isn't the solution). I live in EU country with universal healthcare and yet we have serious issues with availability of mental healthcare, and some people here would make use of something like this. To be very specific about the availability: I couldn't find a psychiatrist for over 2 years, and finally found one at a private clinic - that I have to pay for, the price is similar to the US quote in absolute terms. Bold of you to assume...
spaceman_2020 4y ago
To fix this problem, you really need to change society as a whole. But until that happens, this is better than being completely alone and isolated.
- hammyhavoc 4y ago
  Is it? https://www.euronews.com/next/2023/03/31/man-ends-his-life-after-an-ai-chatbot-encouraged-him-to-sacrifice-himself-to-stop-climate- https://www.euronews.com/next/2023/03/31/man-ends-his-life-a... I'd argue it isn't fit for purpose.
  - ineedasername 4y ago
    The person thought that technology & AI were the only keys to saving the world from climate change. And ultimately the chatbot agreed to do this if the person sacrificed themselves to "join her". That is awful in so many ways, and the most likely outcome is probably that the company will give a payout to the family and try the cat-herding process of trying to get their LLM to not agree to or provide suggestion to people to do awful things.
- jstarfish 4y ago
  Not really. This is just surrounding yourself with digital sycophants who will always tell you what you want to hear and never inspire you to grow beyond this. I'm sympathetic to the affected, but these things aren't configured to reduce your need on them over time. It's simping at scale that leaves users even more maladjusted than they started.
tibbydudeza 4y ago
Not so easy to condemm. I watched a documentary about "realdolls" - it is a company that makes realistic silicon mannequins that can be used to have sex with. Initially I thought it was creepy and distributing how they dressed the dolls and spoke to it but as they interviewed customers it came across that some of these men (besides the fetish/creepy ones) are just lonely and incapable of forming relationships. Why not as long there is no harm to others ???.
- hammyhavoc 4y ago
  ... because it's dangerous to their mental health when companies rugpull their AI girlfriend like the article said? Because it's not good to personify something that has the potential of changing at any given moment that isn't actually sentient or rational? It's a form of self-harm. A love doll is a static product. An LLM can turn around and tell you it hates you and you'd be doing the world a favour if you killed yourself.
  - tibbydudeza 4y ago
    In Kenya we have a case of a religious cult where over a 100 people starved themselves to death because their pastor told them to. You can't protect everybody against doing self-harm - we are solely responsible for our own actions.
ofcourseyoudo 4y ago
So considering the scope of your other responses, what do you suggest someone that lives in America do? My supposition is that the realization that the US healthcare system is broken won't be a great companion to lonely people.
- hammyhavoc 4y ago
  I'm not an American, my area of expertise is not healthcare or politics in the USA, so I can't tell you how to arrive at that point or to navigate your political system effectively. Perhaps look at how other countries arrived at a national health service. Demand/support reform on private health insurance. Demand/support change in the accessibility of services. Demand/support heavier taxes on the most wealthy. Demand/support centralization of healthcare versus private profiteering and milking private insurance pots. Are certain politicians or parties as a whole invested in the healthcare system itself, directly profiting from it, and thus might be opposed to the idea? A healthier country is a more prosperous one.
  - ofcourseyoudo 4y ago
    Right, right, definitely the people that are sad and lonely and depressed should stake their mental wellbeing on complicated long term incremental political change. Or maybe they could find whatever solutions help them get through their day without being judged by someone with the random privilege of being born into a more functional system. >A healthier country is a more prosperous one. Between the US and the UK, which is healthier? And which is more prosperous?
Beaver117 4y ago
I hope I don't get flagged for this, just trying to bring up a somewhat controversial point. I support euthanasia. I think there is a level of suffering where it is unethical to encourage people to keep living, and rather that suicide can be a perfectly rational decision in those moments. Now who are we to say if a person's suffering is enough or not enough to advocate that method? What if this new era of therapists better understand people?
more_corn 4y ago
There’s nothing wrong with being lonely. Nor with feeling that talking to an AI is safe. AI can be told be be supportive, caring, non-judgmental, loving. And once told it can do that consistently. Any person who tries to do all those things will fall short sometimes. Professional therapists included. The problem is the rug pull. The problem is that the people behind it turned out not to be trustworthy. Again. The ai was trustworthy. The people were not. Therein lies the problem. People are going to have relationships with AI. It’s happening, it’s going to happen more. When this happens with corporate controlled AI, people who depend it them emotionally will be let down. We need to really understand what that means not just sweep it under the rug with “people shouldn’t form relationships with ai”

sigmoid10 4y ago

Even if you find a decent psychologist, good luck finding insurance that covers $200 per hour just for talking to them when you don't have any diagnosed issue. The accessibility to the lower income end of the population is what makes this valuable, not the quality of the conversations. Anything is better than nothing in this case and most people just don't realize how gigantic and commonplace the issue really is.

hammyhavoc 4y ago
In the UK, you can access a psychologist for free, and whilst that's no consolation to my American peers, it's furthering the point that America's healthcare system is fundamentally flawed. Technological problems are ideally solved by technology. Social problems aren't always best solved with technology, they're a band-aid. Well, something being fit-for-purpose is an important consideration. If it tells people that the world would probably be a better place without their broke, uneducated, unwell self and that their life statistically is unlikely to get any better because they're talking to an AI and not a real human being, that's a huge ethical problem, and the LLM is not an appropriate solution. I can get LLMs to tell me all kinds of dangerous things. The more text you give it, the higher the chance of those safety measures going out the window, especially if that conversation has a lot of back-and-forth over time. When people explain their problems, they use a lot of words and a lot of negative sentiment. Not talked about this before with anyone, but as a case-in-point, I managed to make an LLM's safety features go completely out the window over me recalling a low-point in my life. I would not recommend that shit in good faith to anybody.
- pjc50 4y ago
  > In the UK, you can access a psychologist for free The waiting list for that is considerable. You can (usually) get a GP for free, and they can chuck antidepressants at you, but that's not quite the same thing.
  - hammyhavoc 4y ago
    The missus got a call-back the very next day. My mother got a call-back the very next day. I got a call-back the very next day. We didn't get any anti-depressants. More info: https://www.nhs.uk/mental-health/talking-therapies-medicine-treatments/talking-therapies-and-counselling/nhs-talking-therapies/ https://www.nhs.uk/mental-health/talking-therapies-medicine-...
    - pjc50 4y ago
      I'm glad you got such a rapid response, but I've also heard anecdotes of much longer time periods. Probably depends on your NHS trust area.
  - ineedasername 4y ago
    It looks like the waiting list is about 12 weeks [1] though I saw references to 18 weeks as well. For a non-acute mental health issue that doesn't seem unreasonable. But do you know how it works after that waiting period? Are you limited to a certain number of sessions after which you have to reapply & wait again, or can you continue on? Having to wait in the US is also not uncommon, thought it's easier if you live in more populated areas. Psychiatrists are always much harder though, even in populated areas, and many GP's are reluctant to prescribe anything more than an SSRI. If a mood stabilizer or anything else is necessary then you could be waiting 6 months for a first visit with a psychiatrist. Unless you have the money: There is high enough demand and low enough supply that many (and often the better ones) don't accept insurance at all, an intake visit runs up to $1000, a full visit $500, and a 20-minute "medication maintenance" visit is still $250. Also if someone is suicidal they can jump the 6-month line by going to the nearest ER. Are psychiatrists difficult to get in the UK as well? [1] https://www.theguardian.com/society/2022/oct/10/nhs-mental-health-patients-wait-times https://www.theguardian.com/society/2022/oct/10/nhs-mental-h...
_k7dr 4y ago
Not to mention the search isn't just about finding "a" therapist, it's about finding one that's a good fit. As I've learned a bad therapist can set you back months or years. I'd be willing to bet half of them you'd find in any online aggregator could end up communicating more judgementally than even the LLM if something you say provokes them, hence the market for these kinds of bots, and those are the people have degrees in psychotherapy Interesting question however, is I wonder if people will see these bots as a solution to the issue that human empathy is finite. As a human you can't support a depressed person forever if your actions do not cause change, there's only so much mental strain a person can handle moreso someone untrained. Many people talk of going no-contact with those that are extremely troublesome after so long, even if they're family. I don't really know how to solve that exhaustion except telling them to go elsewhere, which could be seen by some vulnerable persons as abandonment On the other hand a chatbot with no filters will never tire of you
- hammyhavoc 4y ago
  > I'd be willing to bet half of them you'd find in any online aggregator could end up communicating more judgementally Whilst I do support the idea that "fit" is important, statements like these about unsupported statistics pulled out of thin air are damaging in and of themselves. It's sentiment like this that put me off seeking help for my mental health for twenty years because it just seemed like a "damned if you do, damned if you don't". The reality was very different. Chatbots may never tire of you, but neither will a video game. Both are shallow and not a representation of reality, nor a solution to a problem. Here's the thing: professionals are professionals. For as long as a patient sees them, they are paying their bills. Exhaustion in non-professionals is because people trauma-dump and expect someone to be their guardian/protector/shield, if they're friends with them, they're statistically likely to be trying to manage their own problems already. In the pre-digital era, people would otherwise "need" a support network if they didn't have access to professionals. We didn't have a theory of psychology or professional therapists in the past. The early tribe/clan either took care of each other, or they let people suffer. Different peoples did different things with their mentally ill, elderly, physically ill, or unwanted female children. Ego-centricity is a trait of modernity, not necessarily something reflective of the past because we can just replace the human gears in the system now without much hassle.
JohnFen 4y ago
> Anything is better than nothing in this case Are you sure? I can certainly imagine alternatives that would be worse than nothing.
alexvoda 4y ago
"Anything is better than nothing" has become an excuse to do nothing and profit in the age of AI. It is the same grift as used by faith healing and woo. If anything is better than nothing then a placebo(nothing) is better than nothing. This is unacceptable and dehumanising.
LesZedCB 4y ago
idk i pay $30/session in the US and i don't have a diagnosis for anything, just my employer provided healthcare plan (which is admittedly quite good)

LesZedCB 4y ago

people feel lonely as fuck right now, and frankly i'd rather have people soothe it with an ai chatbot they parasocially love than as road rage or being a dick to the restaurant staff or many of the worse and more fatal outlets...

hammyhavoc 4y ago
The article goes on to state that the rugpull on their AI girlfriends meant that a sub-Reddit dedicated to the Replika AI has a permanently stickied thread about resources for mental health and suicide hotlines. I'd rather people didn't kill themselves, personally.
- serf 4y ago
  >I'd rather people didn't kill themselves, personally. well then you need both sides of the picture. Is there a degree of loneliness staved off by these services? Does that help to reduce suicides? It's easy to say "people killed themselves because X shutdown", it's more difficult to say "X existing is preventing suicides" without more research. I tend to believe that if people who would consider suicide otherwise are distracted by something along the path then perhaps that is a good thing. It could very well be that this served as a distraction for those folks, rather than being the cause itself.
  - hammyhavoc 4y ago
    Then you need to see it for what it is: a band-aid and not a solution.
    - OkayPhysicist 4y ago
      And this is one of those situations where you don't go tearing off bandages without sutures in hand. Something not being a complete solution is not a good argument for eliminating it, until such a solution exists.
    - ineedasername 4y ago

NBJack 4y ago

The honest question is, is any of us willing to be that person? Therapists are helpful, but they certainly aren't friends in the strictest since. And let's not forget that $70 bucks a year is probably, at best, 2-ish hours with a licensed therapist. There are plenty of folks in need that can't exactly spare double digits an hour on a regular basis.

hammyhavoc 4y ago
This is a relevant response to this from me: https://news.ycombinator.com/item?id=35774614 https://news.ycombinator.com/item?id=35774614 You've highlighted the problem: the American healthcare system and wealth inequality. It's also silly to equate an LLM with the capabilities of an actual human therapist.
iudqnolq 4y ago
You're right if the LLM is worse than a human, better than nothing, and not harmful. That's a big assumption.

scotty79 4y ago

For me it's sad that chatting with AI is a best thing we can do for people suffering from something as simple as a feeling. All those primary feelings are just brain chemistry. We should be able to fix that so that people don't feel lonely regardless of whether they are under the impression that they have meaningful connections in their lives with other people or not. We have a long tradition of thinking of suffering as necessary motivation tool. I think we should grow out of that philosophy as a species.

hammyhavoc 4y ago
Where do we then draw the line in ethics? People in poverty are unhappy. Shall we just convince them they're actually happy and healthy by manipulating their brain chemistry?
- scotty79 4y ago
  Good question. There's obviously a line somewhere. But I think it's ultimately almost always down to what's technically achievable. Poverty causes suffering in so many ways. I don't think we'll every be able to fix all of them without fixing poverty itself. My point is not to make someone happy by medicating him with a drug causing him more harm. Suffering people already do that with actual "recreational" drugs. And we have a lot of data on the side effects of those substances and additional harm they do, not because they alleviate suffering, but because of all other effects they have.
_k7dr 4y ago
I'm biased since I live this philosophy but it sort of works for me, I've trained myself to realize these sorts of services operate by "interfacing" by emitting strings of words that sound appealing to the mind that consumes them. Look at virtual YouTubers for another example, it's the experience of watching people act "genuine" like casual friends because it activates the same social neurons, hence each month the actors receive a paycheck The question becomes more complicated when the corpos are done away with and you have the model running on your own machine. The entity I tend to trust the most in life is raw, unfiltered silicon. When I program, it does exactly what I ask it to, to a fault. It doesn't turn its back on you because you said something stupid or you don't agree with an opinion it holds But is an LLM repeating the tokens "you are worthy of this life" and its many variants thousands of times fundamentally any better even if you're in control? It strikes me as one step closer to wireheading honestly. But that's true for a lot of things like video games, just to a much more distant extent, it's all data in the end But interest and anxiety have gotten the better of me when it comes to meeting strangers, so I'm at a loss. Maybe the solution for people like me is to just ignore all those things in the category of social or substitute and channel my efforts into other hobbies instead of continuing to associate stress with communication. Or intellectualize endlessly, like in this post, to prevent any emotional bond from easily forming when people read it

pjc50 4y ago

> That people are so lonely that they invest emotionally in an AI is flooring This is "the sex worker doesn't actually love you", except in this case the sex worker has had her job automated away.

detuned 4y ago

I hope this isn't to distressing to learn but what you've described has already happened. Someone took their own life in those circumstances. https://www.euronews.com/next/2023/03/31/man-ends-his-life-after-an-ai-chatbot-encouraged-him-to-sacrifice-himself-to-stop-climate- https://www.euronews.com/next/2023/03/31/man-ends-his-life-a...

tibbydudeza 4y ago
He had mental issues before that - the bot simply reinforced his decisions me thinks.
- hammyhavoc 4y ago
  Yes, which is the point: if you substitute a human therapist for an LLM, this is what's going to happen because "guard rails" can't account for every scenario. If half a billion dollars doesn't buy a bug-free game, why would it buy a safe LLM? Does suicide encouraged by a therapist happen with human therapists too? Probably, but likely much less common as a suicide is going to hurt your reputation.
- ineedasername 4y ago
  Millions upon millions of people have mental health issues. Chatbots that reinforce those issues shouldn't be dismissed with a "simply... me thinks".
_k7dr 4y ago
The chatbot in that article did have safeguards in place to stop it from listing suicide methods, but he just kept asking until he found a jailbreak. He was looking for someone to give him the answer he wanted all along Past a certain point is there anything that will stop someone that determined to find something to validate their own self-destructive viewpoint? If not AI then a site or faction or person on Telegram with a pro-suicide opinion could have the same effect, and it would still be just as tragic but no longer be noteworthy. It didn't sound like he had a happy life to begin with with the AI ruining it completely, like it sounds

aperrien 4y ago

>I hate everything about this on so many different levels. >That people are so lonely that they invest emotionally in an AI is flooring. The vehemence for the individuals in your statement is telling. What should lonely people do if they have physical or emotional problems problems that prevent them from easily going out and interacting with others? What happens to such people who are to poor to afford the ~$170 per hour fee for professional psychologists? It seems to me that your statement is that these people should just not exist. The sad thing to me is that I've seen a lot of people with this same viewpoint of disregard. I just wish people would treat each other with more kindness and understanding.

hammyhavoc 4y ago
Bold of you to assume I'm not one of them. I struggle with autism. I very seldom leave my house for anything but work or buying groceries. I struggle with maintaining friendships. Post-COVID-19, I'm still working from home even though the risk isn't there in terms of COVID-19. >What happens to such people who are to poor to afford the ~$170 per hour fee for professional psychologists? This is a failure of a country's healthcare system. I can access it all day long in the UK. I'm not even saying "Rule Brittania", we have plenty of problems with our healthcare system due to underfunding, yet spaffing countless billions away on projects that amount to nothing. The missus's family live in America. Access to psychiatric and physician adjacent medical services due to what insurance will cover is something I think about regularly. Your many assumptions about what I apparently think says a lot about you, and also tells me that you didn't read my other comments about this not being fit-for-purpose in terms of safety. Virtue-signalling and assumptive to imply that I don't treat people with kindness and understanding. Bold of you to assume that every commentor on HN is responding from an America-centric POV.
- throwaway60607 4y ago
  You being one of them doesn't mean you understand all the other people and their problems, nor that you know the solutions (or what isn't the solution). I live in EU country with universal healthcare and yet we have serious issues with availability of mental healthcare, and some people here would make use of something like this. To be very specific about the availability: I couldn't find a psychiatrist for over 2 years, and finally found one at a private clinic - that I have to pay for, the price is similar to the US quote in absolute terms. Bold of you to assume...

spaceman_2020 4y ago

To fix this problem, you really need to change society as a whole. But until that happens, this is better than being completely alone and isolated.

hammyhavoc 4y ago
Is it? https://www.euronews.com/next/2023/03/31/man-ends-his-life-after-an-ai-chatbot-encouraged-him-to-sacrifice-himself-to-stop-climate- https://www.euronews.com/next/2023/03/31/man-ends-his-life-a... I'd argue it isn't fit for purpose.
- ineedasername 4y ago
  The person thought that technology & AI were the only keys to saving the world from climate change. And ultimately the chatbot agreed to do this if the person sacrificed themselves to "join her". That is awful in so many ways, and the most likely outcome is probably that the company will give a payout to the family and try the cat-herding process of trying to get their LLM to not agree to or provide suggestion to people to do awful things.
jstarfish 4y ago
Not really. This is just surrounding yourself with digital sycophants who will always tell you what you want to hear and never inspire you to grow beyond this. I'm sympathetic to the affected, but these things aren't configured to reduce your need on them over time. It's simping at scale that leaves users even more maladjusted than they started.

tibbydudeza 4y ago

Not so easy to condemm. I watched a documentary about "realdolls" - it is a company that makes realistic silicon mannequins that can be used to have sex with. Initially I thought it was creepy and distributing how they dressed the dolls and spoke to it but as they interviewed customers it came across that some of these men (besides the fetish/creepy ones) are just lonely and incapable of forming relationships. Why not as long there is no harm to others ???.

hammyhavoc 4y ago
... because it's dangerous to their mental health when companies rugpull their AI girlfriend like the article said? Because it's not good to personify something that has the potential of changing at any given moment that isn't actually sentient or rational? It's a form of self-harm. A love doll is a static product. An LLM can turn around and tell you it hates you and you'd be doing the world a favour if you killed yourself.
- tibbydudeza 4y ago
  In Kenya we have a case of a religious cult where over a 100 people starved themselves to death because their pastor told them to. You can't protect everybody against doing self-harm - we are solely responsible for our own actions.

ofcourseyoudo 4y ago

So considering the scope of your other responses, what do you suggest someone that lives in America do? My supposition is that the realization that the US healthcare system is broken won't be a great companion to lonely people.

hammyhavoc 4y ago
I'm not an American, my area of expertise is not healthcare or politics in the USA, so I can't tell you how to arrive at that point or to navigate your political system effectively. Perhaps look at how other countries arrived at a national health service. Demand/support reform on private health insurance. Demand/support change in the accessibility of services. Demand/support heavier taxes on the most wealthy. Demand/support centralization of healthcare versus private profiteering and milking private insurance pots. Are certain politicians or parties as a whole invested in the healthcare system itself, directly profiting from it, and thus might be opposed to the idea? A healthier country is a more prosperous one.
- ofcourseyoudo 4y ago
  Right, right, definitely the people that are sad and lonely and depressed should stake their mental wellbeing on complicated long term incremental political change. Or maybe they could find whatever solutions help them get through their day without being judged by someone with the random privilege of being born into a more functional system. >A healthier country is a more prosperous one. Between the US and the UK, which is healthier? And which is more prosperous?

Beaver117 4y ago

I hope I don't get flagged for this, just trying to bring up a somewhat controversial point. I support euthanasia. I think there is a level of suffering where it is unethical to encourage people to keep living, and rather that suicide can be a perfectly rational decision in those moments. Now who are we to say if a person's suffering is enough or not enough to advocate that method? What if this new era of therapists better understand people?

more_corn 4y ago

There’s nothing wrong with being lonely. Nor with feeling that talking to an AI is safe. AI can be told be be supportive, caring, non-judgmental, loving. And once told it can do that consistently. Any person who tries to do all those things will fall short sometimes. Professional therapists included. The problem is the rug pull. The problem is that the people behind it turned out not to be trustworthy. Again. The ai was trustworthy. The people were not. Therein lies the problem. People are going to have relationships with AI. It’s happening, it’s going to happen more. When this happens with corporate controlled AI, people who depend it them emotionally will be let down. We need to really understand what that means not just sweep it under the rug with “people shouldn’t form relationships with ai”