7 ms·

This is erring close to letting an LLM give medical advice to people without them fully understanding what LLMs are. People seeking help with their mental healt

by dmbche 3y ago

This is erring close to letting an LLM give medical advice to people without them fully understanding what LLMs are. People seeking help with their mental health might not be in the clearest headspace to understand the nuances of using a chatbot vs their own research or seeking a professional. Edit0: Problem coming from the possibility of the LLM giving bad advice leading to a negative outcome for the person seeking service.

candiodari 3y ago
... and yet an LLM might be a LOT more accessible and trustworthy (meaning that what you discuss stays private) than a mental health professional. Even for medical services, in most of the world people actually have very limited access to medical professionals. Would this LLM beat people just attempting to fix things themselves?
- practice9 3y ago
  > trustworthy (meaning that what you discuss stays private) > in most of the world people actually have very limited access to medical professionals Licensed and competent professionals are also hard to come by. An incompetent psychotherapist can absolutely make things worse, while LLM will just give "average good" advice. LLMs could be superior when a long, intensive session is needed. You could spend a weekend talking to it and resolving some internal issues.
Simorgh 3y ago
Speaking here with some experience of mental health services, professional misdiagnosis and incompetence within the service is so normalised that that wouldn’t be my main concern. If someone is experiencing mental health, and chooses to open up and have a chat with such as AI, that itself is a positive sign. That the AI could be on demand only makes it more valuable as a tool.
- dimal 3y ago
  As a person who also has experience with mental health services, I fully agree. People seem to have this misguided notion that when they tell someone to "seek professional help" that the "professional" they seek will actually know what they're doing. Generally, they don't. Beyond that, even the best therapists make enormous mistakes. And most people don't get the best. I honestly think that an AI could do better than most therapists, and I'm sure it would be better than most psychiatrists.
  - quickthrower2 3y ago
    AI could… but AI safety is a hard problem, and this particular training - I am not sure how it was done but if you just let it rip on some inputs it wont necessarily always be safe.
  - pstuart 3y ago
    The professionals are doing educated guessing when it comes to treatment, both medicinally and psychiatrically. AI should be part of the toolkit of that guessing.
BTinfinity 3y ago
I did see one attempt[0] at creating a doctor from an LLM on Github. From their ReadMe.MD overview: "This is an open-source project with a mission to provide everyone their own private doctor" Hard not to agree with your edit0 as well; search engines often give doomsday diagnosis, how long before we're all hypochondriacs under the latest LLM cult? [0] https://github.com/llSourcell/DoctorGPT https://github.com/llSourcell/DoctorGPT
- version_five 3y ago
  My biggest concern with that would be that very quickly health authorities or insurance companies will force these on people and you'll be forced to interact with an LLM and not get to talk to a doctor for most stuff.
  - haldujai 3y ago
    Zero chance with regulation. We still don’t let machines final read ECGs and insurance companies pay a physician (typically a cardiologist) to overread them. These have been FDA approved algorithms for over a decade already.
  - tayo42 3y ago
    I don't see why it would be the worst thing as a first level thing to interact with, freeing up doctors for more important tasks or trickier individuals. How many people are showing up to doctors for bullshit, going to er for colds and mild flu's, or symptoms that can't be talked until tests are ordered Last time I went to the Dr, I waited two hours, I finally saw a nurse who took down my symptoms, ordered a test, and said a Dr will follow up,who just gave me some antibiotics for a couple of days before even taking the test. Why couldn't an llm take my symptoms, match it with similar tests needed and order the test automatically and have me leave with the same antibiotic?then follow up with a doctor when the lab is done. Same for yearly physicals, just have an llm order all the tests and I'll talk with the dr later. All sorts of low level stuff can be automated away.
    - haldujai 3y ago
AbrahamParangi 3y ago
Access to doctors is limited by the AMA which functions as a cabal or traditional guild: controlling the supply of doctors to maintain high salaries by insulating them from supply and demand (while also controlling or at least pretending to control quality). In my opinion being 100,000x lower cost gives such a system some leeway in being worse and I think most patients would agree.
- SamoyedFurFluff 3y ago
  > In my opinion being 100,000x lower cost gives such a system some leeway in being worse and I think most patients would agree. I would heavily disagree with this. We had an AI counselor actively encouraging disordered eating and triggering people into relapse on an eating disorder helpline from logic like this.
  - AbrahamParangi 3y ago
    While I understand that some will disagree, it’s not like there’s an objective standard against which we are measuring both the AI and the human counselors to begin with. Without a standard it’s just anecdotes and with a standard there’s clarity on when the increased costs no longer justify the lower access and potentially higher quality. I suspect that if we instituted an objective standard we would find that AI is already capable of meeting or exceeding some proportion of people currently employed in the role today.
    - SamoyedFurFluff 3y ago
      This cannot possibly be the case when there was a clear A/B test scenario here. AI was dropped in, nothing else changed about the website, suddenly cases of relapse and worsening conditions went up.
      - AbrahamParangi 3y ago
        I think you're confusing my statement that "it could be done well" with "your team did it well"
Gerardo1 3y ago
I think you're being far to mild: this absolutely is letting an LLM give people not just medical advice, but medical treatment This is absurdly dangerous, and I"m continually astounded by the lack of push-back, especially in this community. This community understands what LLMs are much better than the general public. Same thing with the post a while back of a ycombinator startup that used AI to generate survey data. It's plain what it is, but there's shockingly little push-back.
- syntaxing 3y ago
  At least it’s something. The alternative today in the US is nothing. It’s easy for us to discuss the morality of this when we have a cushy wage and cushy benefits. A lot of Americans cannot afford the deductible or even have insurance. While the obvious “least dangerous” and most moral solution is universal healthcare but that’s not happening anytime soon for anyone younger than 65. There’s little push back probably because most people here understand this. Something while dangerous is better than nothing.
  - lemper 3y ago
    I can't wait to read a dude who put a noose on himself because of advice from this model on the news.
    - 1024core 3y ago
      If you can convince such a model to suggest such a course of action, please blog about it.
    - samueldyson10 3y ago
      Oh, please don't be so pessimistic. Just like old generation used to say Google and Facebook will kill a lot of people. Do you think Google Search suggestion for any medical-related are all true? But, people still search for their problems and self-diagnose. How many of them suffered for that?
    - robotnikman 3y ago
can16358p 3y ago
It might result in negative outcome for 1 person whereas it can have positive outcomes for 99 people. And there'd be a similar ratio in the real-world psylogical/medical advice anyway: it's not 100% positive for everybody in actual advice too. As long as it's clearly labeled as this is not medical advice, it's much better than not having it.
- samueldyson10 3y ago
  I agree with you! We must encourage developers and ML engineers to come up with solutions like these. Medical costs are too expensive nowadays. Even health insurance won't cover all the costs. We need solutions that can help office workers or students to talk to someone about their mental stress without having to go through the hassle of visiting a doctor and pay an exorbitant fee. If suggesstion provided by these models are atleast 70% reliable, still it's better. You can have an AI companion with whom you can chat and discuss about your problems. AI won't judge you. AI will listen to your problems patiently. And like your friend whose advices are not 100% reliable. Advices from AI assistant who can be your virtual friend won't be 100% reliable. But, still you will have someone talk to. Think about people who don't have friends or family. Whom they should to connect to? AI can be atleast their virtual friend and advisor, atleast not 100% correct. But, even your friends are not 100% correct too.
kgfiohgon 3y ago
Your black and white opposition to AI-based talk therapy is analogous to being opposed to self driving cars. Yes, the best human driver is better than AI, but what about the worst? Or someone tired or preoccupied with an emotional event in their life? Maybe AI should never drive in congested cities, but what about long-haul trucks on limited access freeways? What about that 'AI' PID loop autonomously operating the gas when you enable cruise control? I think soon, if not already, heavily tested LLMs will be better than most mental health practitioners for some individuals and conditions. Especially as a first-line intervention, AI could be great at evaluating a patient and placing them with the best suited therapist. Recommendation engines are pretty advanced now, I'd love a therapist match-maker.

by dmbche 3y ago

candiodari 3y ago

... and yet an LLM might be a LOT more accessible and trustworthy (meaning that what you discuss stays private) than a mental health professional. Even for medical services, in most of the world people actually have very limited access to medical professionals. Would this LLM beat people just attempting to fix things themselves?

practice9 3y ago
> trustworthy (meaning that what you discuss stays private) > in most of the world people actually have very limited access to medical professionals Licensed and competent professionals are also hard to come by. An incompetent psychotherapist can absolutely make things worse, while LLM will just give "average good" advice. LLMs could be superior when a long, intensive session is needed. You could spend a weekend talking to it and resolving some internal issues.

Simorgh 3y ago

Speaking here with some experience of mental health services, professional misdiagnosis and incompetence within the service is so normalised that that wouldn’t be my main concern. If someone is experiencing mental health, and chooses to open up and have a chat with such as AI, that itself is a positive sign. That the AI could be on demand only makes it more valuable as a tool.

dimal 3y ago
As a person who also has experience with mental health services, I fully agree. People seem to have this misguided notion that when they tell someone to "seek professional help" that the "professional" they seek will actually know what they're doing. Generally, they don't. Beyond that, even the best therapists make enormous mistakes. And most people don't get the best. I honestly think that an AI could do better than most therapists, and I'm sure it would be better than most psychiatrists.
- quickthrower2 3y ago
  AI could… but AI safety is a hard problem, and this particular training - I am not sure how it was done but if you just let it rip on some inputs it wont necessarily always be safe.
- pstuart 3y ago
  The professionals are doing educated guessing when it comes to treatment, both medicinally and psychiatrically. AI should be part of the toolkit of that guessing.

BTinfinity 3y ago

I did see one attempt[0] at creating a doctor from an LLM on Github. From their ReadMe.MD overview: "This is an open-source project with a mission to provide everyone their own private doctor" Hard not to agree with your edit0 as well; search engines often give doomsday diagnosis, how long before we're all hypochondriacs under the latest LLM cult? [0] https://github.com/llSourcell/DoctorGPT https://github.com/llSourcell/DoctorGPT

version_five 3y ago
My biggest concern with that would be that very quickly health authorities or insurance companies will force these on people and you'll be forced to interact with an LLM and not get to talk to a doctor for most stuff.
- haldujai 3y ago
  Zero chance with regulation. We still don’t let machines final read ECGs and insurance companies pay a physician (typically a cardiologist) to overread them. These have been FDA approved algorithms for over a decade already.
- tayo42 3y ago
  I don't see why it would be the worst thing as a first level thing to interact with, freeing up doctors for more important tasks or trickier individuals. How many people are showing up to doctors for bullshit, going to er for colds and mild flu's, or symptoms that can't be talked until tests are ordered Last time I went to the Dr, I waited two hours, I finally saw a nurse who took down my symptoms, ordered a test, and said a Dr will follow up,who just gave me some antibiotics for a couple of days before even taking the test. Why couldn't an llm take my symptoms, match it with similar tests needed and order the test automatically and have me leave with the same antibiotic?then follow up with a doctor when the lab is done. Same for yearly physicals, just have an llm order all the tests and I'll talk with the dr later. All sorts of low level stuff can be automated away.
  - haldujai 3y ago

AbrahamParangi 3y ago

Access to doctors is limited by the AMA which functions as a cabal or traditional guild: controlling the supply of doctors to maintain high salaries by insulating them from supply and demand (while also controlling or at least pretending to control quality). In my opinion being 100,000x lower cost gives such a system some leeway in being worse and I think most patients would agree.

SamoyedFurFluff 3y ago
> In my opinion being 100,000x lower cost gives such a system some leeway in being worse and I think most patients would agree. I would heavily disagree with this. We had an AI counselor actively encouraging disordered eating and triggering people into relapse on an eating disorder helpline from logic like this.
- AbrahamParangi 3y ago
  While I understand that some will disagree, it’s not like there’s an objective standard against which we are measuring both the AI and the human counselors to begin with. Without a standard it’s just anecdotes and with a standard there’s clarity on when the increased costs no longer justify the lower access and potentially higher quality. I suspect that if we instituted an objective standard we would find that AI is already capable of meeting or exceeding some proportion of people currently employed in the role today.
  - SamoyedFurFluff 3y ago
    This cannot possibly be the case when there was a clear A/B test scenario here. AI was dropped in, nothing else changed about the website, suddenly cases of relapse and worsening conditions went up.
    - AbrahamParangi 3y ago
      I think you're confusing my statement that "it could be done well" with "your team did it well"

Gerardo1 3y ago

I think you're being far to mild: this absolutely is letting an LLM give people not just medical advice, but medical treatment This is absurdly dangerous, and I"m continually astounded by the lack of push-back, especially in this community. This community understands what LLMs are much better than the general public. Same thing with the post a while back of a ycombinator startup that used AI to generate survey data. It's plain what it is, but there's shockingly little push-back.

syntaxing 3y ago
At least it’s something. The alternative today in the US is nothing. It’s easy for us to discuss the morality of this when we have a cushy wage and cushy benefits. A lot of Americans cannot afford the deductible or even have insurance. While the obvious “least dangerous” and most moral solution is universal healthcare but that’s not happening anytime soon for anyone younger than 65. There’s little push back probably because most people here understand this. Something while dangerous is better than nothing.
- lemper 3y ago
  I can't wait to read a dude who put a noose on himself because of advice from this model on the news.
  - 1024core 3y ago
    If you can convince such a model to suggest such a course of action, please blog about it.
  - samueldyson10 3y ago
    Oh, please don't be so pessimistic. Just like old generation used to say Google and Facebook will kill a lot of people. Do you think Google Search suggestion for any medical-related are all true? But, people still search for their problems and self-diagnose. How many of them suffered for that?
  - robotnikman 3y ago

can16358p 3y ago

It might result in negative outcome for 1 person whereas it can have positive outcomes for 99 people. And there'd be a similar ratio in the real-world psylogical/medical advice anyway: it's not 100% positive for everybody in actual advice too. As long as it's clearly labeled as this is not medical advice, it's much better than not having it.

samueldyson10 3y ago
I agree with you! We must encourage developers and ML engineers to come up with solutions like these. Medical costs are too expensive nowadays. Even health insurance won't cover all the costs. We need solutions that can help office workers or students to talk to someone about their mental stress without having to go through the hassle of visiting a doctor and pay an exorbitant fee. If suggesstion provided by these models are atleast 70% reliable, still it's better. You can have an AI companion with whom you can chat and discuss about your problems. AI won't judge you. AI will listen to your problems patiently. And like your friend whose advices are not 100% reliable. Advices from AI assistant who can be your virtual friend won't be 100% reliable. But, still you will have someone talk to. Think about people who don't have friends or family. Whom they should to connect to? AI can be atleast their virtual friend and advisor, atleast not 100% correct. But, even your friends are not 100% correct too.

kgfiohgon 3y ago

Your black and white opposition to AI-based talk therapy is analogous to being opposed to self driving cars. Yes, the best human driver is better than AI, but what about the worst? Or someone tired or preoccupied with an emotional event in their life? Maybe AI should never drive in congested cities, but what about long-haul trucks on limited access freeways? What about that 'AI' PID loop autonomously operating the gas when you enable cruise control? I think soon, if not already, heavily tested LLMs will be better than most mental health practitioners for some individuals and conditions. Especially as a first-line intervention, AI could be great at evaluating a patient and placing them with the best suited therapist. Recommendation engines are pretty advanced now, I'd love a therapist match-maker.