6 ms·

If we’re comparing worst-case scenarios: ChatGPT might confidently hallucinate a very valid-sounding explanation that convinces the patient they have a rare dis

If we’re comparing worst-case scenarios: ChatGPT might confidently hallucinate a very valid-sounding explanation that convinces the patient they have a rare disorder that they spend $1000s of dollars testing for, despite real doctors disagreeing and test results being negative. Maybe it only does this once in a long series of questions, but the seed is planted despite negative test results. Then when the patient asks ChatGPT if the tests could give false negatives, it could provide some very valid-sounding answers that say repeat testing might be necessary. This isn’t a hypothetical. It’s currently happening to an old friend of mine. They won’t let it go because ChatGPT continues to give them the same answers when they ask it the same (leading) questions. At this point he can get ChatGPT to give him any medical answer he wants to hear by rephrasing his questions until ChatGPT tells him what he wants to hear. He’s learned to ask questions like “If someone has symptoms _____ and ____ could they have <rare disease> and if so how would it be treated?” A real doctor would see what’s happening and address the leading questions. ChatGPT just takes it at face value. The difference between ChatGPT and real doctors is that he can iterate on his answer-shopping a hundred times in one sitting, whereas a doctor is going to see what’s happening and stop the patient. ChatGPT is an automated confirmation bias machine for hypochondriacs.

BlackSwanMan 3y ago
[flagged]
ngngngng 3y ago
> they spend $1000s of dollars testing for The subtext you're missing here is that GPT with access to the entire corpus of medical data could undermine the entire money printing machine (referring to US healthcare here). What test would cost thousands of dollars if the only human cost to run it is drawing some blood and putting it in a machine?
- Retric 3y ago
  Abstractions like that are meaningless. Why is the large hadron collider so expensive it’s just throwing stuff at each other and looking at what happens? Random teens could do that… The actual process of doing blood work is often surprisingly complicated.
  - hef19898 3y ago
    I see a resurected Theranos on the horizon. This time powered by by ChatGPT. What could go wrong?
    - alexb_ 3y ago
      Funny enough, I'm actually developing an AI-powered biomedical technological breakthrough that's about to disrupt the medical industry. It uses wearables to enable a blockchain-enabled data management system that also functions as a cloud based SaaS provider, linking with ChatGPT and NFTs to create value for those who are underprivileged, all with only 1 drop of blood. If you want to further this please give me a few billion dollars and I promise something might come :)
      - snovv_crash 3y ago
        I'm only interested if the blood is delivered by drones.
      - Firmwarrior 3y ago
        Sounds like it's not very eco friendly..
    - chaxor 3y ago
      Theranos wasn't a bad idea, and is actually possible. It is extremely unfortunate that it was so badly mismanaged.
      - hef19898 3y ago
        Theranos was a fraud, selling a service based on technology that simply doesn't exist.
        chaxor 3y ago
        Of course... that's absolutely true, and I didn't say otherwise. However, the basic idea actually is possible, but only for a very select few of the tests they proposed (which they still unfortunately didn't really get right of course). My point is that, unfortunately, the tests that are possible to do now won't get a company to form out of it - simply because they ruined the scene for everyone. Anyone trying to get a company going in this area and raising money for it will be met with a complete block of resistance because of the precedent set, even if the technology is actually completely reliable.
- Firmwarrior 3y ago
  I hate doctors and pharmaceutical companies as much as anyone, but those tests are serious business. There's a lot of very hard science and engineering involved in them When they're cheap outside the USA, it often just means the companies aren't attempting to recoup any r&d costs outside the US
  - hkt 3y ago
    Surely it isn't pharmaceuticals companies doing blood tests in the US? In the UK people's bloods get done in hospital labs (phlebotomy doesn't always occur in hospital though)
    - throwaway049 3y ago
      Some of that lab work is contracted out of the NHS, but your NHS doctor is largely steered by clinical guidelines rather than profit or ass-covering. Eg https://www.synnovis.co.uk/about-synnovis https://www.synnovis.co.uk/about-synnovis
    - sammax 3y ago
      The tests are done using proprietary hard- and software, though it is generally made by biotech and not pharmaceutical companies.
namuol 3y ago
Ah, yes. Hypochondriacs. I was a hypochondriac for years until I was able to get an appointment timed such that my symptoms were physically present while I was being assessed (not easy if you have a disease that comes and goes). I really hope you’re not a medical professional that interfaces directly with patients.
- faeriechangling 3y ago
  I was a hypochondriac for decades. I was eventually cured after decades of cropping hypochondria through self-diagnosis of the actual medical condition I had that became formal diagnosis eventually resulting in treatment through some modest lifestyle changes. I'm a BIG supporter of using things like ChatGPT, google, and sci-hub to do your own medical research because the whole system where some physician diagnoses you based on an extremely limited amount of data collected in a haphazard manner after a few minutes of observation because he's experienced and smart or whatever is incredibly dumb. The way people hold it up as the ethical standard which we cannot deviate from because it would be too dangerous is utterly baffling to me. The status quo totally lacks ethics and mostly serves to line the pockets of a cartel of doctors with a monopoly on access to medication and treatment who often condescendingly think patients are simply too irrational to treat themselves without their help. I legitimately cannot wait for this field to mature and medical self-help with AI assistance becomes the norm.
  - throwaway049 3y ago
    I agree with studying the field to help you understand your own health, but I prefer sci hub or any peer reviewed source over an LLM. I'll revise this view as LLMs develop, but right now I'm seeing plausible bs as often as I see good advice.
  - crx07 3y ago
    Fellow hypochondriac here. I was at the point where doctors, hospital staff, and lab techs would immediately warn new practitioners about me so they wouldn't waste finite medical resources in a small town, and I just completely discontinued normal activities out of terror as a result. When I finally blacked out and fractured my spine, first responders detected a lifelong cardiac arrhythmia in the back of an ambulance. Only with that knowledge have I been able to receive treatment and begin to heal emotionally from the gaslighting and medical abuse I experienced while in the care of licensed professionals. AI-assisted medicine will prevent so many of these mistakes in the future. It can't come soon enough as far as I'm concerned.
grumple 3y ago
Is what was described above a worst-case scenario? It accurately describes the way nearly every doctor's visit I or anyone I know has had.
- badloginagain 3y ago
  US healthcare could accurately be described as the literal worst case scenario.
chaxor 3y ago
I think we can agree that both of those cases aren't worst case scenarios, but rather the only case, or best case scenarios (in the US). The only real way of dealing with health (in the US) is just to never go to a physician and to avoid healthcare at all costs. It will never provide any real answers, despite charging amounts that no individual will ever be able to pay. The only thing that healthcare in the US really seems to be capable of dealing with is ER, or more specifically clear topically observable physical injuries by mechanical force. We're still not good enough at it to fix things to be as they were before an incident in all cases, but we're closer than almost any other area of medicine.
- southernplaces7 3y ago
  Your totally generalized dismissal of U.S. healthcare, which despite its many known flaws, routinely treats millions of people in the U.S. per year very effectively, is way the fuck off base and absurd. Not all doctors are the same, not all patients are the same and not all outcomes are worst case scenarios on the part of doctor incompetence. I really hope nobody else takes seriously such a blanket pile of exaggerations to the point of nonsense.
mharig 3y ago
That can happen with physical physicians, too. Happened to my granma when she was 90. She had convinced her doctor prescribing her 5 or 6 drugs. She swallowed them until she was so weak she had to go to hospital, where the tests revealed that she was 100% sane. She didn't like it.

BlackSwanMan 3y ago

[flagged]

ngngngng 3y ago

> they spend $1000s of dollars testing for The subtext you're missing here is that GPT with access to the entire corpus of medical data could undermine the entire money printing machine (referring to US healthcare here). What test would cost thousands of dollars if the only human cost to run it is drawing some blood and putting it in a machine?

Retric 3y ago
Abstractions like that are meaningless. Why is the large hadron collider so expensive it’s just throwing stuff at each other and looking at what happens? Random teens could do that… The actual process of doing blood work is often surprisingly complicated.
- hef19898 3y ago
  I see a resurected Theranos on the horizon. This time powered by by ChatGPT. What could go wrong?
  - alexb_ 3y ago
    Funny enough, I'm actually developing an AI-powered biomedical technological breakthrough that's about to disrupt the medical industry. It uses wearables to enable a blockchain-enabled data management system that also functions as a cloud based SaaS provider, linking with ChatGPT and NFTs to create value for those who are underprivileged, all with only 1 drop of blood. If you want to further this please give me a few billion dollars and I promise something might come :)
    - snovv_crash 3y ago
      I'm only interested if the blood is delivered by drones.
    - Firmwarrior 3y ago
      Sounds like it's not very eco friendly..
  - chaxor 3y ago
    Theranos wasn't a bad idea, and is actually possible. It is extremely unfortunate that it was so badly mismanaged.
    - hef19898 3y ago
      Theranos was a fraud, selling a service based on technology that simply doesn't exist.
      - chaxor 3y ago
        Of course... that's absolutely true, and I didn't say otherwise. However, the basic idea actually is possible, but only for a very select few of the tests they proposed (which they still unfortunately didn't really get right of course). My point is that, unfortunately, the tests that are possible to do now won't get a company to form out of it - simply because they ruined the scene for everyone. Anyone trying to get a company going in this area and raising money for it will be met with a complete block of resistance because of the precedent set, even if the technology is actually completely reliable.
Firmwarrior 3y ago
I hate doctors and pharmaceutical companies as much as anyone, but those tests are serious business. There's a lot of very hard science and engineering involved in them When they're cheap outside the USA, it often just means the companies aren't attempting to recoup any r&d costs outside the US
- hkt 3y ago
  Surely it isn't pharmaceuticals companies doing blood tests in the US? In the UK people's bloods get done in hospital labs (phlebotomy doesn't always occur in hospital though)
  - throwaway049 3y ago
    Some of that lab work is contracted out of the NHS, but your NHS doctor is largely steered by clinical guidelines rather than profit or ass-covering. Eg https://www.synnovis.co.uk/about-synnovis https://www.synnovis.co.uk/about-synnovis
  - sammax 3y ago
    The tests are done using proprietary hard- and software, though it is generally made by biotech and not pharmaceutical companies.

namuol 3y ago

Ah, yes. Hypochondriacs. I was a hypochondriac for years until I was able to get an appointment timed such that my symptoms were physically present while I was being assessed (not easy if you have a disease that comes and goes). I really hope you’re not a medical professional that interfaces directly with patients.

faeriechangling 3y ago
I was a hypochondriac for decades. I was eventually cured after decades of cropping hypochondria through self-diagnosis of the actual medical condition I had that became formal diagnosis eventually resulting in treatment through some modest lifestyle changes. I'm a BIG supporter of using things like ChatGPT, google, and sci-hub to do your own medical research because the whole system where some physician diagnoses you based on an extremely limited amount of data collected in a haphazard manner after a few minutes of observation because he's experienced and smart or whatever is incredibly dumb. The way people hold it up as the ethical standard which we cannot deviate from because it would be too dangerous is utterly baffling to me. The status quo totally lacks ethics and mostly serves to line the pockets of a cartel of doctors with a monopoly on access to medication and treatment who often condescendingly think patients are simply too irrational to treat themselves without their help. I legitimately cannot wait for this field to mature and medical self-help with AI assistance becomes the norm.
- throwaway049 3y ago
  I agree with studying the field to help you understand your own health, but I prefer sci hub or any peer reviewed source over an LLM. I'll revise this view as LLMs develop, but right now I'm seeing plausible bs as often as I see good advice.
- crx07 3y ago
  Fellow hypochondriac here. I was at the point where doctors, hospital staff, and lab techs would immediately warn new practitioners about me so they wouldn't waste finite medical resources in a small town, and I just completely discontinued normal activities out of terror as a result. When I finally blacked out and fractured my spine, first responders detected a lifelong cardiac arrhythmia in the back of an ambulance. Only with that knowledge have I been able to receive treatment and begin to heal emotionally from the gaslighting and medical abuse I experienced while in the care of licensed professionals. AI-assisted medicine will prevent so many of these mistakes in the future. It can't come soon enough as far as I'm concerned.

grumple 3y ago

Is what was described above a worst-case scenario? It accurately describes the way nearly every doctor's visit I or anyone I know has had.

badloginagain 3y ago
US healthcare could accurately be described as the literal worst case scenario.

chaxor 3y ago

I think we can agree that both of those cases aren't worst case scenarios, but rather the only case, or best case scenarios (in the US). The only real way of dealing with health (in the US) is just to never go to a physician and to avoid healthcare at all costs. It will never provide any real answers, despite charging amounts that no individual will ever be able to pay. The only thing that healthcare in the US really seems to be capable of dealing with is ER, or more specifically clear topically observable physical injuries by mechanical force. We're still not good enough at it to fix things to be as they were before an incident in all cases, but we're closer than almost any other area of medicine.

southernplaces7 3y ago
Your totally generalized dismissal of U.S. healthcare, which despite its many known flaws, routinely treats millions of people in the U.S. per year very effectively, is way the fuck off base and absurd. Not all doctors are the same, not all patients are the same and not all outcomes are worst case scenarios on the part of doctor incompetence. I really hope nobody else takes seriously such a blanket pile of exaggerations to the point of nonsense.

mharig 3y ago

That can happen with physical physicians, too. Happened to my granma when she was 90. She had convinced her doctor prescribing her 5 or 6 drugs. She swallowed them until she was so weak she had to go to hospital, where the tests revealed that she was 100% sane. She didn't like it.