3 ms·

Launching Health in ChatGPT to US Users

32 points by mellosouls 3mo ago

casper14 3mo ago
Oh hell no
robot_jesus 3mo ago
I’m intrigued by the power of these models to help me take more control of my healthcare. Sure, doctors’ advice is best, but sometimes I wait 6 months to get to see a specialist who seems hell bent on getting through our appt as quickly as possible. Models are infinitely patient and can see a lot more context about my health. But, the privacy is a big question mark. The page extensively reassures on privacy (no model training, encrypted, no ads, can be deleted). There’s only one part that stands out to me: > Memory remains in your control. To personalize future chats, memories can be created from your Health conversations. Memories are not created directly from connected medical records and Apple Health information. If you’d like to have a conversation without creating memories, you can use Temporary Chat (opens in a new window) or turn memory off in Settings > Memory. Great that you can turn it off in settings but I wonder how many people will do that and how much sensitive context leakage will come out of your conversations and suddenly you’ll get ads for rashes and whatnot.
Wololooo 3mo ago
This is a terrible idea on any way shape or form you can imagine. Even if they say they won't, they will, and there is no telling what they would do with the data itself...
- glimshe 3mo ago
  This isn't so bad vs the alternative. My healthcare data is spread over a bunch of small systems built with poor security practices at the clinics and hospitals I've been to. It's probably out there in some pirate database already, all cleaned up and consolidated. With ChatGPT at least I control what I enter in the prompt to minimize my exposure.
- energy123 3mo ago
  It's a boon for low income people in low income countries. Some random facts: - They avoid doctors because it's expensive and they don't want to burden their families. - There aren't enough doctors, regardless of social spending. - There isn't enough social spending. Even if this only increases bandwidth of existing doctors, or allows lesser-trained nurses to cosplay as a doctor during triage to increase bandwidth, it's a great thing.
- nope1000 3mo ago
  I bet people out there are already uploading all sorts of private or confidential stuff to ChatGPT. I don't think that will make much of a difference.
openasocket 3mo ago
Searching in vain for any mention of HIPAA compliance…
- claytongulick 3mo ago
  HIPAA only applies to data originating from a covered entity. It doesn't apply to information the patient supplies directly. As a patient, I can take out a street sign displaying my health records if I want. It's my data. If I want to upload it to a strange AI or any other app, that's completely fine. HIPAA isn't involved, for either me or the company I upload to. In fact, quite the opposite. CMS has been going to great pains over the past decade or so to create standards to help patients get access to their records and own them themselves. Different states may have additional protections in this case, and I know there has been some discussion at the federal level about regulating these "wellness" apps. As an interesting technical detail not many people know, HIPAA only applies to Covered Entities and Business Associates engaging in the electronic exchange of information using "standard transactions", and was intended to regulate insurance and billing transactions. A doctor that keeps paper charts and bills cash directly to the patient wouldn't fall under HIPAA at all.
  - openasocket 3mo ago
    It’s a pretty fine line, and if they want to do anything beyond just telling you things about your data, actually taking any sort of actions or communications with health care providers, they should qualify as a business associate under HIPAA. And I certainly am never going to share my healthcare data with an AI company without any sort of regulations or oversight, beyond a TOS.
    - claytongulick 3mo ago
      A Business Associate (BA) under HIPAA is a entity that contracts with a Covered Entity (CE) using a Business Associate Agreement (BAA). OpenAI wouldn't be considered a BA unless they had an executed BAA with a CE. There are a wide variety of things, including communication with providers, that OpenAI can do without being regulated by HIPAA. The provider, however would most likely be covered, so they would be under an obligation to not disclose any PHI. As an example, consider a hypothetical "email my doc" app. This app could take health information from the patient and email it to all of their doctors without falling under HIPAA at all. What you're probably thinking of is the HIPAA Privacy Rule (there's more to HIPAA than just that, the Security Rule is pretty involved). Under the Privacy Rule, CEs and BAs can not disclose PHI without a patient's consent, with a few exceptions. If the patient is disclosing that information though, all bets are off (aside from state level regulations).
le-mark 3mo ago
The American Medical Association and health insurance industries can all fuck off. The healthcare situation in the US is so absurd and Kafka-esque I welcome LLMs in medicine. I haven’t seen a MD in years, I’ve only seen nurse practitioners and physician assistants. Add to that how insurance will randomly deny tests, procedures and treatments, I have very little use for the current medical establishment.
- afavour 3mo ago
  > I have very little use for the current medical establishment. What do you think the LLMs are trained on? I agree the current state of US healthcare is a mess. But that doesn’t mean anything is better. The current state of LLMs are so agreeable I worry what they’ll suggest to users who provide even the slightest pushback. “Are you sure I need this test?” “You’re right to question that. It’s unlikely to surface anything you don’t already know” …while a chronic condition develops under the surface
- cynicalsecurity 3mo ago
  Do you have a health insurance?
- codingdave 3mo ago
  Agreed on the problems, but how will an LLM fix any of that? Even if it does somehow give you the right answer, it cannot perform procedures or prescribe medicine. You still need to get an appointment for anything beyond information. Your insurance isn't going to say, "Oh, sure, if chatGPT said you need this, we'll approve it."
- powerpcmac 3mo ago
  It sounds like you need a better patient advocate or your area is very underserved in provider choice. Many people have success with the health system.
- delfinom 3mo ago
  NYS is trying to make it illegal for LLMs to give medical, legal or engineering advice. Because of course its a state built upon exploitation so. https://www.nysenate.gov/legislation/bills/2025/S7263/amendment/A https://www.nysenate.gov/legislation/bills/2025/S7263/amendm... The engineering aspect is quite hilarious. I have a PE license, but don't use it. But the PE definition in NY is wide open. Technically, if you do the right work, even writing software can fall under a PE license, though its incredibly rare. So this could theoretically ban LLM Code assistants.
skilled 3mo ago
I think doctors are going to lose their minds or at the very least get extremely agitated/frustrated as patients increasingly use AI tools to do self-research for symptoms and diagnosis. And I know this because I am guilty of this myself and recently had a dentist tell me to not do that. And this applies to so many health workers, your GP as well as your physical therapist, etc. Recently I had Claude analyze an MRI of my knee and it gave me five different potential problems, but once I got the radiologist report - none of those problems were present, it did not detect that it was chondromalacia that I found out from the radiologist report. Health and the current iteration of LLM's just doesn't seem like a great match.
- BadBadJellyBean 3mo ago
  I know of someone who had a red bump on their arm and asked ChatGPT if that was a problem and if they should see a doctor. It said no so they left it. Turns out it was a tick and they now have lyme disease.
  - afavour 3mo ago
    I have a friend who uploaded a photo of a mark on their back to Claude, which stated it was a cut that was healing. Went to the doctor, it was skin cancer. That said, Claude did say “I’m not a doctor, if in doubt make sure you see one”. So, that’s good… but then, what’s the point in using it?
  - matwood 3mo ago
    I feel like I'm reading comments on LLMs writing code where they romanticize that programmers were perfect before LLMs. Doctors are not infallible. I know plenty of them and they absolutely tell me who and who not to go to. The old joke, what do you call the person who graduated last in their class at med school? Doctor. Lyme disease in particular has all sorts of issues with diagnosis - from real doctors! https://pmc.ncbi.nlm.nih.gov/articles/PMC10778834/ https://pmc.ncbi.nlm.nih.gov/articles/PMC10778834/
    - BadBadJellyBean 3mo ago
jgilias 3mo ago
What can go wrong.
- pluc 3mo ago
  "We'll see!" -USA
  - ActionHank 3mo ago
    "Coverage is denied for the cost of your visit to the doctor and prescribed treatment because you didn't go through our triage concierge chat bot first."
    - ryandrake 3mo ago
      Stop giving them ideas!!
snowstormsun 3mo ago
Hell no
throwatdem12311 3mo ago
Yeaaaaahhhhhh…no
trymas 3mo ago
Americans will do everything but not have socialised health care even though they are already leading the world on taxes spent per capita for health care. https://en.wikipedia.org/wiki/List_of_countries_by_total_health_expenditure_per_capita#/media/File:Average_annual_health_spending._US_dollars_(PPP)_per_person._OECD_countries_and_more.png https://en.wikipedia.org/wiki/List_of_countries_by_total_hea... Also, what is HIPPA?
- andsoitis 3mo ago
  [dead]
- donkey_brains 3mo ago
  It’s a patient privacy law that applies to hippos.
  - ThinkingGuy 3mo ago
    It's an acronym in which neither 'P' stands for 'privacy'
an0malous 3mo ago
I tried posting an Ask HN about my experience using Claude and ChatGPT for medical advice, but it didn't get enough upvotes for any responses. Maybe this is a good place to repost (the part relevant to medical treatment is at the end): Question: Why don't Claude or ChatGPT get message timestamps? Go to any conversation you’ve had with Claude or ChatGPT in their chat apps and ask them what time they sent their last message, and you’ll find they don’t see timestamps for any of the messages in your chat history. They might guess or hallucinate an answer but they’ll be wildly off and if you probe further they should admit they don’t see timestamps at all, they only know the current time. I’ve also noticed generally that if you pick up conversations over multiple days, it doesn’t know that days have passed in between messages. —— Me: How much time has passed since your last message? Claude: I don't have a way to measure the gap between my messages—I don't have a clock running between turns, and nothing in our conversation timestamps the messages for me. So I can't tell you how much time has passed since my last reply. Could've been seconds, could've been days. If it's useful, I can check the current time, but that would only tell you now, not the interval since I last wrote. —— This is very bizarre to me and seems like an obvious flaw in the product, but both major AI companies are doing this and not implementing the obvious solution of surfacing message timestamps. Does anyone know why they don’t do this? For an example of the type of issue this causes, I’ve been using Claude for advice while my dog is going through medical treatment and it’s very concerning because I’ll say I gave him some medicine and then later ask questions about a symptom he’s displaying and Claude doesn’t seem to know whether the medicine was minutes ago or days ago and usually doesn’t ask for clarification either. I’ve learned to either be explicit about how much time has passed or just not ask it questions where timing would be a factor. But I’m surprised because this seems like it should be an easy thing to address.
- mcintyre1994 3mo ago
  I find this weird too! When you say something that prompts them that you’re talking about the current time, they’ll go and fetch it and use that to understand time frames which makes sense. But if you’ve been talking about something tomorrow, and then tomorrow comes and you go back to that chat, they don’t understand that now it’s today/right now.
- 3mo ago
powerpcmac 3mo ago
Specific tools for MDs to record patient notes were found to hallucinate patient details in 45% of cases. They were especially bad on patient mental health. I am against AI in healthcare in most cases, but anything other than augmentation is probably never going to be happening. https://www.auditor.on.ca/en/content/specialreports/specialreports/en26/2026_AI_EN.pdf https://www.auditor.on.ca/en/content/specialreports/specialr...
- papa_bear 3mo ago
  That study cites that 45% of vendors had hallucinations, likely vendors using smaller or cheaper models. So 55% of vendors had no known hallucinations.
tim-tday 3mo ago
Oh god no.
ungreased0675 3mo ago
Wow, this seems like a colossally horrible idea. Give OpenAI your medical records? People would really do that?
rchaud 3mo ago
From this[0] > A lawsuit filed by Scott Winters, a 55-year-old pastor from Florida, claimed ChatGPT told him not to seek medical help after repeatedly asking about symptoms in the build-up to a near-fatal pulmonary embolism. in the same article: > OpenAI's Drew Pusateri told the BBC's US news partner CBS News that "ChatGPT is not a doctor and should never be used as a substitute for medical care, diagnosis or treatment". [0] https://www.bbc.com/news/articles/cwylp3nxp5yo https://www.bbc.com/news/articles/cwylp3nxp5yo
makeyouragent 3mo ago
[flagged]

32 points by mellosouls 3mo ago

Oh hell no

I’m intrigued by the power of these models to help me take more control of my healthcare. Sure, doctors’ advice is best, but sometimes I wait 6 months to get to see a specialist who seems hell bent on getting through our appt as quickly as possible. Models are infinitely patient and can see a lot more context about my health. But, the privacy is a big question mark. The page extensively reassures on privacy (no model training, encrypted, no ads, can be deleted). There’s only one part that stands out to me: > Memory remains in your control. To personalize future chats, memories can be created from your Health conversations. Memories are not created directly from connected medical records and Apple Health information. If you’d like to have a conversation without creating memories, you can use Temporary Chat (opens in a new window) or turn memory off in Settings > Memory. Great that you can turn it off in settings but I wonder how many people will do that and how much sensitive context leakage will come out of your conversations and suddenly you’ll get ads for rashes and whatnot.

Wololooo 3mo ago

This is a terrible idea on any way shape or form you can imagine. Even if they say they won't, they will, and there is no telling what they would do with the data itself...

glimshe 3mo ago
This isn't so bad vs the alternative. My healthcare data is spread over a bunch of small systems built with poor security practices at the clinics and hospitals I've been to. It's probably out there in some pirate database already, all cleaned up and consolidated. With ChatGPT at least I control what I enter in the prompt to minimize my exposure.
energy123 3mo ago
It's a boon for low income people in low income countries. Some random facts: - They avoid doctors because it's expensive and they don't want to burden their families. - There aren't enough doctors, regardless of social spending. - There isn't enough social spending. Even if this only increases bandwidth of existing doctors, or allows lesser-trained nurses to cosplay as a doctor during triage to increase bandwidth, it's a great thing.
nope1000 3mo ago
I bet people out there are already uploading all sorts of private or confidential stuff to ChatGPT. I don't think that will make much of a difference.

openasocket 3mo ago

Searching in vain for any mention of HIPAA compliance…

claytongulick 3mo ago
HIPAA only applies to data originating from a covered entity. It doesn't apply to information the patient supplies directly. As a patient, I can take out a street sign displaying my health records if I want. It's my data. If I want to upload it to a strange AI or any other app, that's completely fine. HIPAA isn't involved, for either me or the company I upload to. In fact, quite the opposite. CMS has been going to great pains over the past decade or so to create standards to help patients get access to their records and own them themselves. Different states may have additional protections in this case, and I know there has been some discussion at the federal level about regulating these "wellness" apps. As an interesting technical detail not many people know, HIPAA only applies to Covered Entities and Business Associates engaging in the electronic exchange of information using "standard transactions", and was intended to regulate insurance and billing transactions. A doctor that keeps paper charts and bills cash directly to the patient wouldn't fall under HIPAA at all.
- openasocket 3mo ago
  It’s a pretty fine line, and if they want to do anything beyond just telling you things about your data, actually taking any sort of actions or communications with health care providers, they should qualify as a business associate under HIPAA. And I certainly am never going to share my healthcare data with an AI company without any sort of regulations or oversight, beyond a TOS.
  - claytongulick 3mo ago
    A Business Associate (BA) under HIPAA is a entity that contracts with a Covered Entity (CE) using a Business Associate Agreement (BAA). OpenAI wouldn't be considered a BA unless they had an executed BAA with a CE. There are a wide variety of things, including communication with providers, that OpenAI can do without being regulated by HIPAA. The provider, however would most likely be covered, so they would be under an obligation to not disclose any PHI. As an example, consider a hypothetical "email my doc" app. This app could take health information from the patient and email it to all of their doctors without falling under HIPAA at all. What you're probably thinking of is the HIPAA Privacy Rule (there's more to HIPAA than just that, the Security Rule is pretty involved). Under the Privacy Rule, CEs and BAs can not disclose PHI without a patient's consent, with a few exceptions. If the patient is disclosing that information though, all bets are off (aside from state level regulations).

le-mark 3mo ago

The American Medical Association and health insurance industries can all fuck off. The healthcare situation in the US is so absurd and Kafka-esque I welcome LLMs in medicine. I haven’t seen a MD in years, I’ve only seen nurse practitioners and physician assistants. Add to that how insurance will randomly deny tests, procedures and treatments, I have very little use for the current medical establishment.

afavour 3mo ago
> I have very little use for the current medical establishment. What do you think the LLMs are trained on? I agree the current state of US healthcare is a mess. But that doesn’t mean anything is better. The current state of LLMs are so agreeable I worry what they’ll suggest to users who provide even the slightest pushback. “Are you sure I need this test?” “You’re right to question that. It’s unlikely to surface anything you don’t already know” …while a chronic condition develops under the surface
cynicalsecurity 3mo ago
Do you have a health insurance?
codingdave 3mo ago
Agreed on the problems, but how will an LLM fix any of that? Even if it does somehow give you the right answer, it cannot perform procedures or prescribe medicine. You still need to get an appointment for anything beyond information. Your insurance isn't going to say, "Oh, sure, if chatGPT said you need this, we'll approve it."
powerpcmac 3mo ago
It sounds like you need a better patient advocate or your area is very underserved in provider choice. Many people have success with the health system.
delfinom 3mo ago
NYS is trying to make it illegal for LLMs to give medical, legal or engineering advice. Because of course its a state built upon exploitation so. https://www.nysenate.gov/legislation/bills/2025/S7263/amendment/A https://www.nysenate.gov/legislation/bills/2025/S7263/amendm... The engineering aspect is quite hilarious. I have a PE license, but don't use it. But the PE definition in NY is wide open. Technically, if you do the right work, even writing software can fall under a PE license, though its incredibly rare. So this could theoretically ban LLM Code assistants.

skilled 3mo ago

I think doctors are going to lose their minds or at the very least get extremely agitated/frustrated as patients increasingly use AI tools to do self-research for symptoms and diagnosis. And I know this because I am guilty of this myself and recently had a dentist tell me to not do that. And this applies to so many health workers, your GP as well as your physical therapist, etc. Recently I had Claude analyze an MRI of my knee and it gave me five different potential problems, but once I got the radiologist report - none of those problems were present, it did not detect that it was chondromalacia that I found out from the radiologist report. Health and the current iteration of LLM's just doesn't seem like a great match.

BadBadJellyBean 3mo ago
I know of someone who had a red bump on their arm and asked ChatGPT if that was a problem and if they should see a doctor. It said no so they left it. Turns out it was a tick and they now have lyme disease.
- afavour 3mo ago
  I have a friend who uploaded a photo of a mark on their back to Claude, which stated it was a cut that was healing. Went to the doctor, it was skin cancer. That said, Claude did say “I’m not a doctor, if in doubt make sure you see one”. So, that’s good… but then, what’s the point in using it?
- matwood 3mo ago
  I feel like I'm reading comments on LLMs writing code where they romanticize that programmers were perfect before LLMs. Doctors are not infallible. I know plenty of them and they absolutely tell me who and who not to go to. The old joke, what do you call the person who graduated last in their class at med school? Doctor. Lyme disease in particular has all sorts of issues with diagnosis - from real doctors! https://pmc.ncbi.nlm.nih.gov/articles/PMC10778834/ https://pmc.ncbi.nlm.nih.gov/articles/PMC10778834/
  - BadBadJellyBean 3mo ago

jgilias 3mo ago

What can go wrong.

pluc 3mo ago
"We'll see!" -USA
- ActionHank 3mo ago
  "Coverage is denied for the cost of your visit to the doctor and prescribed treatment because you didn't go through our triage concierge chat bot first."
  - ryandrake 3mo ago
    Stop giving them ideas!!

snowstormsun 3mo ago

Hell no

throwatdem12311 3mo ago

Yeaaaaahhhhhh…no

trymas 3mo ago

Americans will do everything but not have socialised health care even though they are already leading the world on taxes spent per capita for health care. https://en.wikipedia.org/wiki/List_of_countries_by_total_health_expenditure_per_capita#/media/File:Average_annual_health_spending._US_dollars_(PPP)_per_person._OECD_countries_and_more.png https://en.wikipedia.org/wiki/List_of_countries_by_total_hea... Also, what is HIPPA?

andsoitis 3mo ago
[dead]
donkey_brains 3mo ago
It’s a patient privacy law that applies to hippos.
- ThinkingGuy 3mo ago
  It's an acronym in which neither 'P' stands for 'privacy'

an0malous 3mo ago

I tried posting an Ask HN about my experience using Claude and ChatGPT for medical advice, but it didn't get enough upvotes for any responses. Maybe this is a good place to repost (the part relevant to medical treatment is at the end): Question: Why don't Claude or ChatGPT get message timestamps? Go to any conversation you’ve had with Claude or ChatGPT in their chat apps and ask them what time they sent their last message, and you’ll find they don’t see timestamps for any of the messages in your chat history. They might guess or hallucinate an answer but they’ll be wildly off and if you probe further they should admit they don’t see timestamps at all, they only know the current time. I’ve also noticed generally that if you pick up conversations over multiple days, it doesn’t know that days have passed in between messages. —— Me: How much time has passed since your last message? Claude: I don't have a way to measure the gap between my messages—I don't have a clock running between turns, and nothing in our conversation timestamps the messages for me. So I can't tell you how much time has passed since my last reply. Could've been seconds, could've been days. If it's useful, I can check the current time, but that would only tell you now, not the interval since I last wrote. —— This is very bizarre to me and seems like an obvious flaw in the product, but both major AI companies are doing this and not implementing the obvious solution of surfacing message timestamps. Does anyone know why they don’t do this? For an example of the type of issue this causes, I’ve been using Claude for advice while my dog is going through medical treatment and it’s very concerning because I’ll say I gave him some medicine and then later ask questions about a symptom he’s displaying and Claude doesn’t seem to know whether the medicine was minutes ago or days ago and usually doesn’t ask for clarification either. I’ve learned to either be explicit about how much time has passed or just not ask it questions where timing would be a factor. But I’m surprised because this seems like it should be an easy thing to address.

mcintyre1994 3mo ago
I find this weird too! When you say something that prompts them that you’re talking about the current time, they’ll go and fetch it and use that to understand time frames which makes sense. But if you’ve been talking about something tomorrow, and then tomorrow comes and you go back to that chat, they don’t understand that now it’s today/right now.
3mo ago

powerpcmac 3mo ago

Specific tools for MDs to record patient notes were found to hallucinate patient details in 45% of cases. They were especially bad on patient mental health. I am against AI in healthcare in most cases, but anything other than augmentation is probably never going to be happening. https://www.auditor.on.ca/en/content/specialreports/specialreports/en26/2026_AI_EN.pdf https://www.auditor.on.ca/en/content/specialreports/specialr...

papa_bear 3mo ago
That study cites that 45% of vendors had hallucinations, likely vendors using smaller or cheaper models. So 55% of vendors had no known hallucinations.

tim-tday 3mo ago

Oh god no.

ungreased0675 3mo ago

Wow, this seems like a colossally horrible idea. Give OpenAI your medical records? People would really do that?

rchaud 3mo ago

From this[0] > A lawsuit filed by Scott Winters, a 55-year-old pastor from Florida, claimed ChatGPT told him not to seek medical help after repeatedly asking about symptoms in the build-up to a near-fatal pulmonary embolism. in the same article: > OpenAI's Drew Pusateri told the BBC's US news partner CBS News that "ChatGPT is not a doctor and should never be used as a substitute for medical care, diagnosis or treatment". [0] https://www.bbc.com/news/articles/cwylp3nxp5yo https://www.bbc.com/news/articles/cwylp3nxp5yo

makeyouragent 3mo ago

[flagged]