3 ms·

I don't see the harm in something like this. If you were to train your own LLM specifically on medicine & medical data, then show it your lifetime history of do

by pogue 2y ago

I don't see the harm in something like this. If you were to train your own LLM specifically on medicine & medical data, then show it your lifetime history of doctors visits, blood work, what drugs you're taking, have taken, symptoms and etc etc etc there's a decent probability it could give some perspective you might not be able to get elsewhere - especially in the broken US healthcare system. Having access to multiple specialists in many different fields is something a large % of the population simply doesn't have access to for a variety of reasons. I think the issue is people might assume it's going to spit out your exact diagnosis 100% of the time right off the bat. That's obviously not true, but that's not what I got from reading the description of the project. It sounds more like a potential useful tool for folks to have as an additional option, and then to bring that information to a doctor to see if it pans out. Regardless, one extremely helpful thing I got out of reading the reddit thread was the mention of an unrelated project called Fasten, which will apparently allow you to request all your medical records from many sources and then self host them: Fasten is an open-source, self-hosted, personal/family electronic medical record aggregator, designed to integrate with 100,000's of insurances/hospitals/clinics https://github.com/fastenhealth/fasten-onprem https://github.com/fastenhealth/fasten-onprem

afro88 2y ago
> If you were to train your own LLM specifically on medicine & medical data That's a massive "if" that the rest of your argument hinges on. You can't train your own LLM on your own data. You can fine tune it's responses to be more in line with the format or style you want. But this is usually used as a cost saving exercise when you are feeding s big system or inital prompt for lots of queries. > there's a decent probability it could give some perspective you might not be able to get elsewhere There's also a decent probability that it will give a wildly incorrect perspective that influences how you approach your possible disease in an unproductive way. The author has no idea how accurate the tool is. They list no effectiveness measures. They just talk about the cool things it does (again, without accuracy) and their subjective experience with it. It's the same pattern with LLM based tools we see so often: someone gets lucky with a useful result that no one else has done yet, they assume it will work just as well for all inputs, they build an app around it. You _have_ to measure effectiveness across a wide variety and quantity of inputs because of how inconsistent LLMs still are.
Aurornis 2y ago
> I don't see the harm in something like this. Incorrect self diagnosis is a big problem for doctors. Patients will show up convinced they have one condition and reject their doctors’ attempts to explore other explanations. It’s a big problem right now with TikTok self diagnoses. People view a lot of TikToks and think they know better than their doctor for themselves or, worse, their kids. The easiest example is when people decide they have a bacterial infection when they really have a common cold. They will shop around until they find a doctor who is a pushover and will prescribe antibiotics on demand. This is a big problem right now with antibiotic overprescription. The problem with this specific post, though, is that it’s misleading. The author tries to present this tool as if it did a better job than $100K of medical testing and 5 years of doctors, but when you read the details it’s clear that the results from all of those doctors and medical testing were put into the tool. Many of the comments here and on Reddit are missing this key point: The tool was mirroring the diagnosis from the doctors because it was given the medical records from those doctors, not because it was diagnosing the symptoms from a clean slate.

by pogue 2y ago

afro88 2y ago

> If you were to train your own LLM specifically on medicine & medical data That's a massive "if" that the rest of your argument hinges on. You can't train your own LLM on your own data. You can fine tune it's responses to be more in line with the format or style you want. But this is usually used as a cost saving exercise when you are feeding s big system or inital prompt for lots of queries. > there's a decent probability it could give some perspective you might not be able to get elsewhere There's also a decent probability that it will give a wildly incorrect perspective that influences how you approach your possible disease in an unproductive way. The author has no idea how accurate the tool is. They list no effectiveness measures. They just talk about the cool things it does (again, without accuracy) and their subjective experience with it. It's the same pattern with LLM based tools we see so often: someone gets lucky with a useful result that no one else has done yet, they assume it will work just as well for all inputs, they build an app around it. You _have_ to measure effectiveness across a wide variety and quantity of inputs because of how inconsistent LLMs still are.

Aurornis 2y ago

> I don't see the harm in something like this. Incorrect self diagnosis is a big problem for doctors. Patients will show up convinced they have one condition and reject their doctors’ attempts to explore other explanations. It’s a big problem right now with TikTok self diagnoses. People view a lot of TikToks and think they know better than their doctor for themselves or, worse, their kids. The easiest example is when people decide they have a bacterial infection when they really have a common cold. They will shop around until they find a doctor who is a pushover and will prescribe antibiotics on demand. This is a big problem right now with antibiotic overprescription. The problem with this specific post, though, is that it’s misleading. The author tries to present this tool as if it did a better job than $100K of medical testing and 5 years of doctors, but when you read the details it’s clear that the results from all of those doctors and medical testing were put into the tool. Many of the comments here and on Reddit are missing this key point: The tool was mirroring the diagnosis from the doctors because it was given the medical records from those doctors, not because it was diagnosing the symptoms from a clean slate.