4 ms·

As an expert in machine learning, I don't see how someone would expect this to work. Actually to predict a disease, you need true positives and true negatives.

by ggggtez 11y ago

As an expert in machine learning, I don't see how someone would expect this to work. Actually to predict a disease, you need true positives and true negatives. If you only had access to the true positive data, it would be a lot harder to predict accurately.

askldfhjkasfhd 11y ago
Can you explain what you mean a little more? It seems like by having complete medical records, there would be a great deal you could infer statistically. Do you mean that it's not sufficient to have a medical record that doesn't indicate knee issues? That you would need a medical record that confirms there are no knee issues?
- throwanem 11y ago
  If you want a reliable result, yes. Consider: I may have a medical record that doesn't indicate knee issues because I have correctly been diagnosed with no knee issues. Or I may have such a record because I have been incorrectly so diagnosed, despite the presence of knee issues. Or I may have such a record because I haven't been to a doctor about my knee issues in the years since they've developed, and so no opportunity has arisen for my knee issues to be documented. Simply from a medical record reflecting no evidence of knee issues, you cannot know which of these, if any, is true. As I mentioned in a reply to your neighbor comment, clinical studies are designed to exclude such confounders as these - and that's a considerable part of why such studies are so expensive to design and carry out. It's very difficult to see how such exclusion could be achieved with data of the quality which seems to be involved here.
chris_va 11y ago
Seems pretty simple, really. 1000 patients come in with symptoms that look like cancer at year 0. 100 actually get diagnosed with cancer at some point between year 0 and year 5. Presumably, the remaining 900 didn't have cancer at year 0.
- throwanem 11y ago
  "Seems". You can't know what fraction of your remaining 900 had cancer but weren't diagnosed with it for any of a wide variety of reasons, from death through misadventure, through false negative, to simple loss to followup. Clinical studies are designed to exclude such confounding outcomes. It's very difficult to see how any study of this data could be designed to do likewise.
  - chris_va 11y ago
    True, but I think you can mitigate these issues. Especially for the NHS dataset, since you will either see the patient in there or in a death index (unlike, say, US where they may have just gone to another hospital). Also, the scope here is more like a longitudinal vaccine study than a clinical trial. 50M people will provide a lot of robustness that you wouldn't see in a 1000 patient trial.
    - throwanem 11y ago
      > you will either see the patient in there or in a death index Or you won't see any new information for the patient, because the patient is lost to followup. Or - worse - you'll see new information, but it'll be invisibly erroneous, because random GPs don't work to the standard that physicians administering examinations in clinical studies do. > 50M people will provide a lot of robustness that you wouldn't see in a 1000 patient trial Not if the 1000-patient trial is well designed, and the data of those 50M people is totally uncontrolled and unverified. This isn't warfare - Stalin's dictum has no place here. You can't overcome the flaws of a dirty dataset by adding more dirty data to it, especially when you literally cannot know either the magnitude or the nature of the inaccuracy, or even tell what's accurate from what's not.
      - petra 11y ago
        >> Or - worse - you'll see new information, but it'll be invisibly erroneous, because random GPs don't work to the standard that physicians administering examinations in clinical studies do. So you will take this into account and emphasize the more reliable types of tests, like blood tests. Or you'll find ways of learning which doctors who do the tests more accurately(or train doctors to do so) and which people are more consistent/reliable with their relationship with their docs. Or maybe you'll get a few hypotheses which are relatively likely and that would incentivize the researchers/google to do small clinical trials on them. It's worth a try at least.

by ggggtez 11y ago

askldfhjkasfhd 11y ago

Can you explain what you mean a little more? It seems like by having complete medical records, there would be a great deal you could infer statistically. Do you mean that it's not sufficient to have a medical record that doesn't indicate knee issues? That you would need a medical record that confirms there are no knee issues?

throwanem 11y ago
If you want a reliable result, yes. Consider: I may have a medical record that doesn't indicate knee issues because I have correctly been diagnosed with no knee issues. Or I may have such a record because I have been incorrectly so diagnosed, despite the presence of knee issues. Or I may have such a record because I haven't been to a doctor about my knee issues in the years since they've developed, and so no opportunity has arisen for my knee issues to be documented. Simply from a medical record reflecting no evidence of knee issues, you cannot know which of these, if any, is true. As I mentioned in a reply to your neighbor comment, clinical studies are designed to exclude such confounders as these - and that's a considerable part of why such studies are so expensive to design and carry out. It's very difficult to see how such exclusion could be achieved with data of the quality which seems to be involved here.

chris_va 11y ago

Seems pretty simple, really. 1000 patients come in with symptoms that look like cancer at year 0. 100 actually get diagnosed with cancer at some point between year 0 and year 5. Presumably, the remaining 900 didn't have cancer at year 0.

throwanem 11y ago
"Seems". You can't know what fraction of your remaining 900 had cancer but weren't diagnosed with it for any of a wide variety of reasons, from death through misadventure, through false negative, to simple loss to followup. Clinical studies are designed to exclude such confounding outcomes. It's very difficult to see how any study of this data could be designed to do likewise.
- chris_va 11y ago
  True, but I think you can mitigate these issues. Especially for the NHS dataset, since you will either see the patient in there or in a death index (unlike, say, US where they may have just gone to another hospital). Also, the scope here is more like a longitudinal vaccine study than a clinical trial. 50M people will provide a lot of robustness that you wouldn't see in a 1000 patient trial.
  - throwanem 11y ago
    > you will either see the patient in there or in a death index Or you won't see any new information for the patient, because the patient is lost to followup. Or - worse - you'll see new information, but it'll be invisibly erroneous, because random GPs don't work to the standard that physicians administering examinations in clinical studies do. > 50M people will provide a lot of robustness that you wouldn't see in a 1000 patient trial Not if the 1000-patient trial is well designed, and the data of those 50M people is totally uncontrolled and unverified. This isn't warfare - Stalin's dictum has no place here. You can't overcome the flaws of a dirty dataset by adding more dirty data to it, especially when you literally cannot know either the magnitude or the nature of the inaccuracy, or even tell what's accurate from what's not.
    - petra 11y ago
      >> Or - worse - you'll see new information, but it'll be invisibly erroneous, because random GPs don't work to the standard that physicians administering examinations in clinical studies do. So you will take this into account and emphasize the more reliable types of tests, like blood tests. Or you'll find ways of learning which doctors who do the tests more accurately(or train doctors to do so) and which people are more consistent/reliable with their relationship with their docs. Or maybe you'll get a few hypotheses which are relatively likely and that would incentivize the researchers/google to do small clinical trials on them. It's worth a try at least.