3 ms·

Would the confidence level necessary to trigger a procedure not quickly be adjusted to keep the false positive rate in check?

tptacek 3y ago
How would it be? Again, the base rates of a lot of these conditions are very low, but not low enough to say "this is a zebra, not a horse", so I think a suspicious lump is likely to trigger a CT, PET, or (in the worst case) a biopsy.
- jstummbillig 3y ago
  I don't understand what "base rates" means in this context. If you have a test result, that you think is showing something with a certain probability, which is higher then the current necessary confidence level required to trigger further probing, and it turns out afterwards it was a false positive, my naive way to go about this would be to raise the minimum confidence level afterwards (and vice versa if it's a true positive) Does this fail in horrible ways that I am missing right now?
  - tptacek 3y ago
    Base rate: select a person at random from the population; how likely is a true positive?
SketchySeaBeast 3y ago
What confidence level? Isn't this advocating for the patient to make their own choices around elective scans and procedures?
robwwilliams 3y ago
Hard to say. Consider a patient without complaints with a scan that shows an abnormal bleb in or on the head of the pancreas. Physician 1 thinks it could be a pancreatic tumor given patient age and family history. Physician 2 thinks it is more likely to be a minor lifelong development anomaly with the bleb actually being an intrusive bit of the spleen. Is exploratory surgery needed? Watchful waiting? An acquaintance of mine with the second harmless condition had the head of his pancreas removed and he has suffered greatly for over a decade. Almost killed by FDR misinterpretation.
- dan-robertson 3y ago
  I like this example because it doesn’t rely on the US oddities that people often come up with for this topic (eg doctors doing extra, likely negative EV, procedures for fear of litigation).
hollerith 3y ago
No because the doctor can be successfully sued for not ordering the followup even when the followup would not have been in the patient's best interests. I.e., juries are irrational.

tptacek 3y ago

How would it be? Again, the base rates of a lot of these conditions are very low, but not low enough to say "this is a zebra, not a horse", so I think a suspicious lump is likely to trigger a CT, PET, or (in the worst case) a biopsy.

jstummbillig 3y ago
I don't understand what "base rates" means in this context. If you have a test result, that you think is showing something with a certain probability, which is higher then the current necessary confidence level required to trigger further probing, and it turns out afterwards it was a false positive, my naive way to go about this would be to raise the minimum confidence level afterwards (and vice versa if it's a true positive) Does this fail in horrible ways that I am missing right now?
- tptacek 3y ago
  Base rate: select a person at random from the population; how likely is a true positive?

SketchySeaBeast 3y ago

What confidence level? Isn't this advocating for the patient to make their own choices around elective scans and procedures?

robwwilliams 3y ago

Hard to say. Consider a patient without complaints with a scan that shows an abnormal bleb in or on the head of the pancreas. Physician 1 thinks it could be a pancreatic tumor given patient age and family history. Physician 2 thinks it is more likely to be a minor lifelong development anomaly with the bleb actually being an intrusive bit of the spleen. Is exploratory surgery needed? Watchful waiting? An acquaintance of mine with the second harmless condition had the head of his pancreas removed and he has suffered greatly for over a decade. Almost killed by FDR misinterpretation.

dan-robertson 3y ago
I like this example because it doesn’t rely on the US oddities that people often come up with for this topic (eg doctors doing extra, likely negative EV, procedures for fear of litigation).

hollerith 3y ago

No because the doctor can be successfully sued for not ordering the followup even when the followup would not have been in the patient's best interests. I.e., juries are irrational.