3 ms·

Obviously this algorithm has lots of false positives.

by utopkara 15y ago

politician 15y ago
Why is that obvious?
- pwhelan 15y ago
  A false positive costs much lower than a false negative in this instance so it should follow that FP should be more common. However, without the data it is still speculation.
  - carbocation 15y ago
    Whether false positives or false negatives are more harmful is a difficult question. A false positive might stop millions from taking a life-saving drug. A false negative might cause millions to take a life-threatening drug.
    - pwhelan 15y ago
      By nature a false positive interaction could be re-tested more easily than a false negative though, correct? Also, that fits in with medical "first, do no harm" framework more readily.
      - carbocation 15y ago
        Regarding your second point, I agree. But it must be balanced against the other tenets of medical ethics. At any rate, this seems to be an important study. I look forward to seeing the same study carried out across different EMR databases and then meta-analyzed. That will be extremely powerful.
- rsheridan6 15y ago
  Typically scientists try to reject the null hypothesis (in this case, that a particular pair of drugs do not interact) by showing that it's unlikely (less than a 5% chance) that an apparent interaction is caused by random chance. But if you go looking at millions of possible interactions, you're going to find a lot that seem to meet that criteria by chance. 5% of a million is 50,000. 0.1% of a million is 1,000. That's why drug companies have to prespecify the outcomes they're looking for before starting the study. If you look through your data after the fact, you're going to find something that makes your drug look good just by chance.
- Estragon 15y ago
  An association should be treated with suspicion until it has been replicated in an independent sample. Even then, the association is based on deviation from a simple null model which is little more than a straw man, so its causal basis is obscure. The authors have tried to correct for this by identifying patients with similar characteristics, but you can do so much of that before you destroy your statistical power.

Obviously this algorithm has lots of false positives.

politician 15y ago

Why is that obvious?

pwhelan 15y ago
A false positive costs much lower than a false negative in this instance so it should follow that FP should be more common. However, without the data it is still speculation.
- carbocation 15y ago
  Whether false positives or false negatives are more harmful is a difficult question. A false positive might stop millions from taking a life-saving drug. A false negative might cause millions to take a life-threatening drug.
  - pwhelan 15y ago
    By nature a false positive interaction could be re-tested more easily than a false negative though, correct? Also, that fits in with medical "first, do no harm" framework more readily.
    - carbocation 15y ago
      Regarding your second point, I agree. But it must be balanced against the other tenets of medical ethics. At any rate, this seems to be an important study. I look forward to seeing the same study carried out across different EMR databases and then meta-analyzed. That will be extremely powerful.
rsheridan6 15y ago
Typically scientists try to reject the null hypothesis (in this case, that a particular pair of drugs do not interact) by showing that it's unlikely (less than a 5% chance) that an apparent interaction is caused by random chance. But if you go looking at millions of possible interactions, you're going to find a lot that seem to meet that criteria by chance. 5% of a million is 50,000. 0.1% of a million is 1,000. That's why drug companies have to prespecify the outcomes they're looking for before starting the study. If you look through your data after the fact, you're going to find something that makes your drug look good just by chance.
Estragon 15y ago
An association should be treated with suspicion until it has been replicated in an independent sample. Even then, the association is based on deviation from a simple null model which is little more than a straw man, so its causal basis is obscure. The authors have tried to correct for this by identifying patients with similar characteristics, but you can do so much of that before you destroy your statistical power.