4 ms·

Could you elaborate on this point "no way", for more observations?

6gvONxR4sf7o 6y ago
Think about getting a helicopter to the hospital. It's probably pretty serious, right? You're more likely to die if you get a helicopter evac than if you don't. If all you have recorded is whether they got to the hospital via helicopter or not, and whether they died or not, then you're going to see that helicopters are associated with death. If you could control for what's wrong with the person, you might see that helicopters save lives. If you don't have that data, then just having more observations of (helicopter y/n, died y/n) will just look like helivac is murder. No matter what you do with that data, you just can't control for what you'd need to, because what you need was an unobserved confounder. When you're trying to establish causation, you have to rule out unobserved confounders, which is tricky. There's always something where someone might say "well what if it was X?" and the data doesn't contain that answer. In this particular case, that commenter said that conference dates are independent enough from patient health to say that aside from the important things, everything else is equal. As in, the only difference between the treatments is who is doing the treatment. I disagree with that assertion, but that's the kind of argument you need for causation: Once we've dealt with/controlled for XYZ, the only difference left is the one we're interested in. It's very difficult to demonstrate that the only difference left is what you care about when you can't even see certain variables. Someone says "what if people in group A are more likely to be left handed and it's patient handedness instead of doctor quality that's causing death here" but you didn't measure left handedness. More and more observations of group A without measuring handedness can't rule out that maybe they were lefties. So you either measure handedness, or argue that your setup will even it out (via randomization for example), or argue that handedness just doesn't matter here. And not just for handedness, but for everything imaginable.
- Retric 6y ago
  While intuitively that reasoning seems valid, having more data makes signals more clear. If your sample sizes is say 99.8% of all doctors that’s going to be representative in ways a carefully crafted representative sample of 1,000 doctors simply isn’t. Sampling always loses out to having the entire population. But you say missing data makes the interpretation incorrect. That’s always possible but only in ways that also fit the original data. Thus, you can then preform a study that improves upon the understanding of the correlation, but it’s not going to disprove the original correlation. PS: And even here it’s easy to test with more data, just compare total deaths for the month around conferences.

Think about getting a helicopter to the hospital. It's probably pretty serious, right? You're more likely to die if you get a helicopter evac than if you don't. If all you have recorded is whether they got to the hospital via helicopter or not, and whether they died or not, then you're going to see that helicopters are associated with death. If you could control for what's wrong with the person, you might see that helicopters save lives. If you don't have that data, then just having more observations of (helicopter y/n, died y/n) will just look like helivac is murder. No matter what you do with that data, you just can't control for what you'd need to, because what you need was an unobserved confounder. When you're trying to establish causation, you have to rule out unobserved confounders, which is tricky. There's always something where someone might say "well what if it was X?" and the data doesn't contain that answer. In this particular case, that commenter said that conference dates are independent enough from patient health to say that aside from the important things, everything else is equal. As in, the only difference between the treatments is who is doing the treatment. I disagree with that assertion, but that's the kind of argument you need for causation: Once we've dealt with/controlled for XYZ, the only difference left is the one we're interested in. It's very difficult to demonstrate that the only difference left is what you care about when you can't even see certain variables. Someone says "what if people in group A are more likely to be left handed and it's patient handedness instead of doctor quality that's causing death here" but you didn't measure left handedness. More and more observations of group A without measuring handedness can't rule out that maybe they were lefties. So you either measure handedness, or argue that your setup will even it out (via randomization for example), or argue that handedness just doesn't matter here. And not just for handedness, but for everything imaginable.

Retric 6y ago
While intuitively that reasoning seems valid, having more data makes signals more clear. If your sample sizes is say 99.8% of all doctors that’s going to be representative in ways a carefully crafted representative sample of 1,000 doctors simply isn’t. Sampling always loses out to having the entire population. But you say missing data makes the interpretation incorrect. That’s always possible but only in ways that also fit the original data. Thus, you can then preform a study that improves upon the understanding of the correlation, but it’s not going to disprove the original correlation. PS: And even here it’s easy to test with more data, just compare total deaths for the month around conferences.