3 ms·
Simpson's Paradox is deeper than conventional analyses suggest. I'm going to use the pilots'-late-arriving-flights data: name | delay% alice | 30% bob | 20%
by scythe 13y ago
Simpson's Paradox is deeper than conventional analyses suggest. I'm going to use the pilots'-late-arriving-flights data:
name | delay%
alice | 30%
bob | 20%
So Bob's flights are delayed less often, he's the better pilot, right? Yet:
name | night | day
alice | 7/25 | 1/5
bob | 3/10 | 3/20
So now Alice looks like the better pilot!
But wait, what if the pilots are responsible for scheduling their own flights? Bob's individual batting averages might be somewhat worse than Alice's, but he's making better decisions about when to fly.
But wait, what if Alice and Bob fly out of the same airport(s), and they've agreed to let Bob fly during the day... (this is not meant to be an accurate representation of air traffic control)
When faced with a Simpson-like situation, a correct analysis usually requires considering the chain of causation, in particular, whether any stratification of the data depends on the independent variable being tested. If the stratification is a result of the test variable, it usually isn't a good one. In the Berkeley example, this possibility is the highly unlikely situation that applicants were automatically assigned to departments with gender taken into consideration -- so the stratification was valid.
- DanBC 13y agoname | delay% alice | 30% bob | 20% So Bob's flights are delayed less often, he's the better pilot, right? Yet: name | night | day alice | 7/25 | 1/5 bob | 3/10 | 3/20
- ronaldx 13y agoIn the past, I've considered Simpson's paradox to be nothing more than an amusing quirk of statistics. But, the parent example has helped me to see that Simpson's paradox is frighteningly inevitable. For example, a conclusion like: "People taking a particular drug have worse outcomes." doesn't reflect on the efficacy of the drug at all - because people choosing to take the drug are presumably in greater need. Same in the piloting example above - Alice is scoring worse only because she has the more difficult assignments (perhaps because she is indeed the better pilot).
- baddox 13y agoIt really is a particularly pathological special case of "correlation does not imply causation." Often the key is the presence of a confounding variable.
- steveridout 13y agoI don't see how the second table makes Alice look like the better pilot. She's better than Bob in the night, but worse than Bob in the day, and overall worse. Could someone explain?