4 ms·

This visualization on Wikipedia was what I needed to understand Simpson's Paradox, the descriptions never made a whole lot of sense to me until seeing it like t

by Izkata 2y ago

This visualization on Wikipedia was what I needed to understand Simpson's Paradox, the descriptions never made a whole lot of sense to me until seeing it like this: https://en.wikipedia.org/wiki/Simpson%27s_paradox#/media/File:Simpsons_paradox_-_animation.gif https://en.wikipedia.org/wiki/Simpson%27s_paradox#/media/Fil... Along the same lines of "visualize your data to see what's really going on" is Anscombe's Quartet: https://en.wikipedia.org/w/index.php?title=Anscombe%27s_quartet https://en.wikipedia.org/w/index.php?title=Anscombe%27s_quar... And then there's the Datasaurus [Dozen], which has some fun with the idea behind Anscombe's Quartet: https://en.wikipedia.org/wiki/Datasaurus_dozen https://en.wikipedia.org/wiki/Datasaurus_dozen (you can see it animated here: https://blog.revolutionanalytics.com/2017/05/the-datasaurus-dozen.html https://blog.revolutionanalytics.com/2017/05/the-datasaurus-... )

mjburgess 2y ago
A more general lesson is that correlation and causation are unrelated: the former doesnt imply the latter, and the latter does not imply the former. Just because one thing causes another does not mean it will be correlated with it. There is no contradiction in subsets having different correlations that the parent set. The apparent "paradox" arises from reading the data causally. The purpose of this lesson is to expose these assumptions in interpretation of data. Few seem to get the message though.
- weinzierl 2y ago
  If anyone has doubts about the second claim, think about a hash function. The input certainly causes the output, but they are not correlated in a statistical sense.
  - mjburgess 2y ago
    Consider a medicine which kills everything with kidneys. It perfectly correlates with killing everything with a liver. Consider another medicine which kills everything with kidneys, unless they have a liver (eg., which filters it). Now there is no correlation at all with an effect on the kidneys, nor will there ever be (since all animals with one have the other) unless someone deliberately impairs a liver.
    - Anotheroneagain 2y ago
      Consider a medicine that completely cures Alzheimer's. It also necessarily increases the incidence of other causes of death, as those who won't die of Alzheimer's will die of something else instead.
- kqr 2y ago
  > A more general lesson is that correlation and causation are unrelated This is a bit extreme. The tongue-in-cheek variant I like (which I first read about in the book referenced by TFA) is "no correlation without causation". In order for two things to truly co-vary (and not just by accident, or as a consequence of poor data collection/manipulation), there needs to be some causal connection between the two – although it can be quite distant.
  - mjburgess 2y ago
    > there needs to be some causal connection between the two Umm.. No, there doesn't... This idea features in the earlist 20th C. writings on statistics, but it's pseudoscience. If one carves up the whole history of the entire universe into all possible events, then there's likely to be a (near) infinite number of pairs of events which "perfectly" co-vary without any causal connection whatsoever. Indeed, one could find two galaxies that are necessarily causally isolated and find correlated events. This is, in part, because the properties of two casually independent systems can have indistinguishable distributions -- just by the nature of what a distribution is. It's this sort of thinking that I was aiming to rule out: really they have nothing to do with each other. It's early 20th C. frequentist pseudoscience that has given birth to this supposed connection, and it should be thrown out all together. Causation is a property of natural systems. Correlation is a property of two distributions. These have nothing to do with each other. If you want to "test" for causation, you need to have a causal theory and a causal analysis in which "correlation" shouldn't feature. If you induce correlation by causal intervention, i'd prefer we gave that a different name ("induced correlation") which is relevant to causation -- and it's mostly this confusion which those early eugenticists that created statistics were talking about.
    - Izkata 2y ago
      Examples: https://www.tylervigen.com/spurious-correlations https://www.tylervigen.com/spurious-correlations
    - kqr 2y ago
      > If one carves up the whole history of the entire universe into all possible events, then there's likely to be a (near) infinite number of pairs of events which "perfectly" co-vary without any causal connection whatsoever. But if they are not linked by a stable causal connection, wouldn't they eventually diverge, if we observe long enough?
- godelski 2y ago
  > correlation and causation are unrelated This is incorrect (but what followed is correct). You have extended the meaning of the phrase "correlation does not imply causation" to a stronger case[0]. The correct way to say this is that "correlation are not necessarily related." The other way you might determine this was wrong is that ,,association''[2] always occurs when there is causation. So we have the classic A ⇒ B ⇏ B ⇒ A (A implies B does not imply B implies A), where ordering matters. Last, we should reference Judea Pearl's Ladder of Causality[1]. [0] Another similar example was given to us by Rumsfield with respect to the Iraq WMD search. Where the error was changing "the absence of proof is not proof of absence" to the much stronger "the absence of evidence is not evidence of absence". It also illustrates why we might want to "nitpick" here https://archive.is/20140823194745/http://logbase2.blogspot.com/2009/01/absence-of-evidence.html https://archive.is/20140823194745/http://logbase2.blogspot.c... [1] https://web.cs.ucla.edu/~kaoru/3-layer-causal-hierarchy.pdf https://web.cs.ucla.edu/~kaoru/3-layer-causal-hierarchy.pdf [2] Edit for clarity: The reason I (and Pearl) use the word "association" rather than "correlation" is because in statistics "correlation" often refers to linear relationship. So association clarifies that there is mutual information. There might be masked relationships, so non-linear. But if we are to use the standard vernacular of "correlation" (what most people think) then we could correctly say "causation implies correlation" (or more accurately, "causation implies correlation, but not necessarily linear correlation."). And of course, causation implies high mutual information, but high mutual information does not imply causation :) https://stats.stackexchange.com/questions/26300/does-causation-imply-correlation https://stats.stackexchange.com/questions/26300/does-causati...
  - mjburgess 2y ago
    > ,association''[2] always occurs when there is causation This is incorrect. See Perl's work itself. Association does not occur when there is a collider. https://en.wikipedia.org/wiki/Collider_(statistics) https://en.wikipedia.org/wiki/Collider_(statistics) Since almost all variables we are measuring are on uncontrolled environments, in almost all cases, there is an opportunity to observe no association with causation. I give an example of this above: > Consider another medicine which kills everything with kidneys, unless they have a liver (eg., which filters it). Now there is no correlation at all with an effect on the kidneys, nor will there ever be (since all animals with one have the other) unless someone deliberately impairs a liver.
wodenokoto 2y ago
I'm not sure I understand how that would work if x is time, unless each category is sampled from different time spans.
Leszek 2y ago
I've always been partial to this visual: https://analyticsindiamag.com/wp-content/uploads/2018/12/simpsons-paradox.jpg https://analyticsindiamag.com/wp-content/uploads/2018/12/sim...

by Izkata 2y ago

mjburgess 2y ago

A more general lesson is that correlation and causation are unrelated: the former doesnt imply the latter, and the latter does not imply the former. Just because one thing causes another does not mean it will be correlated with it. There is no contradiction in subsets having different correlations that the parent set. The apparent "paradox" arises from reading the data causally. The purpose of this lesson is to expose these assumptions in interpretation of data. Few seem to get the message though.

weinzierl 2y ago
If anyone has doubts about the second claim, think about a hash function. The input certainly causes the output, but they are not correlated in a statistical sense.
- mjburgess 2y ago
  Consider a medicine which kills everything with kidneys. It perfectly correlates with killing everything with a liver. Consider another medicine which kills everything with kidneys, unless they have a liver (eg., which filters it). Now there is no correlation at all with an effect on the kidneys, nor will there ever be (since all animals with one have the other) unless someone deliberately impairs a liver.
  - Anotheroneagain 2y ago
    Consider a medicine that completely cures Alzheimer's. It also necessarily increases the incidence of other causes of death, as those who won't die of Alzheimer's will die of something else instead.
kqr 2y ago
> A more general lesson is that correlation and causation are unrelated This is a bit extreme. The tongue-in-cheek variant I like (which I first read about in the book referenced by TFA) is "no correlation without causation". In order for two things to truly co-vary (and not just by accident, or as a consequence of poor data collection/manipulation), there needs to be some causal connection between the two – although it can be quite distant.
- mjburgess 2y ago
  > there needs to be some causal connection between the two Umm.. No, there doesn't... This idea features in the earlist 20th C. writings on statistics, but it's pseudoscience. If one carves up the whole history of the entire universe into all possible events, then there's likely to be a (near) infinite number of pairs of events which "perfectly" co-vary without any causal connection whatsoever. Indeed, one could find two galaxies that are necessarily causally isolated and find correlated events. This is, in part, because the properties of two casually independent systems can have indistinguishable distributions -- just by the nature of what a distribution is. It's this sort of thinking that I was aiming to rule out: really they have nothing to do with each other. It's early 20th C. frequentist pseudoscience that has given birth to this supposed connection, and it should be thrown out all together. Causation is a property of natural systems. Correlation is a property of two distributions. These have nothing to do with each other. If you want to "test" for causation, you need to have a causal theory and a causal analysis in which "correlation" shouldn't feature. If you induce correlation by causal intervention, i'd prefer we gave that a different name ("induced correlation") which is relevant to causation -- and it's mostly this confusion which those early eugenticists that created statistics were talking about.
  - Izkata 2y ago
    Examples: https://www.tylervigen.com/spurious-correlations https://www.tylervigen.com/spurious-correlations
  - kqr 2y ago
    > If one carves up the whole history of the entire universe into all possible events, then there's likely to be a (near) infinite number of pairs of events which "perfectly" co-vary without any causal connection whatsoever. But if they are not linked by a stable causal connection, wouldn't they eventually diverge, if we observe long enough?
godelski 2y ago
> correlation and causation are unrelated This is incorrect (but what followed is correct). You have extended the meaning of the phrase "correlation does not imply causation" to a stronger case[0]. The correct way to say this is that "correlation are not necessarily related." The other way you might determine this was wrong is that ,,association''[2] always occurs when there is causation. So we have the classic A ⇒ B ⇏ B ⇒ A (A implies B does not imply B implies A), where ordering matters. Last, we should reference Judea Pearl's Ladder of Causality[1]. [0] Another similar example was given to us by Rumsfield with respect to the Iraq WMD search. Where the error was changing "the absence of proof is not proof of absence" to the much stronger "the absence of evidence is not evidence of absence". It also illustrates why we might want to "nitpick" here https://archive.is/20140823194745/http://logbase2.blogspot.com/2009/01/absence-of-evidence.html https://archive.is/20140823194745/http://logbase2.blogspot.c... [1] https://web.cs.ucla.edu/~kaoru/3-layer-causal-hierarchy.pdf https://web.cs.ucla.edu/~kaoru/3-layer-causal-hierarchy.pdf [2] Edit for clarity: The reason I (and Pearl) use the word "association" rather than "correlation" is because in statistics "correlation" often refers to linear relationship. So association clarifies that there is mutual information. There might be masked relationships, so non-linear. But if we are to use the standard vernacular of "correlation" (what most people think) then we could correctly say "causation implies correlation" (or more accurately, "causation implies correlation, but not necessarily linear correlation."). And of course, causation implies high mutual information, but high mutual information does not imply causation :) https://stats.stackexchange.com/questions/26300/does-causation-imply-correlation https://stats.stackexchange.com/questions/26300/does-causati...
- mjburgess 2y ago
  > ,association''[2] always occurs when there is causation This is incorrect. See Perl's work itself. Association does not occur when there is a collider. https://en.wikipedia.org/wiki/Collider_(statistics) https://en.wikipedia.org/wiki/Collider_(statistics) Since almost all variables we are measuring are on uncontrolled environments, in almost all cases, there is an opportunity to observe no association with causation. I give an example of this above: > Consider another medicine which kills everything with kidneys, unless they have a liver (eg., which filters it). Now there is no correlation at all with an effect on the kidneys, nor will there ever be (since all animals with one have the other) unless someone deliberately impairs a liver.

wodenokoto 2y ago

I'm not sure I understand how that would work if x is time, unless each category is sampled from different time spans.

Leszek 2y ago

I've always been partial to this visual: https://analyticsindiamag.com/wp-content/uploads/2018/12/simpsons-paradox.jpg https://analyticsindiamag.com/wp-content/uploads/2018/12/sim...