8 ms·

Does Causation Imply Correlation?

76 points by Fomite 13y ago

holycow19 13y ago
Wrong verbage here - would suggest seems better?
- Fomite 13y ago
  Probably. One of the answers on there also has a discussion of referring to it as 'correlation', which has a specific statistical meaning that is often ignored when people trot out "Correlation != Causation". A better wordsmithed version of the question would probably be "Does Association Suggest Causation?"
  - sesqu 13y ago
    Correlation does not always mean Pearson's r², which is the thing with a specific statistical meaning. There are other specific correlations, and general correlation.
    - Fomite 13y ago
      Never the less, I think 'association' is a better general term.
TempleOSV2 13y ago
If it is the cause of something, yes it happened. Not sure of rest of logic. Don't care. Might be elephant problem. Did you know there are 16 logical operators. Some operators are complete. NAND is complete but OR is not complete AND is not complete. Usually all we argue is the existence of God. God says... like_like threads really Boo refreshing yep blessing experts who's_to_say heads_I_win_tails_you_lose wazz_up_with_that commanded smack_some_sense_into_you piety I_forgot furious couldn't_be_better You_fix_it these_cans_are_defective my_precious sloth Catastrophic_Success Hasta close_your_eyes you'll_see I_pitty_the_fool whoo_whoo
computer 13y ago
Here's a harder question: is it ever possible to deduce causation based on data alone? We all know that correlation does not imply causation, but is there anything that does? If not: prove it. And if yes, under what conditions exactly?
- pingou 13y ago
  I was wondering the same thing. Imagine we make a study on programmers and observe that 80% of them like science. Then we have a new programmer than wasn't part of the study, can we say he has a 80% chance of liking science? Intuitively I would say yes but I think it's wrong, can someone help?
  - StavrosK 13y ago
    I don't know much more than you, but I'll give it a shot: We can say whatever we like, really. If you mean "is it true", then yes, if our sample was representative of all the programmers and not biased in some way, the new programmer will have a 80% chance of liking science. In reality, though, "80% chance of liking science" is just a best guess. Since all we have to go with is the sample we already studied, we assume it's representative (or try to correct for the bias we know) and make a guess based on that. This has nothing to do with causation, however.
    - tel 13y ago
      I can clarify this. The thing I outline below is a little situation dependent (and matching this hypothetical) but the intuition is generally correct and amenable with yours. One way to see all the pieces is to recognize that at least in theory there are real answers to all of these questions which we are approximating. We could literally round up every "programmer" and ask them whether or not they "like science" and then get a completely accurate statistic that, say, "92% of programmers like science". But that's infeasible (even buying that my quoted words could make literal sense) so instead we run a finite study. This is called taking the subpopulation of a superpopulation and drawing the statistic from that subpopulation. Now we see that in our subpopulation of N participant-programmers, 80% of them liked science. Now let's bring in a new guy. Let's say we brought him in by taking every programmer in the superpopulation (replacing our sampled subpopulation back into the global pool) and picking one guy in a lottery. Provided that was a truly random lottery then we can use the 92% number from before and know that there's a 92% chance that our lottery picked a science-liker. But we don't actually have that information. Instead, we simply have an estimate to 92%, our subpopulation statistic of 80%. If our error bars on guessing incorrectly are fine with a 12% discrepancy then we're still golden, though. But there's one final element—we want to be able to talk about our sampling process and the risks of using it. The only parameter of this process is the size of the subpopulation chosen. For instance, if we had sampled only 5 people then there's about a 1/4 chance that we'd arrive at an estimate of 80% and a 7/10 chance we'd get 100%. This still leaves around 10% of the possible subpopulations we could choose that would tell us something really misleading, like only 20% of programmers like science. This could be called sample statistic stability, or just straight up estimation error. It's rarely taken into account when people say things like "80% of programmers like science" because the usual assumption is that the sampled subpopulation is "big enough" to minimize this error sufficiently. In truth, though, it's hard to know exactly what the possible error of your sampling method really is. So finally, we want to know that, given we saw 80% of our subpopulation likes science, what's the chance that a new, randomly drawn programmer also likes science. The honest truth is that there's a 92% chance, so we're ignoring some amount of error by taking our study on faith and saying that there's an 80% chance. However, given literally no other information, it's still our best bet to assume there's an 80% chance—it's better than randomly picking any other number based on the data we've seen. [0] Finally, if we were to repeat this whole experience over and over again starting with fresh, random draws of the subpopulation then we'd be, on average, correct in our guesses about the ratio of the super population [1]. [0] Bayesians here would say that we should mix in other information we might have to form a better estimate—I think that makes a lot of sense in a situation like this, especially if our subpopulation is tiny, so I'm trying to be really clear that we have no other information. Let's not talk non-informative priors. [1] i.e. if we keep repeatedly re-estimating our "80% statistic" and using it honestly to make our guesses then it would bounce around based on our subpopulation, sometimes 80%, sometimes 100%, sometimes 20%, etc. If we multiply these estimates by how often they occur and sum it all up then we'll get exactly 92%. We'd get there faster just by drawing a bigger subpopulation, though.
jobigoud 13y ago
Wouldn't a Random Number Generator be a case of causation not implying correlation?
- cheald 13y ago
  If you're talking about PRNGs, then no - there is a very specific correlation between successive numbers in a PRNG.
  - ogrisel 13y ago
    But it's highly non-linear. They have an expected Pearson correlation of zero. Note: there are simple counter example of variables that have a Pearson correlation of 0 yet some non-linear dependency structure as in the 3rd row of the plot of: http://en.wikipedia.org/wiki/Correlation_and_dependence http://en.wikipedia.org/wiki/Correlation_and_dependence
    - mistercow 13y ago
      If the question is "does causation imply linear correlation", then the answer is obviously and trivially "no". But we don't need a PRNG to see that. Take a series of chutes whose entrances are lined up in a row, and whose exits are arranged in a circle. Now start dropping balls in the entrances and see where they end up. Obviously there is a causal relationship between where we drop the ball and where it lands, but it's not a linear correlation. But there's still a correlation.
- mistercow 13y ago
  In this example, what variables are you talking about being causally related?
  - ogrisel 13y ago
    The integer seeds of the PRNG on hand and the first (or second or nth) generated pseudo random number on the other hand.
RamiK 13y ago
Most of the time...
anon4 13y ago
My favourite counter-example is the one with a thermostat. If someone controls the thermostat so that the temperature in the room stays constant no matter the temperature outside, then you record the temperature inside and the thermostat setting over many days, the two series will be completely uncorrelated. But you will have a perfect correlation between thermostat setting and outside temperature.
- emiliobumachar 13y ago
  " But you will have a perfect correlation between thermostat setting and outside temperature." You meant a perfect causation between thermostat setting and inside temperature, right?
  - josephlord 13y ago
    The correlation will be between the thermostat output and the outside temperature. The thermostat output (combined with outside temperature) are the causation for inside temperature. There should also be a correlation and causation between thermostat setting and inside temperature.
  - anon4 13y ago
    Well yes, but I also meant what I wrote. When the temperature outside goes down, the thermostat goes up and the other way around. They are perfectly correlated, even though you can't really say one causes the other (outside temperature does in a very roundabout way cause thermostat setting). I like that example because it counters both "causation implies correlation" and "correlation implies causation".
    - baddox 13y ago
      I'm confused. Don't you simply set a thermostat to the desired indoor temperature and leave it? Assuming the desired indoor temperature is constant, I would expect the thermostat setting to be constant, the internal temperature to be nearly constant, the outdoor temperature to be uncorrelated, and the energy usage of the heating/cooling system to be correlated with the outdoor temperature (namely, the difference between outdoor temperature and desired internal temperature).
smoyer 13y ago
Does a headline with two HN memes get extra points?
conjectures 13y ago
This comes down to vocabulary. If by correlation we mean Pearson correlation (linearly related) the answer is no. If by correlation we mean some hand-waving association between A and B then yes. Since if A causes B there is some hand-waving association between them, namely causation. I suggest we reserve 'correlation' for linear relationships and stop using it in the second sense. It's unhelpful and confusing.
- mbq 13y ago
  Two problems: first is that casual association may be invisible to even hand-waving criteria; think good crypto PRNG output which is fully determined by seed and is serial number, but the parameters are practically unfittable from the data because the function is so chaotic. Second thing is that the more hand-waving you allow the more totally incidental relations will look "true". With a modern amounts of hypotheses, even Pearson correlation often becomes useless in this manner.
  - conjectures 13y ago
    mbq, I would not contradict your points. Though first sense 'correlation' could well be useless in many cases at least I know what it means. The second sense is perfidious as it's virtually impossible to deny there's a 'correlation' between any two things since the term so poorly defined.
    - mbq 13y ago
      Note that I am criticising the idea of "general" or "intuitive" correlation; actually useful correlation measures (like Pearson or Spearman) will be just always tied to some model of dependence. This way it cannot be directly tied to the obviously absolute dependence.
jeffcox 13y ago
"Correlation doesn't imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing 'look over there.'" http://xkcd.com/552/ http://xkcd.com/552/

76 points by Fomite 13y ago

holycow19 13y ago

Wrong verbage here - would suggest seems better?

Fomite 13y ago
Probably. One of the answers on there also has a discussion of referring to it as 'correlation', which has a specific statistical meaning that is often ignored when people trot out "Correlation != Causation". A better wordsmithed version of the question would probably be "Does Association Suggest Causation?"
- sesqu 13y ago
  Correlation does not always mean Pearson's r², which is the thing with a specific statistical meaning. There are other specific correlations, and general correlation.
  - Fomite 13y ago
    Never the less, I think 'association' is a better general term.

TempleOSV2 13y ago

If it is the cause of something, yes it happened. Not sure of rest of logic. Don't care. Might be elephant problem. Did you know there are 16 logical operators. Some operators are complete. NAND is complete but OR is not complete AND is not complete. Usually all we argue is the existence of God. God says... like_like threads really Boo refreshing yep blessing experts who's_to_say heads_I_win_tails_you_lose wazz_up_with_that commanded smack_some_sense_into_you piety I_forgot furious couldn't_be_better You_fix_it these_cans_are_defective my_precious sloth Catastrophic_Success Hasta close_your_eyes you'll_see I_pitty_the_fool whoo_whoo

computer 13y ago

Here's a harder question: is it ever possible to deduce causation based on data alone? We all know that correlation does not imply causation, but is there anything that does? If not: prove it. And if yes, under what conditions exactly?

pingou 13y ago
I was wondering the same thing. Imagine we make a study on programmers and observe that 80% of them like science. Then we have a new programmer than wasn't part of the study, can we say he has a 80% chance of liking science? Intuitively I would say yes but I think it's wrong, can someone help?
- StavrosK 13y ago
  I don't know much more than you, but I'll give it a shot: We can say whatever we like, really. If you mean "is it true", then yes, if our sample was representative of all the programmers and not biased in some way, the new programmer will have a 80% chance of liking science. In reality, though, "80% chance of liking science" is just a best guess. Since all we have to go with is the sample we already studied, we assume it's representative (or try to correct for the bias we know) and make a guess based on that. This has nothing to do with causation, however.
  - tel 13y ago
    I can clarify this. The thing I outline below is a little situation dependent (and matching this hypothetical) but the intuition is generally correct and amenable with yours. One way to see all the pieces is to recognize that at least in theory there are real answers to all of these questions which we are approximating. We could literally round up every "programmer" and ask them whether or not they "like science" and then get a completely accurate statistic that, say, "92% of programmers like science". But that's infeasible (even buying that my quoted words could make literal sense) so instead we run a finite study. This is called taking the subpopulation of a superpopulation and drawing the statistic from that subpopulation. Now we see that in our subpopulation of N participant-programmers, 80% of them liked science. Now let's bring in a new guy. Let's say we brought him in by taking every programmer in the superpopulation (replacing our sampled subpopulation back into the global pool) and picking one guy in a lottery. Provided that was a truly random lottery then we can use the 92% number from before and know that there's a 92% chance that our lottery picked a science-liker. But we don't actually have that information. Instead, we simply have an estimate to 92%, our subpopulation statistic of 80%. If our error bars on guessing incorrectly are fine with a 12% discrepancy then we're still golden, though. But there's one final element—we want to be able to talk about our sampling process and the risks of using it. The only parameter of this process is the size of the subpopulation chosen. For instance, if we had sampled only 5 people then there's about a 1/4 chance that we'd arrive at an estimate of 80% and a 7/10 chance we'd get 100%. This still leaves around 10% of the possible subpopulations we could choose that would tell us something really misleading, like only 20% of programmers like science. This could be called sample statistic stability, or just straight up estimation error. It's rarely taken into account when people say things like "80% of programmers like science" because the usual assumption is that the sampled subpopulation is "big enough" to minimize this error sufficiently. In truth, though, it's hard to know exactly what the possible error of your sampling method really is. So finally, we want to know that, given we saw 80% of our subpopulation likes science, what's the chance that a new, randomly drawn programmer also likes science. The honest truth is that there's a 92% chance, so we're ignoring some amount of error by taking our study on faith and saying that there's an 80% chance. However, given literally no other information, it's still our best bet to assume there's an 80% chance—it's better than randomly picking any other number based on the data we've seen. [0] Finally, if we were to repeat this whole experience over and over again starting with fresh, random draws of the subpopulation then we'd be, on average, correct in our guesses about the ratio of the super population [1]. [0] Bayesians here would say that we should mix in other information we might have to form a better estimate—I think that makes a lot of sense in a situation like this, especially if our subpopulation is tiny, so I'm trying to be really clear that we have no other information. Let's not talk non-informative priors. [1] i.e. if we keep repeatedly re-estimating our "80% statistic" and using it honestly to make our guesses then it would bounce around based on our subpopulation, sometimes 80%, sometimes 100%, sometimes 20%, etc. If we multiply these estimates by how often they occur and sum it all up then we'll get exactly 92%. We'd get there faster just by drawing a bigger subpopulation, though.

jobigoud 13y ago

Wouldn't a Random Number Generator be a case of causation not implying correlation?

cheald 13y ago
If you're talking about PRNGs, then no - there is a very specific correlation between successive numbers in a PRNG.
- ogrisel 13y ago
  But it's highly non-linear. They have an expected Pearson correlation of zero. Note: there are simple counter example of variables that have a Pearson correlation of 0 yet some non-linear dependency structure as in the 3rd row of the plot of: http://en.wikipedia.org/wiki/Correlation_and_dependence http://en.wikipedia.org/wiki/Correlation_and_dependence
  - mistercow 13y ago
    If the question is "does causation imply linear correlation", then the answer is obviously and trivially "no". But we don't need a PRNG to see that. Take a series of chutes whose entrances are lined up in a row, and whose exits are arranged in a circle. Now start dropping balls in the entrances and see where they end up. Obviously there is a causal relationship between where we drop the ball and where it lands, but it's not a linear correlation. But there's still a correlation.
mistercow 13y ago
In this example, what variables are you talking about being causally related?
- ogrisel 13y ago
  The integer seeds of the PRNG on hand and the first (or second or nth) generated pseudo random number on the other hand.

RamiK 13y ago

Most of the time...

anon4 13y ago

My favourite counter-example is the one with a thermostat. If someone controls the thermostat so that the temperature in the room stays constant no matter the temperature outside, then you record the temperature inside and the thermostat setting over many days, the two series will be completely uncorrelated. But you will have a perfect correlation between thermostat setting and outside temperature.

emiliobumachar 13y ago
" But you will have a perfect correlation between thermostat setting and outside temperature." You meant a perfect causation between thermostat setting and inside temperature, right?
- josephlord 13y ago
  The correlation will be between the thermostat output and the outside temperature. The thermostat output (combined with outside temperature) are the causation for inside temperature. There should also be a correlation and causation between thermostat setting and inside temperature.
- anon4 13y ago
  Well yes, but I also meant what I wrote. When the temperature outside goes down, the thermostat goes up and the other way around. They are perfectly correlated, even though you can't really say one causes the other (outside temperature does in a very roundabout way cause thermostat setting). I like that example because it counters both "causation implies correlation" and "correlation implies causation".
  - baddox 13y ago
    I'm confused. Don't you simply set a thermostat to the desired indoor temperature and leave it? Assuming the desired indoor temperature is constant, I would expect the thermostat setting to be constant, the internal temperature to be nearly constant, the outdoor temperature to be uncorrelated, and the energy usage of the heating/cooling system to be correlated with the outdoor temperature (namely, the difference between outdoor temperature and desired internal temperature).

smoyer 13y ago

Does a headline with two HN memes get extra points?

conjectures 13y ago

This comes down to vocabulary. If by correlation we mean Pearson correlation (linearly related) the answer is no. If by correlation we mean some hand-waving association between A and B then yes. Since if A causes B there is some hand-waving association between them, namely causation. I suggest we reserve 'correlation' for linear relationships and stop using it in the second sense. It's unhelpful and confusing.

mbq 13y ago
Two problems: first is that casual association may be invisible to even hand-waving criteria; think good crypto PRNG output which is fully determined by seed and is serial number, but the parameters are practically unfittable from the data because the function is so chaotic. Second thing is that the more hand-waving you allow the more totally incidental relations will look "true". With a modern amounts of hypotheses, even Pearson correlation often becomes useless in this manner.
- conjectures 13y ago
  mbq, I would not contradict your points. Though first sense 'correlation' could well be useless in many cases at least I know what it means. The second sense is perfidious as it's virtually impossible to deny there's a 'correlation' between any two things since the term so poorly defined.
  - mbq 13y ago
    Note that I am criticising the idea of "general" or "intuitive" correlation; actually useful correlation measures (like Pearson or Spearman) will be just always tied to some model of dependence. This way it cannot be directly tied to the obviously absolute dependence.

jeffcox 13y ago

"Correlation doesn't imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing 'look over there.'" http://xkcd.com/552/ http://xkcd.com/552/