5 ms·

Here's a harder question: is it ever possible to deduce causation based on data alone? We all know that correlation does not imply causation, but is there anyth

Here's a harder question: is it ever possible to deduce causation based on data alone? We all know that correlation does not imply causation, but is there anything that does? If not: prove it. And if yes, under what conditions exactly?

pingou 13y ago
I was wondering the same thing. Imagine we make a study on programmers and observe that 80% of them like science. Then we have a new programmer than wasn't part of the study, can we say he has a 80% chance of liking science? Intuitively I would say yes but I think it's wrong, can someone help?
- StavrosK 13y ago
  I don't know much more than you, but I'll give it a shot: We can say whatever we like, really. If you mean "is it true", then yes, if our sample was representative of all the programmers and not biased in some way, the new programmer will have a 80% chance of liking science. In reality, though, "80% chance of liking science" is just a best guess. Since all we have to go with is the sample we already studied, we assume it's representative (or try to correct for the bias we know) and make a guess based on that. This has nothing to do with causation, however.
  - tel 13y ago
    I can clarify this. The thing I outline below is a little situation dependent (and matching this hypothetical) but the intuition is generally correct and amenable with yours. One way to see all the pieces is to recognize that at least in theory there are real answers to all of these questions which we are approximating. We could literally round up every "programmer" and ask them whether or not they "like science" and then get a completely accurate statistic that, say, "92% of programmers like science". But that's infeasible (even buying that my quoted words could make literal sense) so instead we run a finite study. This is called taking the subpopulation of a superpopulation and drawing the statistic from that subpopulation. Now we see that in our subpopulation of N participant-programmers, 80% of them liked science. Now let's bring in a new guy. Let's say we brought him in by taking every programmer in the superpopulation (replacing our sampled subpopulation back into the global pool) and picking one guy in a lottery. Provided that was a truly random lottery then we can use the 92% number from before and know that there's a 92% chance that our lottery picked a science-liker. But we don't actually have that information. Instead, we simply have an estimate to 92%, our subpopulation statistic of 80%. If our error bars on guessing incorrectly are fine with a 12% discrepancy then we're still golden, though. But there's one final element—we want to be able to talk about our sampling process and the risks of using it. The only parameter of this process is the size of the subpopulation chosen. For instance, if we had sampled only 5 people then there's about a 1/4 chance that we'd arrive at an estimate of 80% and a 7/10 chance we'd get 100%. This still leaves around 10% of the possible subpopulations we could choose that would tell us something really misleading, like only 20% of programmers like science. This could be called sample statistic stability, or just straight up estimation error. It's rarely taken into account when people say things like "80% of programmers like science" because the usual assumption is that the sampled subpopulation is "big enough" to minimize this error sufficiently. In truth, though, it's hard to know exactly what the possible error of your sampling method really is. So finally, we want to know that, given we saw 80% of our subpopulation likes science, what's the chance that a new, randomly drawn programmer also likes science. The honest truth is that there's a 92% chance, so we're ignoring some amount of error by taking our study on faith and saying that there's an 80% chance. However, given literally no other information, it's still our best bet to assume there's an 80% chance—it's better than randomly picking any other number based on the data we've seen. [0] Finally, if we were to repeat this whole experience over and over again starting with fresh, random draws of the subpopulation then we'd be, on average, correct in our guesses about the ratio of the super population [1]. [0] Bayesians here would say that we should mix in other information we might have to form a better estimate—I think that makes a lot of sense in a situation like this, especially if our subpopulation is tiny, so I'm trying to be really clear that we have no other information. Let's not talk non-informative priors. [1] i.e. if we keep repeatedly re-estimating our "80% statistic" and using it honestly to make our guesses then it would bounce around based on our subpopulation, sometimes 80%, sometimes 100%, sometimes 20%, etc. If we multiply these estimates by how often they occur and sum it all up then we'll get exactly 92%. We'd get there faster just by drawing a bigger subpopulation, though.
- tel 13y ago
  I responded at length to this elsewhere, but quite simply this isn't causation yet. This statement is entirely about the information in our head, not the effect of actions on the world. The causal statement would be "training to become a programmer makes you like science more" and it already begins to indicate that we'd want to observe people who both do and do not decide to become programmers and question their preferences over time in order to have relevant information.
a1369209993 13y ago
http://lesswrong.com/lw/qr/timeless_causality/ http://lesswrong.com/lw/qr/timeless_causality/
- deleted 13y ago
  [deleted]
zmk_ 13y ago
You can use instrumental variables to distinguish between correlation and causation.
hessenwolf 13y ago
http://ftp.cs.ucla.edu/pub/stat_ser/r402.pdf http://ftp.cs.ucla.edu/pub/stat_ser/r402.pdf Much better discussion of it than I offer. abstract: The do -calculus was developed in 1995 to facilitate the identification of causal effects in non-parametric mod- els. The completeness proofs of [ Huang and Valtorta, 2006 ] and [ Shpitser and Pearl, 2006 ] and the graphi- cal criteria of [ Tian and Shpitser, 2010 ] have laid this identification problem to rest. Recent explorations un- veil the usefulness of the do -calculus in three addi- tional areas: mediation analysis [ Pearl, 2012 ] , trans- portability [ Pearl and Bareinboim, 2011 ] and meta- synthesis. Meta-synthesis (freshly coined) is the task of fusing empirical results from several diverse stud- ies, conducted on heterogeneous populations and un- der different conditions, so as to synthesize an esti- mate of a causal relation in some target environment, potentially different from those under study. The talk surveys these results with emphasis on the challenges posed by meta-synthesis. For background material, see 〈 http://bayes.cs.ucla.edu/csl http://bayes.cs.ucla.edu/csl papers.html 〉
baddox 13y ago
The reason that's a hard question is that while most people have what seems to be an "obvious" definition of "causation" in their head, it turns out to be fairly difficult to define in precise (and especially testable/decidable/falsifiable) terms.
brc 13y ago
It's very difficult to prove, but easy to disprove (change in one thing does not cause expected change in other). Sometimes disproving the causation is enough to gain valuable knowledge.
im3w1l 13y ago
Yes, under some assumptions. What you need though is free randomness. Free randomness is basically random information that isn't caused by anything. What you do is you perform a number of experiments, where you use free randomness to set the independent variable. By observing what happens to the dependent variable, you can deduce causation. Now, the challenge is that free randomness doesn't exist. But with a good approximation (i.e. it has little common cause with the dependent variable, and is caused very little by the dependent variable. a dice roll would in many cases be a very good approximation) we can get a result with high confidence. This can be done based on data alone, if someone else has performed the experiments, or through a "natural experiment".
- tel 13y ago
  This is but one method—random assignment breaks causal linkages in a way that isolates the causal pathways of interest.
tel 13y ago
This is heavily studied by Judea Pearl. I highly recommend his papers and books. The high-level summary is that you need to specify your causal assumptions and then use that information to justify or disprove experiments and tests as being sufficient to make causal claims.
- cma 13y ago
  He won a Turing Award in 2011 for his work. Haven't read it yet, but this is supposedly where to start: http://www.amazon.com/Causality-Reasoning-Inference-Judea-Pearl/dp/052189560X http://www.amazon.com/Causality-Reasoning-Inference-Judea-Pe...
  - tel 13y ago
    That's a fantastic book, but it goes into a great deal of detail. There are a few overview papers that are easier to get into.
wcbeard10 13y ago
Michael Nielsen has a good writeup: "If correlation doesn’t imply causation, then what does?| [0]: http://www.michaelnielsen.org/ddi/if-correlation-doesnt-imply-causation-then-what-does/ http://www.michaelnielsen.org/ddi/if-correlation-doesnt-impl...
rm999 13y ago
In general, no. Simple counterexample: everyday it rains my grass is wet, and everyday it doesn't rain the grass is dry. Does the rain cause the grass to be wet? Well, probably. But maybe my grass is in a greenhouse, and I only turn on my sprinkler when it rains so the grass gets a "natural" amount of water. In this case the wet grass is not causally linked to the rain, even though it looks exactly like the situation where it is. My action of turning on the sprinkler is known as a "confounding factor". You need to make assumptions to be able to draw a causation from data. This is what people are doing when they design controlled experiments.
- tedsanders 13y ago
  I'm not an expert in algebraic statistics, but I think sometimes the answer is yes: http://acritch.com/media/math/critch-causality-talk-slides.pdf http://acritch.com/media/math/critch-causality-talk-slides.p... You have to assume the data you have is all that there is, and sometimes the math will only give a class of causal graphs instead of a single causal graph.
  - rm999 13y ago
    From what I can tell, that presentation is assuming a certain class of graph structures in its DACB example - for example that there isn't an element E that can causally link the other nodes, emulating a causal relationship without there being one. Assuming a class of structures is not a general approach; in my example this would be the equivalent of assuming there is no sprinkler, but how can you make that assumption from the data alone? There's another issue: their approach is statistical, so even if you assume a class of graph structures you can only draw conclusions that the causal structure "almost surely" exists, not that it does exist. What this means is that even within infinite data, you can't prove that the other structures are impossible. If I turn on my sprinkler when it rains the first {very large number} times, you may conclude that there is almost surely a causal link between the rain and wet grass, but there is no reason I have to do it. You aren't left with a proof.
- benched 13y ago
  If you only turn on the sprinkler when it rains, the wet grass is still causally linked to the rain. There's just extra mediation - you.
mathattack 13y ago
This is very much the field of Econometrics, perhaps the subset of Mathematics with the weakest theoretical footing. The short answer is not 100% because you can never 100% prove there isn't some other bigger cause that you just don't see. [1] Once you accept that, there are ways to test theories on the data (requires some knowledge of the subject matter) to piece it out. In this case, you're not going to get an absolute law of science, but you might get a clue for a policy decision. James Heckman [0] is one of several who make this their life's work. He has applied this analysis to fields like early childhood education. [0] http://www.nber.org/papers/w13934 http://www.nber.org/papers/w13934 [1] http://en.wikibooks.org/wiki/Econometric_Theory/Regression_versus_Causation_and_Correlation http://en.wikibooks.org/wiki/Econometric_Theory/Regression_v...

pingou 13y ago

I was wondering the same thing. Imagine we make a study on programmers and observe that 80% of them like science. Then we have a new programmer than wasn't part of the study, can we say he has a 80% chance of liking science? Intuitively I would say yes but I think it's wrong, can someone help?

StavrosK 13y ago
I don't know much more than you, but I'll give it a shot: We can say whatever we like, really. If you mean "is it true", then yes, if our sample was representative of all the programmers and not biased in some way, the new programmer will have a 80% chance of liking science. In reality, though, "80% chance of liking science" is just a best guess. Since all we have to go with is the sample we already studied, we assume it's representative (or try to correct for the bias we know) and make a guess based on that. This has nothing to do with causation, however.
- tel 13y ago
  I can clarify this. The thing I outline below is a little situation dependent (and matching this hypothetical) but the intuition is generally correct and amenable with yours. One way to see all the pieces is to recognize that at least in theory there are real answers to all of these questions which we are approximating. We could literally round up every "programmer" and ask them whether or not they "like science" and then get a completely accurate statistic that, say, "92% of programmers like science". But that's infeasible (even buying that my quoted words could make literal sense) so instead we run a finite study. This is called taking the subpopulation of a superpopulation and drawing the statistic from that subpopulation. Now we see that in our subpopulation of N participant-programmers, 80% of them liked science. Now let's bring in a new guy. Let's say we brought him in by taking every programmer in the superpopulation (replacing our sampled subpopulation back into the global pool) and picking one guy in a lottery. Provided that was a truly random lottery then we can use the 92% number from before and know that there's a 92% chance that our lottery picked a science-liker. But we don't actually have that information. Instead, we simply have an estimate to 92%, our subpopulation statistic of 80%. If our error bars on guessing incorrectly are fine with a 12% discrepancy then we're still golden, though. But there's one final element—we want to be able to talk about our sampling process and the risks of using it. The only parameter of this process is the size of the subpopulation chosen. For instance, if we had sampled only 5 people then there's about a 1/4 chance that we'd arrive at an estimate of 80% and a 7/10 chance we'd get 100%. This still leaves around 10% of the possible subpopulations we could choose that would tell us something really misleading, like only 20% of programmers like science. This could be called sample statistic stability, or just straight up estimation error. It's rarely taken into account when people say things like "80% of programmers like science" because the usual assumption is that the sampled subpopulation is "big enough" to minimize this error sufficiently. In truth, though, it's hard to know exactly what the possible error of your sampling method really is. So finally, we want to know that, given we saw 80% of our subpopulation likes science, what's the chance that a new, randomly drawn programmer also likes science. The honest truth is that there's a 92% chance, so we're ignoring some amount of error by taking our study on faith and saying that there's an 80% chance. However, given literally no other information, it's still our best bet to assume there's an 80% chance—it's better than randomly picking any other number based on the data we've seen. [0] Finally, if we were to repeat this whole experience over and over again starting with fresh, random draws of the subpopulation then we'd be, on average, correct in our guesses about the ratio of the super population [1]. [0] Bayesians here would say that we should mix in other information we might have to form a better estimate—I think that makes a lot of sense in a situation like this, especially if our subpopulation is tiny, so I'm trying to be really clear that we have no other information. Let's not talk non-informative priors. [1] i.e. if we keep repeatedly re-estimating our "80% statistic" and using it honestly to make our guesses then it would bounce around based on our subpopulation, sometimes 80%, sometimes 100%, sometimes 20%, etc. If we multiply these estimates by how often they occur and sum it all up then we'll get exactly 92%. We'd get there faster just by drawing a bigger subpopulation, though.
tel 13y ago
I responded at length to this elsewhere, but quite simply this isn't causation yet. This statement is entirely about the information in our head, not the effect of actions on the world. The causal statement would be "training to become a programmer makes you like science more" and it already begins to indicate that we'd want to observe people who both do and do not decide to become programmers and question their preferences over time in order to have relevant information.

a1369209993 13y ago

http://lesswrong.com/lw/qr/timeless_causality/ http://lesswrong.com/lw/qr/timeless_causality/

deleted 13y ago
[deleted]

zmk_ 13y ago

You can use instrumental variables to distinguish between correlation and causation.

hessenwolf 13y ago

http://ftp.cs.ucla.edu/pub/stat_ser/r402.pdf http://ftp.cs.ucla.edu/pub/stat_ser/r402.pdf Much better discussion of it than I offer. abstract: The do -calculus was developed in 1995 to facilitate the identification of causal effects in non-parametric mod- els. The completeness proofs of [ Huang and Valtorta, 2006 ] and [ Shpitser and Pearl, 2006 ] and the graphi- cal criteria of [ Tian and Shpitser, 2010 ] have laid this identification problem to rest. Recent explorations un- veil the usefulness of the do -calculus in three addi- tional areas: mediation analysis [ Pearl, 2012 ] , trans- portability [ Pearl and Bareinboim, 2011 ] and meta- synthesis. Meta-synthesis (freshly coined) is the task of fusing empirical results from several diverse stud- ies, conducted on heterogeneous populations and un- der different conditions, so as to synthesize an esti- mate of a causal relation in some target environment, potentially different from those under study. The talk surveys these results with emphasis on the challenges posed by meta-synthesis. For background material, see 〈 http://bayes.cs.ucla.edu/csl http://bayes.cs.ucla.edu/csl papers.html 〉

baddox 13y ago

The reason that's a hard question is that while most people have what seems to be an "obvious" definition of "causation" in their head, it turns out to be fairly difficult to define in precise (and especially testable/decidable/falsifiable) terms.

brc 13y ago

It's very difficult to prove, but easy to disprove (change in one thing does not cause expected change in other). Sometimes disproving the causation is enough to gain valuable knowledge.

im3w1l 13y ago

Yes, under some assumptions. What you need though is free randomness. Free randomness is basically random information that isn't caused by anything. What you do is you perform a number of experiments, where you use free randomness to set the independent variable. By observing what happens to the dependent variable, you can deduce causation. Now, the challenge is that free randomness doesn't exist. But with a good approximation (i.e. it has little common cause with the dependent variable, and is caused very little by the dependent variable. a dice roll would in many cases be a very good approximation) we can get a result with high confidence. This can be done based on data alone, if someone else has performed the experiments, or through a "natural experiment".

tel 13y ago
This is but one method—random assignment breaks causal linkages in a way that isolates the causal pathways of interest.

tel 13y ago

This is heavily studied by Judea Pearl. I highly recommend his papers and books. The high-level summary is that you need to specify your causal assumptions and then use that information to justify or disprove experiments and tests as being sufficient to make causal claims.

cma 13y ago
He won a Turing Award in 2011 for his work. Haven't read it yet, but this is supposedly where to start: http://www.amazon.com/Causality-Reasoning-Inference-Judea-Pearl/dp/052189560X http://www.amazon.com/Causality-Reasoning-Inference-Judea-Pe...
- tel 13y ago
  That's a fantastic book, but it goes into a great deal of detail. There are a few overview papers that are easier to get into.

wcbeard10 13y ago

Michael Nielsen has a good writeup: "If correlation doesn’t imply causation, then what does?| [0]: http://www.michaelnielsen.org/ddi/if-correlation-doesnt-imply-causation-then-what-does/ http://www.michaelnielsen.org/ddi/if-correlation-doesnt-impl...

rm999 13y ago

In general, no. Simple counterexample: everyday it rains my grass is wet, and everyday it doesn't rain the grass is dry. Does the rain cause the grass to be wet? Well, probably. But maybe my grass is in a greenhouse, and I only turn on my sprinkler when it rains so the grass gets a "natural" amount of water. In this case the wet grass is not causally linked to the rain, even though it looks exactly like the situation where it is. My action of turning on the sprinkler is known as a "confounding factor". You need to make assumptions to be able to draw a causation from data. This is what people are doing when they design controlled experiments.

tedsanders 13y ago
I'm not an expert in algebraic statistics, but I think sometimes the answer is yes: http://acritch.com/media/math/critch-causality-talk-slides.pdf http://acritch.com/media/math/critch-causality-talk-slides.p... You have to assume the data you have is all that there is, and sometimes the math will only give a class of causal graphs instead of a single causal graph.
- rm999 13y ago
  From what I can tell, that presentation is assuming a certain class of graph structures in its DACB example - for example that there isn't an element E that can causally link the other nodes, emulating a causal relationship without there being one. Assuming a class of structures is not a general approach; in my example this would be the equivalent of assuming there is no sprinkler, but how can you make that assumption from the data alone? There's another issue: their approach is statistical, so even if you assume a class of graph structures you can only draw conclusions that the causal structure "almost surely" exists, not that it does exist. What this means is that even within infinite data, you can't prove that the other structures are impossible. If I turn on my sprinkler when it rains the first {very large number} times, you may conclude that there is almost surely a causal link between the rain and wet grass, but there is no reason I have to do it. You aren't left with a proof.
benched 13y ago
If you only turn on the sprinkler when it rains, the wet grass is still causally linked to the rain. There's just extra mediation - you.

mathattack 13y ago

This is very much the field of Econometrics, perhaps the subset of Mathematics with the weakest theoretical footing. The short answer is not 100% because you can never 100% prove there isn't some other bigger cause that you just don't see. [1] Once you accept that, there are ways to test theories on the data (requires some knowledge of the subject matter) to piece it out. In this case, you're not going to get an absolute law of science, but you might get a clue for a policy decision. James Heckman [0] is one of several who make this their life's work. He has applied this analysis to fields like early childhood education. [0] http://www.nber.org/papers/w13934 http://www.nber.org/papers/w13934 [1] http://en.wikibooks.org/wiki/Econometric_Theory/Regression_versus_Causation_and_Correlation http://en.wikibooks.org/wiki/Econometric_Theory/Regression_v...