8 ms·

Causality in machine learning

lngnmn 10y ago
Causality must be defined as a fully-observable model and then proved by a replicable experiment, it cannot be established by any number of mere observations. Statistics or probability are unrelated to causality in principle.
- randcraw 10y ago
  "Causal inference in statistics: An overview", Judea Pearl http://ftp.cs.ucla.edu/pub/stat_ser/r350.pdf http://ftp.cs.ucla.edu/pub/stat_ser/r350.pdf "If correlation doesn’t imply causation, then what does?", Michael Nielsen http://www.michaelnielsen.org/ddi/if-correlation-doesnt-imply-causation-then-what-does/ http://www.michaelnielsen.org/ddi/if-correlation-doesnt-impl...
  - lngnmn 10y ago
    > If correlation doesn’t imply causation, then what does? Experiment. Prove of implementation, the way molecular biologists do it.
    - pc2g4d 10y ago
      The answer is more complicated than that. Pearl has shown that it some situations it's possible to answer causal questions without running experiments. Basically, you can ask causal questions using the "do-calculus" he developed: P(Dog barks | do(kick the dog) ) The `do` modifies your model to treat "kick the dog" as observed, breaking its dependence on any other variables, e.g. the doorbell ringing or the cat hissing. With those links broken, the model becomes simpler, and often the causal question can be answered using observational rather than experimental quantities. At least, that's the idea. Not sure how broadly accepted it is, but Pearl is a towering figure and his work seemed robust at least to my not-so-expert eyes.
      - lngnmn 10y ago
        That would still be a strong correlation, but not causation. One could add a weight to each observable state and select for the biggest weight so far, but it would have nothing to do with the actual causes. Would a dead dog bark when kicked? There are some principles which cannot be undone by any amount of hipsterism and sophisticated sectarian bullshitting. There is no way to jump from observation to causality without knowing the implementation. At least in this particular universe. In the realm of models (or ideas) it could be seem doable, but a map is not a territory, model does not represent reality until proven experimentally.
        alexmlamb2 10y ago
        Well, I'd agree that you probably can't go from correlation to causation without introducing assumptions - but there are situations where you can try to make assumptions. For example, in the social sciences it's common to look for "instrument variables" which are random and can't be affected by anything else. If you can find these types of variables then you can pretend that you're looking at the results from an experiment. There are also methods like discontinuity analysis, which argue that if your system is discontinuously effected by a variable, but the causes will effect the variable in a continuous way, then you can look at changes around that specific point as being causal.
        webreac 10y ago
        If the experiment you need to prove a model is an experiment that occurs naturally without intervention, you can prove a model just with observation.
        lngnmn 10y ago
        An experiment must be carefully designed according to the working hypothesis and control. Natural phenomena is not a formal experiment.
        rahimnathwani 10y ago
        Are you saying that it's impossible to draw conclusions from natural experiments? The 'family size' example here is nice: https://en.m.wikipedia.org/wiki/Natural_experiment#Recent_examples https://en.m.wikipedia.org/wiki/Natural_experiment#Recent_ex... I don't see why you need to design a new experiment if data that satisfy your requirements already exist. For example, if I wanted to know whether people given the name 'george' are more or less likely than the population as a whole to have at least one child, must I recruit pregnant women, and assign half of them to name their child 'george' and allow the others to choose any name? Would it not be better to look at existing data to compare Georges and non-Georges, perhaps slicing by socioeconomic variables? The results would be instant (rather than requiring 65 years to gather) and the dataset larger.
    - jessriedel 10y ago
      The key input for demonstrating causality is statistical independence, not human involvement. If there is an independent (exogenous) variable influencing the system, these can function fine ("natural experiment").
    - Gatsky 10y ago
      Except that molecular biology is a perfect example of how causal inference in complex systems fails. Hidden context, uncontrolled experimental variables, pleotropic effects, experimental noise all conspire to produce conclusions that cannot be reproduced or if reproduced, cannot be generalised. The contrast with physics is stark - theoretical physicists predict experimental results (eg Higgs boson) many years before the experiment is performed. I don't think that in biology, we merely need to prove a succession of hypotheses to achieve the kind of explanatory causal models we want. A theoretical framework is needed, and at the moment, there is not even an outline of such a framework.
  - fnl 10y ago
    But the whole argument in "casual calculus" is based on the assumption that you have a situation where controlled, randomized experiments are impossible. So why/how is that route not possible to design into recommender engines, churn prediction, or contextual bandits?
    - nabla9 10y ago
      The discovery of causal relationships from purely observational data is called causal discovery. It's possible to detect the direction of causality with observation with some additional assumptions that are reasonable general. For example Additive Noise Models (ANM) assume that there is an additive noise structure in observational distribution. The key assumption is that if X causes Y, the noise in X can have an effect on Y but not vice versa. Additive noise model is near 80 per cent accurate in correctly determining cause-and-effect across large number of datasets. --- Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks Joris M. Mooij, Jonas Peters, Dominik Janzing, Jakob Zscheischler, Bernhard Schölkopf; 17(32):1−102, 2016. http://jmlr.org/papers/v17/14-518.html http://jmlr.org/papers/v17/14-518.html Center for Causal Discovery web site http://www.ccd.pitt.edu/ http://www.ccd.pitt.edu/
      - lngnmn 10y ago
        > The key assumption is that if X causes Y, the noise in X can have an effect on Y but not vice versa. Fine. What if you missed a hidden variable Z which also causes Y? What if there is also X1 which causes Y when no X is present. For example, a mapping of genetic mutations to actual diseases cannot be done from purely observational data without knowing the implementation - how particular proteins interact in this or that pipeline. Possible causal relationships does not establish or prove causality itself.
        nabla9 10y ago
        >Fine. What if you missed a hidden variable Z which also causes Y? What if there is also X1 which causes Y when no X is present. Noise going trough Z -> Y is not present in X -> Y. Detecting X1 when X is not present is trivial. Y is not present and the noise from Y is not present. There is another cause besides Y. >Possible causal relationships does not establish or prove causality itself. Noise models can establish the arrow direction with very high probability. The strength of causality and other factors are considered separately
  - fnl 10y ago
    But the whole argument in "casual calculus" is based on the assumption that you have a situation where controlled, randomized experiments are impossible. So why/how is that route not possible to design into recommender engines, churn prediction, or contextual bandits?
nonbel 10y ago
Causality has never been of much interest to me. I never understood why others make such a big deal out of it. Maybe it is a real thing, maybe illusion, definitely some kind of heuristic... but shouldn't we be searching for useful "laws" (eg F ~ m1*m2/r^2) rather than "cause"? I know some have denied causality a place in science: "In the following paper I wish, first, to maintain that the word "cause" is so inextricably bound up with misleading associations as to make its complete extrusion from the philosophical vocabulary desirable; secondly, to inquire what principle, if any, is employed in science in place of the supposed "law of causality" which philosophers imagine to be employed; thirdly, to exhibit certain confusions, especially in regard to teleology and determinism, which appear to me to be connected with erroneous notions as to causality." http://www.hist-analytic.com/Russellcause.pdf http://www.hist-analytic.com/Russellcause.pdf
- epistasis 10y ago
  As usual, Pearl has some good writing on this: http://bayes.cs.ucla.edu/BOOK-2K/causality2-epilogue.pdf http://bayes.cs.ucla.edu/BOOK-2K/causality2-epilogue.pdf >Take, for instance, Newton’s law: >f = ma. >The rules of algebra permit us to write this law in a wild variety of syntactic forms, all meaning the same thing – that if we know any two of the three quantities, the third is determined. >Yet, in ordinary discourse we say that force causes acceleration – not that accelera- tion causes force, and we feel very strongly about this distinction. >Likewise, we say that the ratio f/a helps us determine the mass, not that it causes the mass. >Such distinctions are not supported by the equations of physics, and this leads us to ask whether the whole causal vocabulary is purely metaphysical, “surviving, like the monarchy . . .”. >Fortunately, very few physicists paid atten- tion to Russell’s enigma. They continued to write equations in the office and talk cause–effect in the cafeteria; with astonishing success they smashed the atom, invented the transistor and the laser.
  - nonbel 10y ago
    How about: In ordinary discourse we say that "the earth orbits the sun" – not that "the sun orbits the earth", and we feel very strongly about this distinction. It just seems like discussion of "what causes what" is missing the point, just like the above argument. That one persisted for centuries (millenia?) until it was pointed out the answer is neither! It depends on your reference frame.
dajohnson89 10y ago
The unofficial blog phenomenon intrigues me. This is a great post, but what happens when an unofficial blog (or Twitter account) posts something controversial? Breaks the law? What recourse does the company (or govt agency) have? I'm not really denouncing the unofficial blogs, but I can see it making PR people uncomfortable, and understandably so. On a personal level, it seems like an easy way to use the name of your employer (btw, how do we know they're really employees of X) as a springboard for popularity, without the huge responsibility that comes along with officially representing the organization.
tbonza 10y ago
Correlation is not causality. I'm so glad someone put this article together
nkurz 10y ago
I worked on a project a few years ago for a UC Berkeley biostatistics professor who believes he has a theoretically proven approach of using non-randomized observational data to create an asymptotically-efficient unbiased estimator of counterfactual treatments. My impression as a programmer who has only dabbled in machine learning is that this is a phenomenal claim far beyond the state of the art. My job was to take an inefficient proof-of-concept R package, and make it computationally and memory efficient enough to run on real-world datasets. I failed totally. The obvious explanation would be that I just wasn't able to understand the math involved well enough to implement the algorithm despite immersing myself in it for months. My personal guess though is that both the paper and the reference implementation were flawed in some way that made the task impossible. Anyway, the paper is here: Targeted Maximum Likelihood Estimation for Dynamic and Static Longitudinal Marginal Structural Working Models Petersen, Schwab, Gruber, Blaser, Schomacher, and van der Laan; J Causal Inference 2015 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4405134/pdf/nihms675179.pdf https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4405134/pdf/nih... And the R package here: https://github.com/joshuaschwab/ltmle https://github.com/joshuaschwab/ltmle. My understanding was never perfect, but my belief is that there is kernel of insight in this approach that has not yet been explored in machine learning. Alternatively, maybe it works as is, and just needs a better implementation. I'd love to see someone implement this approach, or fix it, or discredit it. As it is, I think it's potentially incredibly valuable work that is getting very little attention.
- nurettin 10y ago
  This is very interesting from a development perspective. You seem to have tested the working parts of the algorithm and broke it down to multiple passes when appropriate, I am surprised your work did not reach completion. You did all the right things. What exactly was considered wrong with the output? How did this fail?
  - nkurz 10y ago
    How did this fail? I'm still trying to understand this, and I worry that much of it is due to my personal failings. I think I'm much worse than most people at making headway on problems when I know that my understanding is flawed, and I rationalize this as part of my moral code as a sort of "first do no harm". But I'm scared to discard my moral instincts for sake of convenience, out of fear that I wouldn't know where to stop. In this case, I couldn't find an interpretation of the paper that matched my interpretation of the sample code, and I couldn't find an interpretation of either that seemed plausibly correct. The flaws in each seemed so obvious to me that I had to presume that I was not interpreting either correctly, and I concluded that I must be missing something essential. There wasn't much in the way of test structure other than eyeing up the results and declaring it to be OK. I was scared that if I inadvertently implemented something incompatible with both the paper and the code, my errors would never be caught. None of the grad students seemed to fully understand the paper, and the professor wasn't familiar with the R code. My clumsiness with the terminology of the field made it difficult for me to communicate with the professor. I feel terrible about the whole thing, but don't know what in particular I should have done differently.

Causality must be defined as a fully-observable model and then proved by a replicable experiment, it cannot be established by any number of mere observations. Statistics or probability are unrelated to causality in principle.

randcraw 10y ago
"Causal inference in statistics: An overview", Judea Pearl http://ftp.cs.ucla.edu/pub/stat_ser/r350.pdf http://ftp.cs.ucla.edu/pub/stat_ser/r350.pdf "If correlation doesn’t imply causation, then what does?", Michael Nielsen http://www.michaelnielsen.org/ddi/if-correlation-doesnt-imply-causation-then-what-does/ http://www.michaelnielsen.org/ddi/if-correlation-doesnt-impl...
- lngnmn 10y ago
  > If correlation doesn’t imply causation, then what does? Experiment. Prove of implementation, the way molecular biologists do it.
  - pc2g4d 10y ago
    The answer is more complicated than that. Pearl has shown that it some situations it's possible to answer causal questions without running experiments. Basically, you can ask causal questions using the "do-calculus" he developed: P(Dog barks | do(kick the dog) ) The `do` modifies your model to treat "kick the dog" as observed, breaking its dependence on any other variables, e.g. the doorbell ringing or the cat hissing. With those links broken, the model becomes simpler, and often the causal question can be answered using observational rather than experimental quantities. At least, that's the idea. Not sure how broadly accepted it is, but Pearl is a towering figure and his work seemed robust at least to my not-so-expert eyes.
    - lngnmn 10y ago
      That would still be a strong correlation, but not causation. One could add a weight to each observable state and select for the biggest weight so far, but it would have nothing to do with the actual causes. Would a dead dog bark when kicked? There are some principles which cannot be undone by any amount of hipsterism and sophisticated sectarian bullshitting. There is no way to jump from observation to causality without knowing the implementation. At least in this particular universe. In the realm of models (or ideas) it could be seem doable, but a map is not a territory, model does not represent reality until proven experimentally.
      - alexmlamb2 10y ago
        Well, I'd agree that you probably can't go from correlation to causation without introducing assumptions - but there are situations where you can try to make assumptions. For example, in the social sciences it's common to look for "instrument variables" which are random and can't be affected by anything else. If you can find these types of variables then you can pretend that you're looking at the results from an experiment. There are also methods like discontinuity analysis, which argue that if your system is discontinuously effected by a variable, but the causes will effect the variable in a continuous way, then you can look at changes around that specific point as being causal.
      - webreac 10y ago
        If the experiment you need to prove a model is an experiment that occurs naturally without intervention, you can prove a model just with observation.
        lngnmn 10y ago
        An experiment must be carefully designed according to the working hypothesis and control. Natural phenomena is not a formal experiment.
        rahimnathwani 10y ago
        Are you saying that it's impossible to draw conclusions from natural experiments? The 'family size' example here is nice: https://en.m.wikipedia.org/wiki/Natural_experiment#Recent_examples https://en.m.wikipedia.org/wiki/Natural_experiment#Recent_ex... I don't see why you need to design a new experiment if data that satisfy your requirements already exist. For example, if I wanted to know whether people given the name 'george' are more or less likely than the population as a whole to have at least one child, must I recruit pregnant women, and assign half of them to name their child 'george' and allow the others to choose any name? Would it not be better to look at existing data to compare Georges and non-Georges, perhaps slicing by socioeconomic variables? The results would be instant (rather than requiring 65 years to gather) and the dataset larger.
  - jessriedel 10y ago
    The key input for demonstrating causality is statistical independence, not human involvement. If there is an independent (exogenous) variable influencing the system, these can function fine ("natural experiment").
  - Gatsky 10y ago
    Except that molecular biology is a perfect example of how causal inference in complex systems fails. Hidden context, uncontrolled experimental variables, pleotropic effects, experimental noise all conspire to produce conclusions that cannot be reproduced or if reproduced, cannot be generalised. The contrast with physics is stark - theoretical physicists predict experimental results (eg Higgs boson) many years before the experiment is performed. I don't think that in biology, we merely need to prove a succession of hypotheses to achieve the kind of explanatory causal models we want. A theoretical framework is needed, and at the moment, there is not even an outline of such a framework.
- fnl 10y ago
  But the whole argument in "casual calculus" is based on the assumption that you have a situation where controlled, randomized experiments are impossible. So why/how is that route not possible to design into recommender engines, churn prediction, or contextual bandits?
  - nabla9 10y ago
    The discovery of causal relationships from purely observational data is called causal discovery. It's possible to detect the direction of causality with observation with some additional assumptions that are reasonable general. For example Additive Noise Models (ANM) assume that there is an additive noise structure in observational distribution. The key assumption is that if X causes Y, the noise in X can have an effect on Y but not vice versa. Additive noise model is near 80 per cent accurate in correctly determining cause-and-effect across large number of datasets. --- Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks Joris M. Mooij, Jonas Peters, Dominik Janzing, Jakob Zscheischler, Bernhard Schölkopf; 17(32):1−102, 2016. http://jmlr.org/papers/v17/14-518.html http://jmlr.org/papers/v17/14-518.html Center for Causal Discovery web site http://www.ccd.pitt.edu/ http://www.ccd.pitt.edu/
    - lngnmn 10y ago
      > The key assumption is that if X causes Y, the noise in X can have an effect on Y but not vice versa. Fine. What if you missed a hidden variable Z which also causes Y? What if there is also X1 which causes Y when no X is present. For example, a mapping of genetic mutations to actual diseases cannot be done from purely observational data without knowing the implementation - how particular proteins interact in this or that pipeline. Possible causal relationships does not establish or prove causality itself.
      - nabla9 10y ago
        >Fine. What if you missed a hidden variable Z which also causes Y? What if there is also X1 which causes Y when no X is present. Noise going trough Z -> Y is not present in X -> Y. Detecting X1 when X is not present is trivial. Y is not present and the noise from Y is not present. There is another cause besides Y. >Possible causal relationships does not establish or prove causality itself. Noise models can establish the arrow direction with very high probability. The strength of causality and other factors are considered separately
- fnl 10y ago
  But the whole argument in "casual calculus" is based on the assumption that you have a situation where controlled, randomized experiments are impossible. So why/how is that route not possible to design into recommender engines, churn prediction, or contextual bandits?

nonbel 10y ago

Causality has never been of much interest to me. I never understood why others make such a big deal out of it. Maybe it is a real thing, maybe illusion, definitely some kind of heuristic... but shouldn't we be searching for useful "laws" (eg F ~ m1*m2/r^2) rather than "cause"? I know some have denied causality a place in science: "In the following paper I wish, first, to maintain that the word "cause" is so inextricably bound up with misleading associations as to make its complete extrusion from the philosophical vocabulary desirable; secondly, to inquire what principle, if any, is employed in science in place of the supposed "law of causality" which philosophers imagine to be employed; thirdly, to exhibit certain confusions, especially in regard to teleology and determinism, which appear to me to be connected with erroneous notions as to causality." http://www.hist-analytic.com/Russellcause.pdf http://www.hist-analytic.com/Russellcause.pdf

epistasis 10y ago
As usual, Pearl has some good writing on this: http://bayes.cs.ucla.edu/BOOK-2K/causality2-epilogue.pdf http://bayes.cs.ucla.edu/BOOK-2K/causality2-epilogue.pdf >Take, for instance, Newton’s law: >f = ma. >The rules of algebra permit us to write this law in a wild variety of syntactic forms, all meaning the same thing – that if we know any two of the three quantities, the third is determined. >Yet, in ordinary discourse we say that force causes acceleration – not that accelera- tion causes force, and we feel very strongly about this distinction. >Likewise, we say that the ratio f/a helps us determine the mass, not that it causes the mass. >Such distinctions are not supported by the equations of physics, and this leads us to ask whether the whole causal vocabulary is purely metaphysical, “surviving, like the monarchy . . .”. >Fortunately, very few physicists paid atten- tion to Russell’s enigma. They continued to write equations in the office and talk cause–effect in the cafeteria; with astonishing success they smashed the atom, invented the transistor and the laser.
- nonbel 10y ago
  How about: In ordinary discourse we say that "the earth orbits the sun" – not that "the sun orbits the earth", and we feel very strongly about this distinction. It just seems like discussion of "what causes what" is missing the point, just like the above argument. That one persisted for centuries (millenia?) until it was pointed out the answer is neither! It depends on your reference frame.

dajohnson89 10y ago

The unofficial blog phenomenon intrigues me. This is a great post, but what happens when an unofficial blog (or Twitter account) posts something controversial? Breaks the law? What recourse does the company (or govt agency) have? I'm not really denouncing the unofficial blogs, but I can see it making PR people uncomfortable, and understandably so. On a personal level, it seems like an easy way to use the name of your employer (btw, how do we know they're really employees of X) as a springboard for popularity, without the huge responsibility that comes along with officially representing the organization.

tbonza 10y ago

Correlation is not causality. I'm so glad someone put this article together

nkurz 10y ago

I worked on a project a few years ago for a UC Berkeley biostatistics professor who believes he has a theoretically proven approach of using non-randomized observational data to create an asymptotically-efficient unbiased estimator of counterfactual treatments. My impression as a programmer who has only dabbled in machine learning is that this is a phenomenal claim far beyond the state of the art. My job was to take an inefficient proof-of-concept R package, and make it computationally and memory efficient enough to run on real-world datasets. I failed totally. The obvious explanation would be that I just wasn't able to understand the math involved well enough to implement the algorithm despite immersing myself in it for months. My personal guess though is that both the paper and the reference implementation were flawed in some way that made the task impossible. Anyway, the paper is here: Targeted Maximum Likelihood Estimation for Dynamic and Static Longitudinal Marginal Structural Working Models Petersen, Schwab, Gruber, Blaser, Schomacher, and van der Laan; J Causal Inference 2015 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4405134/pdf/nihms675179.pdf https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4405134/pdf/nih... And the R package here: https://github.com/joshuaschwab/ltmle https://github.com/joshuaschwab/ltmle. My understanding was never perfect, but my belief is that there is kernel of insight in this approach that has not yet been explored in machine learning. Alternatively, maybe it works as is, and just needs a better implementation. I'd love to see someone implement this approach, or fix it, or discredit it. As it is, I think it's potentially incredibly valuable work that is getting very little attention.

nurettin 10y ago
This is very interesting from a development perspective. You seem to have tested the working parts of the algorithm and broke it down to multiple passes when appropriate, I am surprised your work did not reach completion. You did all the right things. What exactly was considered wrong with the output? How did this fail?
- nkurz 10y ago
  How did this fail? I'm still trying to understand this, and I worry that much of it is due to my personal failings. I think I'm much worse than most people at making headway on problems when I know that my understanding is flawed, and I rationalize this as part of my moral code as a sort of "first do no harm". But I'm scared to discard my moral instincts for sake of convenience, out of fear that I wouldn't know where to stop. In this case, I couldn't find an interpretation of the paper that matched my interpretation of the sample code, and I couldn't find an interpretation of either that seemed plausibly correct. The flaws in each seemed so obvious to me that I had to presume that I was not interpreting either correctly, and I concluded that I must be missing something essential. There wasn't much in the way of test structure other than eyeing up the results and declaring it to be OK. I was scared that if I inadvertently implemented something incompatible with both the paper and the code, my errors would never be caught. None of the grad students seemed to fully understand the paper, and the professor wasn't familiar with the R code. My clumsiness with the terminology of the field made it difficult for me to communicate with the professor. I feel terrible about the whole thing, but don't know what in particular I should have done differently.