3 ms·

Rule 2 (“causation creates correlation”) would be strongly disputed by a lot of people. It relies on the assumption of “faithfulness” which is not discussed unt

Rule 2 (“causation creates correlation”) would be strongly disputed by a lot of people. It relies on the assumption of “faithfulness” which is not discussed until the bottom of the article. This is a very innocent sounding assumption but it’s actually quite strong. In particular it may be violated when there are control systems or strategic agents as part of the system you want to study — which is often the case for causal inference. In such scenarios (eg the famous thermostat example) you could have strong causal links which are invisible in the data.

apwheele 2y ago
This was my thought as well. I don't like showing the scatterplots in these examples, as "correlation" I think is more associated with the correlation coefficient than the more generic independence that the author means in this scenario. E.g. a U shape in the scatterplot may have a zero correlation coefficient but is not conditionally independent.
- bdjsiqoocwk 2y ago
  > E.g. a U shape in the scatterplot may have a zero correlation coefficient but is not conditionally independent. Ok this is correct, but has nothing to do with causality. Whether or not two variables are correlated and whether or not they are independent, and when one does or doesn't imply the other, is a conversation that can be had without resorting to the concept of causality at all. And in fact that's how the subject is taught at an introductory level basically 100% of the times.
  - cubefox 2y ago
    > Ok this is correct, but has nothing to do with causality. It does. Dependence and independence have a lot to do with causation, as the article explains. > Whether or not two variables are correlated and whether or not they are independent, and when one does or doesn't imply the other, is a conversation that can be had without resorting to the concept of causality at all. Yes, but this is irrelevant. It's like saying "whether or not someone is married is a conversation that can be had without resorting to the concept of a bachelor at all". You can talk about (in)dependence without talking about causation, but you can't talk in detail about causation without talking about (in)dependence.
- ordu 2y ago
  From the article: > NB: Correlated does not mean linearly correlated > For simplicity, I have used linear correlations in all the example R code. In real life, however, the pattern of correlation/association/mutual information we should expect depends entirely on the functional form of the causal relationships involved.
  - dash2 2y ago
    The standard mathematical definition of correlation means linear correlation. If you are talking about non-independence, it would be better to use that language. This early mistake made me think the author is not really an expert.
    - cubefox 2y ago
      What is an appropriate measure of (in)dependence though, if not Pearson correlation? Such that you feed a scatter plot into the formula for this measure, and if the measure returns 0 dependence, the variables are independent.
      - currymj 2y ago
        it's a tough problem. there are various schemes for estimating mutual information from samples. if you do that and mutual information is very close to zero, then I guess you can claim the two rvs are independent. But these estimators are pretty noisy and also often computationally frustrating (the ones I'm familiar with require doing a bunch of nearest-neighbor search between all the points). I agree with the OP that it's better to say "non-independence" and avoid confusion, at the same time, I disagree that linear correlation is actually the standard definition. In many fields, especially those where nobody ever expects linear relationships, it is not and everybody uses "correlated" to mean "not independent".
        cubefox 2y ago
        Yeah. It would be simpler to talk about causal graphs if the nodes represented only events instead of arbitrary variables, because independence between events is much simpler to determine: X and Y are independent iff P(X) * P(Y) = P(X and Y). For events there also exists a measure of dependence: The so-called odds ratio. It is not influenced by the marginal probabilities, unlike Pearson correlation (called "phi coefficient" for events) or pointwise mutual information. Of course in practice events are usually not a possible simplification.
- currymj 2y ago
  This is a separate issue and also a good point. Correlation sometimes means “Pearson’s correlation coefficient” and sometimes means “anything but completely independent” and it’s often unclear. In this context I mean the latter.
BenoitP 2y ago
I'd argue you both could be right. Your comment could lead to a definition of intelligence. Organisms capable of causally influencing deterministic systems to their advantage can be marked as intelligent. The complexity of which would determine the degree of intelligence. Your point is great in that it pinpoints also the notions of agency scopes. In all the causal DAGs it feels like there are implicit regions: ones where we can influence or not, intervene or not, observe or not, where one is responsible for or not. An intelligent agent is one capable of modelling a system, influence it, and bias it. Such that it can reach and exploit an existing corner case of it. I talk about a corner case because of entropy and murphy's law. For a given energy, there are way many more unadvantageous states than advantageous one. And the intelligence of a system is the complexity required to wield the entropy reduction of an energy source.
- joe_the_user 2y ago
  Two problem with this. 1. There are many other ways that correlation doesn't imply causation. 2. The phenomenon the gp describes doesn't require broad intelligence but just reactiveness - a thermostat or a guided missile could have this.
bubblyworld 2y ago
For anyone else who went down a rabbit hole - this paper describes the problem control systems present for these methodologies: https://www.sciencedirect.com/science/article/abs/pii/B9780128189481000046 https://www.sciencedirect.com/science/article/abs/pii/B97801... (paywalled link, but it's available on a well-known useful website)
deleted 2y ago
[deleted]
SpaceManNabs 2y ago
My fav way to intuit this is this example https://stats.stackexchange.com/questions/85363/simple-examples-of-uncorrelated-but-not-independent-x-and-y https://stats.stackexchange.com/questions/85363/simple-examp... Blew my mind the first time I saw it. Not the same definitions one to one (author specifically talks about correlation vs linear correlation) but same idea.
kyllo 2y ago
Indeed, causally linked variables need not be correlated in observed data; bias in the opposite direction of the causal effect may approximately equal or exceed it in magnitude and "mask" the correlation. Chapter 1 of this popular causal inference book demonstrates this with a few examples: https://mixtape.scunning.com/01-introduction#do-not-confuse-correlation-with-causality https://mixtape.scunning.com/01-introduction#do-not-confuse-...

apwheele 2y ago

This was my thought as well. I don't like showing the scatterplots in these examples, as "correlation" I think is more associated with the correlation coefficient than the more generic independence that the author means in this scenario. E.g. a U shape in the scatterplot may have a zero correlation coefficient but is not conditionally independent.

bdjsiqoocwk 2y ago
> E.g. a U shape in the scatterplot may have a zero correlation coefficient but is not conditionally independent. Ok this is correct, but has nothing to do with causality. Whether or not two variables are correlated and whether or not they are independent, and when one does or doesn't imply the other, is a conversation that can be had without resorting to the concept of causality at all. And in fact that's how the subject is taught at an introductory level basically 100% of the times.
- cubefox 2y ago
  > Ok this is correct, but has nothing to do with causality. It does. Dependence and independence have a lot to do with causation, as the article explains. > Whether or not two variables are correlated and whether or not they are independent, and when one does or doesn't imply the other, is a conversation that can be had without resorting to the concept of causality at all. Yes, but this is irrelevant. It's like saying "whether or not someone is married is a conversation that can be had without resorting to the concept of a bachelor at all". You can talk about (in)dependence without talking about causation, but you can't talk in detail about causation without talking about (in)dependence.
ordu 2y ago
From the article: > NB: Correlated does not mean linearly correlated > For simplicity, I have used linear correlations in all the example R code. In real life, however, the pattern of correlation/association/mutual information we should expect depends entirely on the functional form of the causal relationships involved.
- dash2 2y ago
  The standard mathematical definition of correlation means linear correlation. If you are talking about non-independence, it would be better to use that language. This early mistake made me think the author is not really an expert.
  - cubefox 2y ago
    What is an appropriate measure of (in)dependence though, if not Pearson correlation? Such that you feed a scatter plot into the formula for this measure, and if the measure returns 0 dependence, the variables are independent.
    - currymj 2y ago
      it's a tough problem. there are various schemes for estimating mutual information from samples. if you do that and mutual information is very close to zero, then I guess you can claim the two rvs are independent. But these estimators are pretty noisy and also often computationally frustrating (the ones I'm familiar with require doing a bunch of nearest-neighbor search between all the points). I agree with the OP that it's better to say "non-independence" and avoid confusion, at the same time, I disagree that linear correlation is actually the standard definition. In many fields, especially those where nobody ever expects linear relationships, it is not and everybody uses "correlated" to mean "not independent".
      - cubefox 2y ago
        Yeah. It would be simpler to talk about causal graphs if the nodes represented only events instead of arbitrary variables, because independence between events is much simpler to determine: X and Y are independent iff P(X) * P(Y) = P(X and Y). For events there also exists a measure of dependence: The so-called odds ratio. It is not influenced by the marginal probabilities, unlike Pearson correlation (called "phi coefficient" for events) or pointwise mutual information. Of course in practice events are usually not a possible simplification.
currymj 2y ago
This is a separate issue and also a good point. Correlation sometimes means “Pearson’s correlation coefficient” and sometimes means “anything but completely independent” and it’s often unclear. In this context I mean the latter.

BenoitP 2y ago

I'd argue you both could be right. Your comment could lead to a definition of intelligence. Organisms capable of causally influencing deterministic systems to their advantage can be marked as intelligent. The complexity of which would determine the degree of intelligence. Your point is great in that it pinpoints also the notions of agency scopes. In all the causal DAGs it feels like there are implicit regions: ones where we can influence or not, intervene or not, observe or not, where one is responsible for or not. An intelligent agent is one capable of modelling a system, influence it, and bias it. Such that it can reach and exploit an existing corner case of it. I talk about a corner case because of entropy and murphy's law. For a given energy, there are way many more unadvantageous states than advantageous one. And the intelligence of a system is the complexity required to wield the entropy reduction of an energy source.

joe_the_user 2y ago
Two problem with this. 1. There are many other ways that correlation doesn't imply causation. 2. The phenomenon the gp describes doesn't require broad intelligence but just reactiveness - a thermostat or a guided missile could have this.

bubblyworld 2y ago

For anyone else who went down a rabbit hole - this paper describes the problem control systems present for these methodologies: https://www.sciencedirect.com/science/article/abs/pii/B9780128189481000046 https://www.sciencedirect.com/science/article/abs/pii/B97801... (paywalled link, but it's available on a well-known useful website)

deleted 2y ago

[deleted]

SpaceManNabs 2y ago

My fav way to intuit this is this example https://stats.stackexchange.com/questions/85363/simple-examples-of-uncorrelated-but-not-independent-x-and-y https://stats.stackexchange.com/questions/85363/simple-examp... Blew my mind the first time I saw it. Not the same definitions one to one (author specifically talks about correlation vs linear correlation) but same idea.

kyllo 2y ago

Indeed, causally linked variables need not be correlated in observed data; bias in the opposite direction of the causal effect may approximately equal or exceed it in magnitude and "mask" the correlation. Chapter 1 of this popular causal inference book demonstrates this with a few examples: https://mixtape.scunning.com/01-introduction#do-not-confuse-correlation-with-causality https://mixtape.scunning.com/01-introduction#do-not-confuse-...