3 ms·

I don't quite like this answer, because it simplifies a lot of things about correlations which are just not really accurate. > Corrleations are meaningful beca

I don't quite like this answer, because it simplifies a lot of things about correlations which are just not really accurate. > Corrleations are meaningful because they indicate how strongly variables can predict other variables. This is not wrong per se, but may be wrongly interpreted. 1) I think it is important to stress that the reverse is not true. If a correlation is 0, it does not mean the variables are independent. It means they are linearly independent. There are a lot of things out there which are very much linked but not with a linear relationship. 2) This is only true if the correlation was done properly. For instance, correlating non-stationary variables will spuriously yield a high correlation, giving a false impression of link. The classic example is trying to correlate stock prices between themselves. An other example of spurious correlation is when not enough points are given to get an accurate result. Looking at tstat, r2, etc. is as important as looking at the actual correlation coefficient. 3) You use the word "predict" here, which I don't quite like. Correlation can predict outcomes only in the case where you have proven that past information will continue to occur. This is not always the case. > A positive correlation between two variables means they tend to move in the same direction. This is misleading also. The variables tend to move in the same direction _relatively to their mean_! I can plot you very different looking curves that are highly correlated just because their mean vary differently. Especially because of potential outliers. > And it turns out that a strong positive correlation can be a good place to hunt for possible causality, such as hunting for something that causing both variables to change together. I tend to dislike that thought process. It is very tempting to infer causality from correlation. Especially when automating correlation at scale is affordable, so generating spurious correlation is easy. To me correlation is more a way to reinforce a hypothesis of causality that is already suspected. It won't give you the direction, but may hint you that you are on the right track. I mean, correlation should not be a clue to discover causality, as there are too many false positives. Correlation should be an intermediary step in studying a potential causality.

bollu 6y ago
Thank you for this. I learnt probability from Sheldon Ross. I now realise I don't grok correlations at all! I had the exact same mistakes in my mental model of correlation as OP. Where do I go (what do I read) to get a good account of correlation?
monoideism 6y ago
Good critiques, but I disagree with the last one. At least in practice in the sciences, correlation is frequently used to find good candidates for causation. In many scenarios, correlation is pretty much your only hope for finding a causal link between some variable and an outcome of interest.
- monoideism 6y ago
  To clarify, I'm not suggesting that correlation implies causation here (indeed, I think they take away your programmer card if you can't quickly parrot the phrase, "correlation doesn't mean causation"). From the NIST Engineer's Handbook: "Generally, we first need to find and explore correlations and then try to establish causal relationships. It is much easier to find correlations as these are just properties of the data." So I'm saying that correlation is often used (and misused) in an attempt to find good candidates for causation. This is how much science and engineering is practiced. If you disagree with this, you disagree with some fundamentals of how the scientific method is used (ie, hypothesis forming). > To me correlation is more a way to reinforce a hypothesis of causality that is already suspected It certainly can be used in this way, but that doesn't mean it's the only scientifically-valid way to use it.
TheOtherHobbes 6y ago
This is a very good answer. Perhaps one thing to add is that the underlying problem is always a mistaken belief that you can draw a general conclusion from a limited data set. The data may be flawed in many ways, so there is no automatic assumption that any conclusion is correct. And - as this answer says - this is a completely different process to discovering a robust causal mechanism that can reliably explain a correlation and predict its persistence into the future.

bollu 6y ago

Thank you for this. I learnt probability from Sheldon Ross. I now realise I don't grok correlations at all! I had the exact same mistakes in my mental model of correlation as OP. Where do I go (what do I read) to get a good account of correlation?

monoideism 6y ago

Good critiques, but I disagree with the last one. At least in practice in the sciences, correlation is frequently used to find good candidates for causation. In many scenarios, correlation is pretty much your only hope for finding a causal link between some variable and an outcome of interest.

monoideism 6y ago
To clarify, I'm not suggesting that correlation implies causation here (indeed, I think they take away your programmer card if you can't quickly parrot the phrase, "correlation doesn't mean causation"). From the NIST Engineer's Handbook: "Generally, we first need to find and explore correlations and then try to establish causal relationships. It is much easier to find correlations as these are just properties of the data." So I'm saying that correlation is often used (and misused) in an attempt to find good candidates for causation. This is how much science and engineering is practiced. If you disagree with this, you disagree with some fundamentals of how the scientific method is used (ie, hypothesis forming). > To me correlation is more a way to reinforce a hypothesis of causality that is already suspected It certainly can be used in this way, but that doesn't mean it's the only scientifically-valid way to use it.

TheOtherHobbes 6y ago

This is a very good answer. Perhaps one thing to add is that the underlying problem is always a mistaken belief that you can draw a general conclusion from a limited data set. The data may be flawed in many ways, so there is no automatic assumption that any conclusion is correct. And - as this answer says - this is a completely different process to discovering a robust causal mechanism that can reliably explain a correlation and predict its persistence into the future.