5 ms·

Correlation does not imply causation.

onetimemanytime 8y ago
they are certainly not charging less for targeted ads, the opposite is true.
Maro 8y ago
Not strictly related to this discussion, but perhaps worth sharing. I worked at a smaller tech company, and we kept saying "Correlation does not imply causation", and trying to come up with ways to separate them, etc. We spent/wasted a lot of time on this, explaining it to non-mathy people (which leads to even less confidence in data). Then I went to work at one of the big tech companies, which is probably one of the best in the world at testing/data for product, and there I never heard it. If there was correlation, unless it was clearly spurious or a statistical fluke, they pushed on it hard. "It looks like more X means more DAUs. So let's manufacture more X." And it works pretty well for that company. My takeaway is, I stopped thinking about this too much. Today, if there's correlation, unless it's clearly spurious/etc, and unless somebody has a better idea, I tell the product/eng team to push hard on it and don't even mention correlation vs causation.
- syastrov 8y ago
  I have experienced the same thing myself, but I would explain it differently. Even in science where the focus is actually on proving things, it is difficult to prove causation. I see this more as a question of whether you act on a correlation that you’ve discovered or whether you simply talk about it. While it’s fine to discuss things, it sounds like at the small company you worked at, that that was more important than taking action (or perhaps the company was too insecure to make a decision about what to do). At the big company, the focus was on making experiments and seeing what the results are, and then iterating on that. In the end, the business doesn’t care whether something causes something else, it is only interested in adjusting things, measuring results and if they increase some important metrics, then the business is happy. That said, it can be taken to an extreme where only the metrics are thought of as important, rather than other qualitative or hard-to-measure things. It is much easier with web companies who can quickly roll out changes and use AB testing to run experiments. Also, those companies with larger user bases will have better sets of data on which to base their decisions. Companies that are well established will also have historical data that will make it easier to interpret results. It also sounds like at the first company, the tech team was interested in being theoretically correct, while at the second one, the engineers were interested in trying things. It is an understandable problem in some ways: good programmers are lazy and won’t want to make unnecessary changes, so they tend to argue against changes that haven’t been proven ahead of time to yield results. However, for the business to thrive, knobs must be adjusted and experiments made. And those programmers have to turn the knobs (or make them adjustable by someone else). So there is a strong counter-motive at play there. However, good programmers who are also engineers with a focus on the product / business will understand the importance of doing this. It’s important for the engineering team to be aligned with the business and it helps to have cross functional teams. Otherwise you may end up with departments protecting their own interests. See also Conway’s law and its derivatives.
- alien_at_work 8y ago
  >Today, if there's correlation, unless it's clearly spurious/etc, and unless somebody has a better idea, I tell the product/eng team to push hard on it and don't even mention correlation vs causation. Because it's actually a stupid thing to say. Correlation does not conclusively prove causation would be better. It certainly gives a sensible place to look.
  - systoll 8y ago
    'Correlation does not imply causation' only means that causation is not a logical consequence of correlation. It's no broader a statement than the one you prefer.
    - alien_at_work 8y ago
      Except that's not right. It may be. In fact persuing correlations is how we do science. At some point we gain enough confidence in the correlations that we call it causation.
      - systoll 8y ago
        'correlation does not imply causation' is just the negation of 'if correlation, then causation'. What do you think the relevant difference between the terms 'imply' and 'conclusively prove' is?
        alien_at_work 8y ago
        I view 'imply' as 'suggest' and 'prove' is 'this is a fact'. Correlation does in fact imply causation (which is why correlations are investigated). It just doesn't prove the causation. It may be that the two things you're looking at are both related to some other cause (e.g. ice cream sales and murders increase in summer, supposedly. This does not mean ice cream sales has a direct relation to murder but they probably both have a relation to outside tempature), it may even be pure coincidence. But if you see a correlation you see a place to begin your investigations.

they are certainly not charging less for targeted ads, the opposite is true.

Maro 8y ago

Not strictly related to this discussion, but perhaps worth sharing. I worked at a smaller tech company, and we kept saying "Correlation does not imply causation", and trying to come up with ways to separate them, etc. We spent/wasted a lot of time on this, explaining it to non-mathy people (which leads to even less confidence in data). Then I went to work at one of the big tech companies, which is probably one of the best in the world at testing/data for product, and there I never heard it. If there was correlation, unless it was clearly spurious or a statistical fluke, they pushed on it hard. "It looks like more X means more DAUs. So let's manufacture more X." And it works pretty well for that company. My takeaway is, I stopped thinking about this too much. Today, if there's correlation, unless it's clearly spurious/etc, and unless somebody has a better idea, I tell the product/eng team to push hard on it and don't even mention correlation vs causation.

syastrov 8y ago
I have experienced the same thing myself, but I would explain it differently. Even in science where the focus is actually on proving things, it is difficult to prove causation. I see this more as a question of whether you act on a correlation that you’ve discovered or whether you simply talk about it. While it’s fine to discuss things, it sounds like at the small company you worked at, that that was more important than taking action (or perhaps the company was too insecure to make a decision about what to do). At the big company, the focus was on making experiments and seeing what the results are, and then iterating on that. In the end, the business doesn’t care whether something causes something else, it is only interested in adjusting things, measuring results and if they increase some important metrics, then the business is happy. That said, it can be taken to an extreme where only the metrics are thought of as important, rather than other qualitative or hard-to-measure things. It is much easier with web companies who can quickly roll out changes and use AB testing to run experiments. Also, those companies with larger user bases will have better sets of data on which to base their decisions. Companies that are well established will also have historical data that will make it easier to interpret results. It also sounds like at the first company, the tech team was interested in being theoretically correct, while at the second one, the engineers were interested in trying things. It is an understandable problem in some ways: good programmers are lazy and won’t want to make unnecessary changes, so they tend to argue against changes that haven’t been proven ahead of time to yield results. However, for the business to thrive, knobs must be adjusted and experiments made. And those programmers have to turn the knobs (or make them adjustable by someone else). So there is a strong counter-motive at play there. However, good programmers who are also engineers with a focus on the product / business will understand the importance of doing this. It’s important for the engineering team to be aligned with the business and it helps to have cross functional teams. Otherwise you may end up with departments protecting their own interests. See also Conway’s law and its derivatives.
alien_at_work 8y ago
>Today, if there's correlation, unless it's clearly spurious/etc, and unless somebody has a better idea, I tell the product/eng team to push hard on it and don't even mention correlation vs causation. Because it's actually a stupid thing to say. Correlation does not conclusively prove causation would be better. It certainly gives a sensible place to look.
- systoll 8y ago
  'Correlation does not imply causation' only means that causation is not a logical consequence of correlation. It's no broader a statement than the one you prefer.
  - alien_at_work 8y ago
    Except that's not right. It may be. In fact persuing correlations is how we do science. At some point we gain enough confidence in the correlations that we call it causation.
    - systoll 8y ago
      'correlation does not imply causation' is just the negation of 'if correlation, then causation'. What do you think the relevant difference between the terms 'imply' and 'conclusively prove' is?
      - alien_at_work 8y ago
        I view 'imply' as 'suggest' and 'prove' is 'this is a fact'. Correlation does in fact imply causation (which is why correlations are investigated). It just doesn't prove the causation. It may be that the two things you're looking at are both related to some other cause (e.g. ice cream sales and murders increase in summer, supposedly. This does not mean ice cream sales has a direct relation to murder but they probably both have a relation to outside tempature), it may even be pure coincidence. But if you see a correlation you see a place to begin your investigations.