4 ms·

Hill for the data scientist: an xkcd story

122 points by dsr12 9y ago

digi_owl 9y ago
As an aside, i feel there has been a lack of such strips from XKCD lately.
- Normal_gaussian 9y ago
  Strips in article: * https://xkcd.com/552/ https://xkcd.com/552/ * https://xkcd.com/1592/ https://xkcd.com/1592/ * https://xkcd.com/242/ https://xkcd.com/242/ * https://xkcd.com/1217/ https://xkcd.com/1217/ * https://xkcd.com/925/ https://xkcd.com/925/ * https://xkcd.com/323/ https://xkcd.com/323/ * https://xkcd.com/605/ https://xkcd.com/605/ * https://xkcd.com/1170/ https://xkcd.com/1170/ * https://xkcd.com/1462/ https://xkcd.com/1462/ * https://xkcd.com/882/ https://xkcd.com/882/ Latest strip at time of writing is https://xkcd.com/1772/ https://xkcd.com/1772/ Current strip is https://xkcd.com/1868/ https://xkcd.com/1868/ Eyeballing it I think its still likely uniform even if there hasn't been one that would have made it into the article in the last 100.
Symmetry 9y ago
There's another way of looking for causality directly from data. If A and B are uncorrelated but both correlate with C then you know that C isn't causally upstream of either.
- arca_vorago 9y ago
  Interesting, is there a law or algorithm/equation for that?
  - epistasis 9y ago
    Edit: sorry, instrumental variables do not describe what the original poster was talking about, but could be useful for understanding the connections between correlation and causation in other ways. The concept of an instrumental variable https://en.m.wikipedia.org/wiki/Instrumental_variable https://en.m.wikipedia.org/wiki/Instrumental_variable covers some of it. Judea Pearl's book Causality summarizes lots of recent (past few decades) work in the area, with a focus on his theoretical framework. There are lots of algorithms for discovering the structural of causality for sets of variables larger than three. Having interventional data always makes these things easier, but it's not the only way to get at causality.
  - deleted 9y ago
    [deleted]
- digi_owl 9y ago
  So if you have an A and a C and want to figure out the causality you need to look for a B that correlates with C but not A?
- Nydhal 9y ago
  I always thought correlation was transitive.Isn't it? It is surely reflexive but I might have to go back and study more about those properties.
- ue_ 9y ago
  How can two things not correlate with each other, but both correlate with a third thing? This seems like saying x=5, y=3, and that there's some variable z which x and y are both equal to?
  - marcosdumay 9y ago
    The simplest way to explain is when when both cause C some % of the time. It's useful to run a simulation if you doubt it.
  - lowkeyokay 9y ago
    The number of people who like the taste of vanilla (I suppose) doesn't correlate with hours of sunlight in some area. Both correlate with amount of ice cream sold.
joshwa 9y ago
Anyone else frustrated hovering over the strips looking for the title-text?

122 points by dsr12 9y ago

digi_owl 9y ago

As an aside, i feel there has been a lack of such strips from XKCD lately.

Normal_gaussian 9y ago
Strips in article: * https://xkcd.com/552/ https://xkcd.com/552/ * https://xkcd.com/1592/ https://xkcd.com/1592/ * https://xkcd.com/242/ https://xkcd.com/242/ * https://xkcd.com/1217/ https://xkcd.com/1217/ * https://xkcd.com/925/ https://xkcd.com/925/ * https://xkcd.com/323/ https://xkcd.com/323/ * https://xkcd.com/605/ https://xkcd.com/605/ * https://xkcd.com/1170/ https://xkcd.com/1170/ * https://xkcd.com/1462/ https://xkcd.com/1462/ * https://xkcd.com/882/ https://xkcd.com/882/ Latest strip at time of writing is https://xkcd.com/1772/ https://xkcd.com/1772/ Current strip is https://xkcd.com/1868/ https://xkcd.com/1868/ Eyeballing it I think its still likely uniform even if there hasn't been one that would have made it into the article in the last 100.

Symmetry 9y ago

There's another way of looking for causality directly from data. If A and B are uncorrelated but both correlate with C then you know that C isn't causally upstream of either.

arca_vorago 9y ago
Interesting, is there a law or algorithm/equation for that?
- epistasis 9y ago
  Edit: sorry, instrumental variables do not describe what the original poster was talking about, but could be useful for understanding the connections between correlation and causation in other ways. The concept of an instrumental variable https://en.m.wikipedia.org/wiki/Instrumental_variable https://en.m.wikipedia.org/wiki/Instrumental_variable covers some of it. Judea Pearl's book Causality summarizes lots of recent (past few decades) work in the area, with a focus on his theoretical framework. There are lots of algorithms for discovering the structural of causality for sets of variables larger than three. Having interventional data always makes these things easier, but it's not the only way to get at causality.
- deleted 9y ago
  [deleted]
digi_owl 9y ago
So if you have an A and a C and want to figure out the causality you need to look for a B that correlates with C but not A?
Nydhal 9y ago
I always thought correlation was transitive.Isn't it? It is surely reflexive but I might have to go back and study more about those properties.
ue_ 9y ago
How can two things not correlate with each other, but both correlate with a third thing? This seems like saying x=5, y=3, and that there's some variable z which x and y are both equal to?
- marcosdumay 9y ago
  The simplest way to explain is when when both cause C some % of the time. It's useful to run a simulation if you doubt it.
- lowkeyokay 9y ago
  The number of people who like the taste of vanilla (I suppose) doesn't correlate with hours of sunlight in some area. Both correlate with amount of ice cream sold.

joshwa 9y ago

Anyone else frustrated hovering over the strips looking for the title-text?