4 ms·
This is really interesting. Trying to summarize the idea of the paper using their example: If people swim more on days when ice cream sales are high, one might
by Houshalter 10y ago
This is really interesting. Trying to summarize the idea of the paper using their example: If people swim more on days when ice cream sales are high, one might conclude that ice cream causes swimming. But if you have information about the temperature, you can fit a statistical model that predicts swimming. The model will learn that temperature predicts swimming perfectly, and the feature of ice cream isn't used at all. So you could say after controlling for temperature, ice cream no longer correlates with swimming.
But what if your temperature estimate is noisy or imprecise? For example, if you only have information if the temperature was hotter than 75 degrees, but not exactly what it was. Then this trick no longer works. The temperature variable predicts swimming a lot, but not completely. So ice cream sales are still useful, and so still correlate with swimming after controlling for temperature.
I wonder if the way to solve this is using bayes nets. You could test a model where the variables "swimming", "ice cream sales", and "observed temperature", are all caused by some unobserved third variable. This model should fit the data better than alternative models, like ones where "ice cream sales" cause swimming and temperature. But simple linear models can't represent dependencies like this.
- nkurz 10y agoPerhaps you know it, but there's another classic example involving ice cream. It was thought in the 1940's (by the public, by doctors, and by scientists) that ice cream consumption caused children to contract polio. More ice cream is consumed in the summer, there are more polio cases in the summer: https://theglyptodon.wordpress.com/2012/08/21/polio-caused-by-ice-cream/ https://theglyptodon.wordpress.com/2012/08/21/polio-caused-b...
- Noseshine 10y agoMy personal favorite: http://tylervigen.com/view_correlation?id=1597 http://tylervigen.com/view_correlation?id=1597 US spending on science, space, and technology correlates with Suicides by hanging, strangulation and suffocation
- blahi 10y agoThis is called mediation. Only if newly hatched data science machine learning hackers read statistics textbooks instead of coursera moocs...
- Houshalter 10y agoLooking at the wikipedia page, it looks like a crude form of bayes nets. I'm talking about testing all possible causal structures, and seeing which ones fit the data the best.
- sjg007 10y agoSee Causality by Judea Pearl.
- Houshalter 10y agoIt's on my reading list, though I'm already aware of many of the ideas in the book from other sources.
- blahi 10y agoSee the Wikipedia page about data dredging while you are at it then.
- Houshalter 10y agoThat's irrelevant. I'm suggesting testing a limited number of possible hypotheses, all of which are very plausible possibilities. In any case, bayesian inference is immune to this kind of overfitting. It will give you a set of models and their probability, given some reasonable prior.
- blahi 10y agoI just hope you work at some gaming company or another place that doesn't deal with really important decisions.