3 ms·
This is exactly one of the things that makes ML, AI, or whatever so interesting. Sometimes it does find correlations which a human might miss because the human
by ratsbane 7y ago
This is exactly one of the things that makes ML, AI, or whatever so interesting. Sometimes it does find correlations which a human might miss because the human thinks there is no way those two things are linked - but in fact they are.
"For example, a correlation may exist between changes in temperature in an obscure Australian town and price changes in the U.S. stock market. A person would know that the two events have no connection."
- tlb 7y agoIf you're willing to entertain that many hypotheses, you need a huge amount of data to not be fooled by noise. For instance, there might be 1B hypotheses of the form "temperature in some town is correlated with the price of some stock". You'd need p<0.000000001 to not have false positives. Even the stock market doesn't have enough data for that.
- emmelaich 7y agoMy objection is to the absoluteness of the phrase > A person would know that the two events have no connection. When a negative statement like this is presented as absolute and categorical it's very easy to disagree with it.
- emmelaich 7y agoIndeed, I can imagine a few mechanisms for such a correlation. 1. El Nino / Southern Oscillation affects the temperature in Australia and (especially South) America which would affect farm yields and hence US stock market prices. 2. Extra hot temperatures at a uranium mine near a town in Australia might limit productivity of that mine, causing uranium prices to change. Just two off the top of my head. The effects might be only very mildly correlated, but not zero.
- yongjik 7y agoI'm genuinely confused if you're saying ML is better or worse for finding such correlations, because humans are also extremely good at finding patterns where there's none, as this website hilariously demonstrates: https://tylervigen.com/spurious-correlations https://tylervigen.com/spurious-correlations
- AstralStorm 7y agoAnd sometimes it cheats, when a criminal matcher turns out to be a smile and skin tone detector.
- GuiA 7y agoRecommended reading: “Everything is correlated” https://www.gwern.net/Everything https://www.gwern.net/Everything