4 ms·

I've already started snarkily saying that machine learning experts are people who believe that their degree in statistics allows them to ignore the correlation

I've already started snarkily saying that machine learning experts are people who believe that their degree in statistics allows them to ignore the correlation causation fallacy. To put that point another way: It's commonly known that shark attacks are rare. But that's because we know where sharks are. We don't swim in beaches where sharks can easily swim, and rarely far out enough for them to be prowling. When swimming in deep water, shark attacks are comparable to risks of sports like cycling [1]. Sharks aren't going to kill you, but if we didn't have institutional knowledge of where they attacked (and continued watersports at the rate we currently have), you'd likely know someone who died of one. [1 ]https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3941575/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3941575/

bryanrasmussen 7y ago
it sounds to me like you're assuming the reason why people don't swim in deep water is because there might be sharks there?
- GauntletWizard 7y ago
  One of the reasons that few people swim in deep water is because there are some number of sharks there. Most problems reach equalibriums. Some problems have a very strong S-Curve effects. "How deep is the water" and "probability of dying of a shark attack" has a sigmoid curve, where as soon as you go into water deep enough for sharks to actually swim, the danger goes from "Unheard of if not impossible" to "Potentially Life Threatening but within acceptable risk territory". Once out there, it keeps going up. People don't swim in deep water because deep water is not generally accessible. One of the reasons that deep water is not accessible is because deep water also contain sharks. These have reached an equilibrium. Second example of this fallacy: "Helicopter parents" is a commonly cited problem [1]. Parents are overprotective of children. The rise of helicopter parenting has coincided with a reduction in child mortality rate from non-natural causes. Wouldn't that indicate that helicopter parenting is working? The answer ranges from "No" to "Maybe". Children killed by car accidents correlates with a decline on overall accident rate - But much of that is driven by initiatives like MADD, which has worked to improve general auto safety as the most effective way of protecting children. These factors can be compared to stoichiometric math - They are trending towards equilibriums, but probably have not yet reached them, and as other reactions happen around them the setpoints they are trending to change. [1] https://www.washingtonpost.com/news/wonk/wp/2015/04/14/theres-never-been-a-safer-time-to-be-a-kid-in-america/?utm_term=.c988ab4dcff1 https://www.washingtonpost.com/news/wonk/wp/2015/04/14/there...
  - bryanrasmussen 7y ago
    >One of the reasons that few people swim in >deep water is because there are some >number of sharks there. I can't help but think even if there were no sharks at all the amount of people swimming in deep water would not increase that much, because I find it likely that the other causes of people not swimming in deep water have a much greater effect on how many people actually swim in deeper water. Also there are people now who swim in deep water specifically to swim with Sharks.
jayd16 7y ago
You're kind of missing the point though. Correlation is enough to get results. Worrying about the edge cases where correlation falls apart is simply worrying about insufficient training data.
- TazeTSchnitzel 7y ago
  And what ML system in practice will not have some insufficiencies in the training data? The wonderful thing about ML is we won't even know the problems unless we extensively experimentally test the resulting model!
- rmellow 7y ago
  What kind of results do you expect from blindly using correlation? Building a e.g. credit rating model with this kind of mentality will just result in a racially biased model.
  - jayd16 7y ago
    Why the hypothetical? Valid uses are already implemented. Salient points on a face do not cause a face but you can still get very good facial recognition. Its a parlor trick perhaps, but its silly to ignore the results. As for building a biased system, again its just about the data you use and you could easily build an "intelligent" system that had a racial bias. We should worry about automation that simply pushes the status quo but I don't think that's a problem specific to statistical based ML.
  - bryanrasmussen 7y ago
    I suppose it will result in a model with all sorts of biases among which a racial bias will be one.

bryanrasmussen 7y ago

it sounds to me like you're assuming the reason why people don't swim in deep water is because there might be sharks there?

GauntletWizard 7y ago
One of the reasons that few people swim in deep water is because there are some number of sharks there. Most problems reach equalibriums. Some problems have a very strong S-Curve effects. "How deep is the water" and "probability of dying of a shark attack" has a sigmoid curve, where as soon as you go into water deep enough for sharks to actually swim, the danger goes from "Unheard of if not impossible" to "Potentially Life Threatening but within acceptable risk territory". Once out there, it keeps going up. People don't swim in deep water because deep water is not generally accessible. One of the reasons that deep water is not accessible is because deep water also contain sharks. These have reached an equilibrium. Second example of this fallacy: "Helicopter parents" is a commonly cited problem [1]. Parents are overprotective of children. The rise of helicopter parenting has coincided with a reduction in child mortality rate from non-natural causes. Wouldn't that indicate that helicopter parenting is working? The answer ranges from "No" to "Maybe". Children killed by car accidents correlates with a decline on overall accident rate - But much of that is driven by initiatives like MADD, which has worked to improve general auto safety as the most effective way of protecting children. These factors can be compared to stoichiometric math - They are trending towards equilibriums, but probably have not yet reached them, and as other reactions happen around them the setpoints they are trending to change. [1] https://www.washingtonpost.com/news/wonk/wp/2015/04/14/theres-never-been-a-safer-time-to-be-a-kid-in-america/?utm_term=.c988ab4dcff1 https://www.washingtonpost.com/news/wonk/wp/2015/04/14/there...
- bryanrasmussen 7y ago
  >One of the reasons that few people swim in >deep water is because there are some >number of sharks there. I can't help but think even if there were no sharks at all the amount of people swimming in deep water would not increase that much, because I find it likely that the other causes of people not swimming in deep water have a much greater effect on how many people actually swim in deeper water. Also there are people now who swim in deep water specifically to swim with Sharks.

jayd16 7y ago

You're kind of missing the point though. Correlation is enough to get results. Worrying about the edge cases where correlation falls apart is simply worrying about insufficient training data.

TazeTSchnitzel 7y ago
And what ML system in practice will not have some insufficiencies in the training data? The wonderful thing about ML is we won't even know the problems unless we extensively experimentally test the resulting model!
rmellow 7y ago
What kind of results do you expect from blindly using correlation? Building a e.g. credit rating model with this kind of mentality will just result in a racially biased model.
- jayd16 7y ago
  Why the hypothetical? Valid uses are already implemented. Salient points on a face do not cause a face but you can still get very good facial recognition. Its a parlor trick perhaps, but its silly to ignore the results. As for building a biased system, again its just about the data you use and you could easily build an "intelligent" system that had a racial bias. We should worry about automation that simply pushes the status quo but I don't think that's a problem specific to statistical based ML.
- bryanrasmussen 7y ago
  I suppose it will result in a model with all sorts of biases among which a racial bias will be one.