3 ms·

A bit of a tangent, but I suspect I'm missing the meat of the Pearl DL debate - any good links? I ask this because I also think of DL as "just" curve fitting th

A bit of a tangent, but I suspect I'm missing the meat of the Pearl DL debate - any good links? I ask this because I also think of DL as "just" curve fitting that works extremely well for some families of input. Is that far off from the view of most DL people?

ml3v 8y ago
Well there are certainly heavyweight researchers who believe the DL can organize and recognize higher principles. Andrew Ng and Geoff Hinton are among them. But DL nets can be very fragile solutions, that only curve fit to the training data. You dont want to recognize the same picture of the puppy, you want to recognize all puppies. Others who disagree are trying to integrate logical approaches with DL into novel archetectures. I see it as more of pendulum moving back and forth. But Pearl's work is in Causal Statistics and is a hard core Bayesian. https://www.quantamagazine.org/to-build-truly-intelligent-machines-teach-them-cause-and-effect-20180515/ https://www.quantamagazine.org/to-build-truly-intelligent-ma... It's not like his complaints aren't being addressed in the field, they are just not easy problems (transfer learning, world models) In a neural analogy, you don't haven't built a brain, you have a reflex.
- aaaaaaaaaab 8y ago
  *Geoff Hinton
- YeGoblynQueenne 8y ago
  >> It's not like his complaints aren't being addressed in the field, they are just not easy problems (transfer learning, world models) Machine learning is rapidly losing its ability to innovate as a field and the reason is the extreme focus on deep learning for classification- and secondarily the fact that the billions of corporate funding now attract researchers from the sciences who have no background in computer science or AI and haven't got a clue what the fuss is about that people like Perl are making. Here it is from Hinton himself: One big challenge the community faces is that if you want to get a paper published in machine learning now it's got to have a table in it, with all these different data sets across the top, and all these different methods along the side, and your method has to look like the best one. If it doesn’t look like that, it’s hard to get published. I don't think that's encouraging people to think about radically new ideas. Now if you send in a paper that has a radically new idea, there's no chance in hell it will get accepted, because it's going to get some junior reviewer who doesn't understand it. Or it’s going to get a senior reviewer who's trying to review too many papers and doesn't understand it first time round and assumes it must be nonsense. Anything that makes the brain hurt is not going to get accepted. And I think that's really bad. https://www.wired.com/story/googles-ai-guru-computers-think-more-like-brains/ https://www.wired.com/story/googles-ai-guru-computers-think-...
Jach 8y ago
The tldr from my reading is that Pearl thinks you can't ascend the "ladder of causality" with pure DL -- i.e. while you can make associations to answer simple probabilistic questions, you can't create a system that can answer questions like "if this intervention is made at t0, what is the likely outcome at t1?" or consider counterfactuals like "would my mother have lived longer if she didn't drink?" You'd probably be best served by reading Pearl himself, his recent Book of Why is good, this much earlier paper[0] kind of lays out a few things too as for his reasoning that causal inference needs different tools. There's also some stuff in the blogosphere about an earlier dispute you might find fun, i.e. the methods of using causal inference tools. Example[1]. [0] https://ftp.cs.ucla.edu/pub/stat_ser/r284-reprint.pdf https://ftp.cs.ucla.edu/pub/stat_ser/r284-reprint.pdf [1] https://statmodeling.stat.columbia.edu/2009/07/05/disputes_about/ https://statmodeling.stat.columbia.edu/2009/07/05/disputes_a...

ml3v 8y ago

Well there are certainly heavyweight researchers who believe the DL can organize and recognize higher principles. Andrew Ng and Geoff Hinton are among them. But DL nets can be very fragile solutions, that only curve fit to the training data. You dont want to recognize the same picture of the puppy, you want to recognize all puppies. Others who disagree are trying to integrate logical approaches with DL into novel archetectures. I see it as more of pendulum moving back and forth. But Pearl's work is in Causal Statistics and is a hard core Bayesian. https://www.quantamagazine.org/to-build-truly-intelligent-machines-teach-them-cause-and-effect-20180515/ https://www.quantamagazine.org/to-build-truly-intelligent-ma... It's not like his complaints aren't being addressed in the field, they are just not easy problems (transfer learning, world models) In a neural analogy, you don't haven't built a brain, you have a reflex.

aaaaaaaaaab 8y ago
*Geoff Hinton
YeGoblynQueenne 8y ago
>> It's not like his complaints aren't being addressed in the field, they are just not easy problems (transfer learning, world models) Machine learning is rapidly losing its ability to innovate as a field and the reason is the extreme focus on deep learning for classification- and secondarily the fact that the billions of corporate funding now attract researchers from the sciences who have no background in computer science or AI and haven't got a clue what the fuss is about that people like Perl are making. Here it is from Hinton himself: One big challenge the community faces is that if you want to get a paper published in machine learning now it's got to have a table in it, with all these different data sets across the top, and all these different methods along the side, and your method has to look like the best one. If it doesn’t look like that, it’s hard to get published. I don't think that's encouraging people to think about radically new ideas. Now if you send in a paper that has a radically new idea, there's no chance in hell it will get accepted, because it's going to get some junior reviewer who doesn't understand it. Or it’s going to get a senior reviewer who's trying to review too many papers and doesn't understand it first time round and assumes it must be nonsense. Anything that makes the brain hurt is not going to get accepted. And I think that's really bad. https://www.wired.com/story/googles-ai-guru-computers-think-more-like-brains/ https://www.wired.com/story/googles-ai-guru-computers-think-...

Jach 8y ago

The tldr from my reading is that Pearl thinks you can't ascend the "ladder of causality" with pure DL -- i.e. while you can make associations to answer simple probabilistic questions, you can't create a system that can answer questions like "if this intervention is made at t0, what is the likely outcome at t1?" or consider counterfactuals like "would my mother have lived longer if she didn't drink?" You'd probably be best served by reading Pearl himself, his recent Book of Why is good, this much earlier paper[0] kind of lays out a few things too as for his reasoning that causal inference needs different tools. There's also some stuff in the blogosphere about an earlier dispute you might find fun, i.e. the methods of using causal inference tools. Example[1]. [0] https://ftp.cs.ucla.edu/pub/stat_ser/r284-reprint.pdf https://ftp.cs.ucla.edu/pub/stat_ser/r284-reprint.pdf [1] https://statmodeling.stat.columbia.edu/2009/07/05/disputes_about/ https://statmodeling.stat.columbia.edu/2009/07/05/disputes_a...