4 ms·

I am making an inference here, but I imagine his complaint is with the lack of ability to derive A and B in your "If A, then B" example. His goal is to have a p

by jamez1 8y ago

I am making an inference here, but I imagine his complaint is with the lack of ability to derive A and B in your "If A, then B" example. His goal is to have a program that would have a model of reality and be able to postulate different models of reality, I don't think it's that unreasonable to claim we don't know how to build that using mathematics yet. You do make a good point about his agenda though, out of the entire world it's hard to imagine someone more bias to wanting to see causality be the new machine learning topic du jour!

YeGoblynQueenne 8y ago
>> I am making an inference here, but I imagine his complaint is with the lack of ability to derive A and B in your "If A, then B" example. Full disclosure: I work with Meta-Interpretive Learning, a class of algorithms that can do exactly that, for my PhD. See [1] for an overview and [2] for a more detailed explanation (I am not an author in any of those papers). MIL can go a lot further than deriving implications in propositional logic- it learns first-order logic theories, which should cover professor Pearl's expectations quite adequately. For instance, in a MIL setting it's perfectly possible to learn a theory for e.g. cause_of(A,B), from examples of what A causes what B. As to learning counterfactuals, my thesis advisor has published papers on a Robot Scientist, a machine that carries out scientific experiments from start to end: it forms hypotheses and carries out experiments to test them, then refines its theories, etc [3]. That last paper was published in Nature- which makes it even harder to see how Pearl can claim that nobody knows how to do this sort of thing in machine learning. I understand if people just want to ride on the coattails of deep learning and only talk about that in the press, but it's a bit annoying all the same, to see well-established results ignored by someone so learned. ___________ [1] Meta-Interpretive Learning: achievements and challenges https://www.doc.ic.ac.uk/~shm/Papers/rulemlabs.pdf https://www.doc.ic.ac.uk/~shm/Papers/rulemlabs.pdf [2] Meta-interpretive learning of higher-order dyadic datalog: predicate invention revisited https://www.doc.ic.ac.uk/~shm/Papers/metagolDMLJ.pdf https://www.doc.ic.ac.uk/~shm/Papers/metagolDMLJ.pdf [3] Functional genomic hypothesis generation and experimentation by a robot scientist https://www.nature.com/articles/nature02236 https://www.nature.com/articles/nature02236
- jamez1 8y ago
  That's interesting, thanks for the links. I have mucked around with program induction using expression trees but had not come across MIL before. It looks like MIL would generalize quite well to bayesian networks, a perfect bridge between his past research and what he's depicting as the future of ML research.

by jamez1 8y ago

YeGoblynQueenne 8y ago

>> I am making an inference here, but I imagine his complaint is with the lack of ability to derive A and B in your "If A, then B" example. Full disclosure: I work with Meta-Interpretive Learning, a class of algorithms that can do exactly that, for my PhD. See [1] for an overview and [2] for a more detailed explanation (I am not an author in any of those papers). MIL can go a lot further than deriving implications in propositional logic- it learns first-order logic theories, which should cover professor Pearl's expectations quite adequately. For instance, in a MIL setting it's perfectly possible to learn a theory for e.g. cause_of(A,B), from examples of what A causes what B. As to learning counterfactuals, my thesis advisor has published papers on a Robot Scientist, a machine that carries out scientific experiments from start to end: it forms hypotheses and carries out experiments to test them, then refines its theories, etc [3]. That last paper was published in Nature- which makes it even harder to see how Pearl can claim that nobody knows how to do this sort of thing in machine learning. I understand if people just want to ride on the coattails of deep learning and only talk about that in the press, but it's a bit annoying all the same, to see well-established results ignored by someone so learned. ___________ [1] Meta-Interpretive Learning: achievements and challenges https://www.doc.ic.ac.uk/~shm/Papers/rulemlabs.pdf https://www.doc.ic.ac.uk/~shm/Papers/rulemlabs.pdf [2] Meta-interpretive learning of higher-order dyadic datalog: predicate invention revisited https://www.doc.ic.ac.uk/~shm/Papers/metagolDMLJ.pdf https://www.doc.ic.ac.uk/~shm/Papers/metagolDMLJ.pdf [3] Functional genomic hypothesis generation and experimentation by a robot scientist https://www.nature.com/articles/nature02236 https://www.nature.com/articles/nature02236

jamez1 8y ago
That's interesting, thanks for the links. I have mucked around with program induction using expression trees but had not come across MIL before. It looks like MIL would generalize quite well to bayesian networks, a perfect bridge between his past research and what he's depicting as the future of ML research.