13 ms·

Thanks for the comment and references. Good luck with your PhD program. > ... Connections with PGMs... hmm, not sure what you mean. What I meant is that ILP,

by fizixer 10y ago

Thanks for the comment and references. Good luck with your PhD program. > ... Connections with PGMs... hmm, not sure what you mean. What I meant is that ILP, PGM, HMM, are some of the topics that were being explored as an extension to GOFAI in the 1990s and first half of 2000s before the 2006 breakthrough results by Hinton which drastically shifted the focus towards NNs, Deep learning, etc. Essentially these topics were being explored as a result of the slow realization that logic is not enough to represent highly complex systems, and there is a need to move towards a hybrid of logical and probabilistic approaches. I'm keenly interested in these areas but currently busy with graduate work related to numerical analysis and scientific computing. In our circles 'adaptive' is the big word, and there is attempt to utilize AI/ML methodologies in scientific simulation, but it's at a very early stage. Once my work is finished, I hope to get more involved in ML related research and applications.

YeGoblynQueenne 10y ago
>> What I meant is that ILP, PGM, HMM, are some of the topics that were being explored as an extension to GOFAI in the 1990s and first half of 2000s before the 2006 breakthrough results by Hinton which drastically shifted the focus towards NNs, Deep learning, etc. The way I know it, PGMs and HMMs are firmly within GOFAI- they're basically graphs with weights. Although I guess you can say the same thing about ANNs, the primary purpose of PGMs and HMMs is to represent structure (specifically, dependency relations). Which is a very GOFAI thing, to my mind. It really depends on where you put the distinction between GOFAI and something else. My understanding is that the commonly accepted split is between AI with a (symbolic) representation and AI without, and the original proponent of the latter is Rodney Brooks who coined the term "Nouvelle AI" (sometimes "Nouveau") that is usually used as the modern counterpart to GOFAI. Personally, I believe the distinction is a bit stale, especially if you think that ANNs are actually older than expert systems and that even if you look just a couple of years in the past you'll see that one of the most popular (family of) machine learning algorithms was Decision Tree learners, which are basically a perfect mix of the old algorithms and the new: a propositional logic rule-base built from examples using information-theoretic measures. As to the revolution brought by Hinton, this is a bit political but I think that's just a good story, with little grounding in reality. Here's an example: https://webcache.googleusercontent.com/search?q=cache:2JwVBryZomkJ:https://archive.ics.uci.edu/ml/datasets/Kinship+&cd=1&hl=en&ct=clnk&gl=uk&client=firefox-b https://webcache.googleusercontent.com/search?q=cache:2JwVBr... That's a (cached) page from an online machine learning data repository, with data sets taken from various papers etc. The page is about a familial relationship data set from two papers, one by Hinton and one by Quinlan (the Decision Tree guy). Hinton's paper is from 1986, Quinlan's from 1989. Hinton's paper has 43 citations, all but one of which is from 2002 or earlier and going back to the '90s (Quinlan's only has one, from 1990). Which tells me that Hinton's "breakthrough result" did not revive interest in ANNs, neither was he the tenacious, lone underdog, that he likes to present thimself as: interest in ANNs and specifically Hinton's work has always been very popular, as has been the work of others (er, Schmidhuber). My understanding is that machine learning goes way back, and basically starts with GOFAI: the first thing people noticed about expert systems was the difficulty of hand-crafting non-trivial rule-bases, so they looked for ways to automate that (see, e.g. Michalski's work that goes back to the '60s) and eventually realised that you didn't actually need the expert system if you could just automate the rule extraction. But that's also probably just a cute but inaccurate narrative- there was a great deal of work in machine learning and a great deal of interest in it, that was not labelled as such, for instance, in "inductive inference" and so on. tl;dr: when we talk of machine learning and AI these days we're hardly even scratching the surface. There's a lot of past work that most university courses simply don't have the time to teach and we newer students are missing lots of useful bits of it... and are probably condemned to reinvent.

by fizixer 10y ago

YeGoblynQueenne 10y ago

>> What I meant is that ILP, PGM, HMM, are some of the topics that were being explored as an extension to GOFAI in the 1990s and first half of 2000s before the 2006 breakthrough results by Hinton which drastically shifted the focus towards NNs, Deep learning, etc. The way I know it, PGMs and HMMs are firmly within GOFAI- they're basically graphs with weights. Although I guess you can say the same thing about ANNs, the primary purpose of PGMs and HMMs is to represent structure (specifically, dependency relations). Which is a very GOFAI thing, to my mind. It really depends on where you put the distinction between GOFAI and something else. My understanding is that the commonly accepted split is between AI with a (symbolic) representation and AI without, and the original proponent of the latter is Rodney Brooks who coined the term "Nouvelle AI" (sometimes "Nouveau") that is usually used as the modern counterpart to GOFAI. Personally, I believe the distinction is a bit stale, especially if you think that ANNs are actually older than expert systems and that even if you look just a couple of years in the past you'll see that one of the most popular (family of) machine learning algorithms was Decision Tree learners, which are basically a perfect mix of the old algorithms and the new: a propositional logic rule-base built from examples using information-theoretic measures. As to the revolution brought by Hinton, this is a bit political but I think that's just a good story, with little grounding in reality. Here's an example: https://webcache.googleusercontent.com/search?q=cache:2JwVBryZomkJ:https://archive.ics.uci.edu/ml/datasets/Kinship+&cd=1&hl=en&ct=clnk&gl=uk&client=firefox-b https://webcache.googleusercontent.com/search?q=cache:2JwVBr... That's a (cached) page from an online machine learning data repository, with data sets taken from various papers etc. The page is about a familial relationship data set from two papers, one by Hinton and one by Quinlan (the Decision Tree guy). Hinton's paper is from 1986, Quinlan's from 1989. Hinton's paper has 43 citations, all but one of which is from 2002 or earlier and going back to the '90s (Quinlan's only has one, from 1990). Which tells me that Hinton's "breakthrough result" did not revive interest in ANNs, neither was he the tenacious, lone underdog, that he likes to present thimself as: interest in ANNs and specifically Hinton's work has always been very popular, as has been the work of others (er, Schmidhuber). My understanding is that machine learning goes way back, and basically starts with GOFAI: the first thing people noticed about expert systems was the difficulty of hand-crafting non-trivial rule-bases, so they looked for ways to automate that (see, e.g. Michalski's work that goes back to the '60s) and eventually realised that you didn't actually need the expert system if you could just automate the rule extraction. But that's also probably just a cute but inaccurate narrative- there was a great deal of work in machine learning and a great deal of interest in it, that was not labelled as such, for instance, in "inductive inference" and so on. tl;dr: when we talk of machine learning and AI these days we're hardly even scratching the surface. There's a lot of past work that most university courses simply don't have the time to teach and we newer students are missing lots of useful bits of it... and are probably condemned to reinvent.