6 ms·

The trendy way to approach the problem these days is to throw massive amounts of data at a deep learning model and let the model try to learn the rules. And my

The trendy way to approach the problem these days is to throw massive amounts of data at a deep learning model and let the model try to learn the rules. And my irony detector can't help but squalk out the famous minsky koan: "What are you doing?", asked Minsky. "I am training a randomly wired neural net to play Tic-tac-toe", Sussman replied. "Why is the net wired randomly?", asked Minsky. "I do not want it to have any preconceptions of how to play", Sussman said. Minsky then shut his eyes. "Why do you close your eyes?" Sussman asked his teacher. "So that the room will be empty." At that moment, Sussman was enlightened. Machine Learning has its use cases, that's for sure, but I can't help but laugh at the person who eschews a well understood model of a well understood system by an experienced and trained expert human in favor of a magic black box that at its best might converge on the expert's understanding after enough time and data. There is no shame in building off of what is already known and understood.

trapperkeeper79 10y ago
Any tips for introductory/intermediate material on the topic? In undergrad, we used Russel and Norvig. Surely there are more advanced books or courses.
- saosebastiao 10y ago
  On expert systems specifically no. With the AI winter of the early 90's, not only did expert systems research die, but so did publisher interest in the topic. "Expert Systems" is effectively a dead field of study. That being said, the field of Operations Research, at least philosophically speaking, has picked up where AI researchers dropped it off. They've fully embraced the idea that human experts can model many systems extremely well, and have built incredible tools to do so: mathematical modeling and optimization, constraint programming, boolsat, graphical models, etc. Effectively speaking, if you think expert systems are cool, the next logical step is to delve into constraint programming, which is a sort of evolution of logic programming. I'd recommend the Minizinc Tutorial for a practical introduction with a nice DSL. Constraint Processing by Rina Dechter is a great intro with a more academic bent. I'd say mathematical modeling has been Ops Research's greatest success, and I'd definitely recommend Model Building with Mathematical Programming by Paul Williams. I'd also say that Baysians have embraced these ideas (that of building upon human expertise) within the field of Machine Learning far more than other ML researchers have. My recommendations here are probably less helpful...I've only ever toyed with Bayesian learning models, but never employed them professionally. But I would recommend Doing Bayesian Data Analysis by Kruschke. It was very helpful as an introductory material.
  - tannhaeuser 10y ago
    Constraint logic programming indeed evolved from logic programming, but it encompasses such diverse topics as finite-domain propagation, integer interval propagation, SAT solvers, linear and restricted polynomial optimization, discrete planning and constraint satisfaction strategy meta languages that the only commonality seems the more or less Prolog-like syntactical presentation. These vastly different formalisms were cast into terms such as CP(x) (meaning constraint programming over x, for x the reals or other domains) but this doesn't give you a solution strategy (the solution strategies being as varied as math itself).
  - PaulHoule 10y ago
    Today we have production rules systems such as iLog and Drools which are head and shoulders better than OPS5 and other "expert system shells" from the golden age of A.I. These are in widespread use for a few applications. Probably every bank has at least one iLog instance running for enforcing business rules. Another one is "complex event processing", where a RETE engine makes it easy to aggregate small events into larger events. Also, a few efforts, such as Inform7 and Clara have tried to push the boundaries of programming for non-experts and real-life applications. It is interesting, however, that the technology is not further applied, and a deep analysis of that could be worthwhile. For instance, RETE networks can eat "callback hell" situations for lunch, much like the complex event processing case. Instead, however, we are seeing one awful Javascript framework after another, and coroutines pushed as a very narrow answer to the problems on the server. Note that all forms of A.I. have ties to optimization. For instance, usually when you train a neural net you are minimizing some kind of an error function. Drools (and iLog) both have optimization frameworks, etc. The recent discussion of "superintelligence" has been marked by both a lack of imagination and any awareness of previous work on the subject. For instance, Rules engines are a fairly direct answer to "AI Safety" and "AI Ethics" problems that there is so much handwringing about. Most areas that require computers to be "creative" amount to some kind of multi-objective optimization, and even if rules can't make a system good, they can at least prevent the worst abuses.
    - saosebastiao 10y ago
- agumonkey 10y ago
  Some prolog books have chapters on how to implement ExpSys on it. I don't know much about this field, but in case you have nothing better .. ps: this book https://www.amazon.ca/Prolog-Programming-Artificial-Intelligence-4th/dp/0321417461 https://www.amazon.ca/Prolog-Programming-Artificial-Intellig...
empath75 10y ago
But I think experience has already shown that machine learning models blow away expert systems in a lot of domains.
- ch4s3 10y ago
  > in a lot of domains Right, but not some domains where expert systems are not only simpler to build but perform better. The can also be audited and evaluated for correctness.
- giardini 10y ago
  empath75 says:> "But I think experience has already shown that machine learning models blow away expert systems in a lot of domains." I don't think there have been many head-to-head comparisons. Academic researchers simply ceased using "expert system" in their research grant applications and began to use "neural network". For real-world applications your choice of model would likely depends on the data available (e.g., human expert vs historical data on electronic media). I think this Quora posting characterizes well the status of expert systems vis-a-vis neural networks today: "Artificial Intelligence: Are Expert Systems outdated?" https://www.quora.com/Artificial-Intelligence-Are-Expert-Systems-outdated https://www.quora.com/Artificial-Intelligence-Are-Expert-Sys...

trapperkeeper79 10y ago

Any tips for introductory/intermediate material on the topic? In undergrad, we used Russel and Norvig. Surely there are more advanced books or courses.

saosebastiao 10y ago
On expert systems specifically no. With the AI winter of the early 90's, not only did expert systems research die, but so did publisher interest in the topic. "Expert Systems" is effectively a dead field of study. That being said, the field of Operations Research, at least philosophically speaking, has picked up where AI researchers dropped it off. They've fully embraced the idea that human experts can model many systems extremely well, and have built incredible tools to do so: mathematical modeling and optimization, constraint programming, boolsat, graphical models, etc. Effectively speaking, if you think expert systems are cool, the next logical step is to delve into constraint programming, which is a sort of evolution of logic programming. I'd recommend the Minizinc Tutorial for a practical introduction with a nice DSL. Constraint Processing by Rina Dechter is a great intro with a more academic bent. I'd say mathematical modeling has been Ops Research's greatest success, and I'd definitely recommend Model Building with Mathematical Programming by Paul Williams. I'd also say that Baysians have embraced these ideas (that of building upon human expertise) within the field of Machine Learning far more than other ML researchers have. My recommendations here are probably less helpful...I've only ever toyed with Bayesian learning models, but never employed them professionally. But I would recommend Doing Bayesian Data Analysis by Kruschke. It was very helpful as an introductory material.
- tannhaeuser 10y ago
  Constraint logic programming indeed evolved from logic programming, but it encompasses such diverse topics as finite-domain propagation, integer interval propagation, SAT solvers, linear and restricted polynomial optimization, discrete planning and constraint satisfaction strategy meta languages that the only commonality seems the more or less Prolog-like syntactical presentation. These vastly different formalisms were cast into terms such as CP(x) (meaning constraint programming over x, for x the reals or other domains) but this doesn't give you a solution strategy (the solution strategies being as varied as math itself).
- PaulHoule 10y ago
  Today we have production rules systems such as iLog and Drools which are head and shoulders better than OPS5 and other "expert system shells" from the golden age of A.I. These are in widespread use for a few applications. Probably every bank has at least one iLog instance running for enforcing business rules. Another one is "complex event processing", where a RETE engine makes it easy to aggregate small events into larger events. Also, a few efforts, such as Inform7 and Clara have tried to push the boundaries of programming for non-experts and real-life applications. It is interesting, however, that the technology is not further applied, and a deep analysis of that could be worthwhile. For instance, RETE networks can eat "callback hell" situations for lunch, much like the complex event processing case. Instead, however, we are seeing one awful Javascript framework after another, and coroutines pushed as a very narrow answer to the problems on the server. Note that all forms of A.I. have ties to optimization. For instance, usually when you train a neural net you are minimizing some kind of an error function. Drools (and iLog) both have optimization frameworks, etc. The recent discussion of "superintelligence" has been marked by both a lack of imagination and any awareness of previous work on the subject. For instance, Rules engines are a fairly direct answer to "AI Safety" and "AI Ethics" problems that there is so much handwringing about. Most areas that require computers to be "creative" amount to some kind of multi-objective optimization, and even if rules can't make a system good, they can at least prevent the worst abuses.
  - saosebastiao 10y ago
agumonkey 10y ago
Some prolog books have chapters on how to implement ExpSys on it. I don't know much about this field, but in case you have nothing better .. ps: this book https://www.amazon.ca/Prolog-Programming-Artificial-Intelligence-4th/dp/0321417461 https://www.amazon.ca/Prolog-Programming-Artificial-Intellig...

empath75 10y ago

But I think experience has already shown that machine learning models blow away expert systems in a lot of domains.

ch4s3 10y ago
> in a lot of domains Right, but not some domains where expert systems are not only simpler to build but perform better. The can also be audited and evaluated for correctness.
giardini 10y ago
empath75 says:> "But I think experience has already shown that machine learning models blow away expert systems in a lot of domains." I don't think there have been many head-to-head comparisons. Academic researchers simply ceased using "expert system" in their research grant applications and began to use "neural network". For real-world applications your choice of model would likely depends on the data available (e.g., human expert vs historical data on electronic media). I think this Quora posting characterizes well the status of expert systems vis-a-vis neural networks today: "Artificial Intelligence: Are Expert Systems outdated?" https://www.quora.com/Artificial-Intelligence-Are-Expert-Systems-outdated https://www.quora.com/Artificial-Intelligence-Are-Expert-Sys...