4 ms·

Logic programming and productions are a wonderful idea that I studied in undergrad back in pre-2000. These days, people seem hostile towards them. People just d

Logic programming and productions are a wonderful idea that I studied in undergrad back in pre-2000. These days, people seem hostile towards them. People just don't like the idea of specifying rules. I couldn't find modern intro textbooks on the topic or classes that cover it on MOOCs. I'm very surprised and confused.

saosebastiao 10y ago
The trendy way to approach the problem these days is to throw massive amounts of data at a deep learning model and let the model try to learn the rules. And my irony detector can't help but squalk out the famous minsky koan: "What are you doing?", asked Minsky. "I am training a randomly wired neural net to play Tic-tac-toe", Sussman replied. "Why is the net wired randomly?", asked Minsky. "I do not want it to have any preconceptions of how to play", Sussman said. Minsky then shut his eyes. "Why do you close your eyes?" Sussman asked his teacher. "So that the room will be empty." At that moment, Sussman was enlightened. Machine Learning has its use cases, that's for sure, but I can't help but laugh at the person who eschews a well understood model of a well understood system by an experienced and trained expert human in favor of a magic black box that at its best might converge on the expert's understanding after enough time and data. There is no shame in building off of what is already known and understood.
- trapperkeeper79 10y ago
  Any tips for introductory/intermediate material on the topic? In undergrad, we used Russel and Norvig. Surely there are more advanced books or courses.
  - saosebastiao 10y ago
    On expert systems specifically no. With the AI winter of the early 90's, not only did expert systems research die, but so did publisher interest in the topic. "Expert Systems" is effectively a dead field of study. That being said, the field of Operations Research, at least philosophically speaking, has picked up where AI researchers dropped it off. They've fully embraced the idea that human experts can model many systems extremely well, and have built incredible tools to do so: mathematical modeling and optimization, constraint programming, boolsat, graphical models, etc. Effectively speaking, if you think expert systems are cool, the next logical step is to delve into constraint programming, which is a sort of evolution of logic programming. I'd recommend the Minizinc Tutorial for a practical introduction with a nice DSL. Constraint Processing by Rina Dechter is a great intro with a more academic bent. I'd say mathematical modeling has been Ops Research's greatest success, and I'd definitely recommend Model Building with Mathematical Programming by Paul Williams. I'd also say that Baysians have embraced these ideas (that of building upon human expertise) within the field of Machine Learning far more than other ML researchers have. My recommendations here are probably less helpful...I've only ever toyed with Bayesian learning models, but never employed them professionally. But I would recommend Doing Bayesian Data Analysis by Kruschke. It was very helpful as an introductory material.
    - tannhaeuser 10y ago
      Constraint logic programming indeed evolved from logic programming, but it encompasses such diverse topics as finite-domain propagation, integer interval propagation, SAT solvers, linear and restricted polynomial optimization, discrete planning and constraint satisfaction strategy meta languages that the only commonality seems the more or less Prolog-like syntactical presentation. These vastly different formalisms were cast into terms such as CP(x) (meaning constraint programming over x, for x the reals or other domains) but this doesn't give you a solution strategy (the solution strategies being as varied as math itself).
    - PaulHoule 10y ago
      Today we have production rules systems such as iLog and Drools which are head and shoulders better than OPS5 and other "expert system shells" from the golden age of A.I. These are in widespread use for a few applications. Probably every bank has at least one iLog instance running for enforcing business rules. Another one is "complex event processing", where a RETE engine makes it easy to aggregate small events into larger events. Also, a few efforts, such as Inform7 and Clara have tried to push the boundaries of programming for non-experts and real-life applications. It is interesting, however, that the technology is not further applied, and a deep analysis of that could be worthwhile. For instance, RETE networks can eat "callback hell" situations for lunch, much like the complex event processing case. Instead, however, we are seeing one awful Javascript framework after another, and coroutines pushed as a very narrow answer to the problems on the server. Note that all forms of A.I. have ties to optimization. For instance, usually when you train a neural net you are minimizing some kind of an error function. Drools (and iLog) both have optimization frameworks, etc. The recent discussion of "superintelligence" has been marked by both a lack of imagination and any awareness of previous work on the subject. For instance, Rules engines are a fairly direct answer to "AI Safety" and "AI Ethics" problems that there is so much handwringing about. Most areas that require computers to be "creative" amount to some kind of multi-objective optimization, and even if rules can't make a system good, they can at least prevent the worst abuses.
      - saosebastiao 10y ago
  - agumonkey 10y ago
    Some prolog books have chapters on how to implement ExpSys on it. I don't know much about this field, but in case you have nothing better .. ps: this book https://www.amazon.ca/Prolog-Programming-Artificial-Intelligence-4th/dp/0321417461 https://www.amazon.ca/Prolog-Programming-Artificial-Intellig...
- empath75 10y ago
  But I think experience has already shown that machine learning models blow away expert systems in a lot of domains.
  - ch4s3 10y ago
    > in a lot of domains Right, but not some domains where expert systems are not only simpler to build but perform better. The can also be audited and evaluated for correctness.
  - giardini 10y ago
    empath75 says:> "But I think experience has already shown that machine learning models blow away expert systems in a lot of domains." I don't think there have been many head-to-head comparisons. Academic researchers simply ceased using "expert system" in their research grant applications and began to use "neural network". For real-world applications your choice of model would likely depends on the data available (e.g., human expert vs historical data on electronic media). I think this Quora posting characterizes well the status of expert systems vis-a-vis neural networks today: "Artificial Intelligence: Are Expert Systems outdated?" https://www.quora.com/Artificial-Intelligence-Are-Expert-Systems-outdated https://www.quora.com/Artificial-Intelligence-Are-Expert-Sys...
ansible 10y ago
I'm not hostile to them per-se, but I don't see much usefulness in systems like this one these days. For some kinds of video game agents, traditional logic programming might be fine. But for anything that I would consider using in the real world, I would want to use probabilistic knowledge representation and reasoning. Even for many kinds of video games you'd want that instead. Like in a first person shooter where the agents have limited knowledge of the world state, you want to be able to reason about the other player's position and status without cheating, so that the agent can be more realistic and fair.
- TuringTest 10y ago
  I work for a living in a company that creates products to calculate timetables and schedules for transport companies. The schedules and timetables need to be exactly, defined according to a large number of requirements (hard constraints and weighted preferences). While these exact schedules could be created through probabilistic means, the systematic approach of logic search provides more consistent results than a stochastic search, and the imperative programming style allows finer control over the search algorithm than what would be easy to achieve in a pure logic language.
  - naasking 10y ago
    http://okmij.org/ftp/kakuritu/logic-programming.html http://okmij.org/ftp/kakuritu/logic-programming.html
tannhaeuser 10y ago
Some of what logic programming represented became absorbed into "semantic web" technologies (RDF, SPARQL, OWL/OWL2), and went down with it, though I think actually OWL2 (description logic) isn't half bad and has lots of vertical use in eg. bibliography systems, taxonomic meta knowledge bases for medical and other research support systems, and backends for graph databases like OpenGraph. I remember RDF being used for metadata in Linux desktop search software, and folks hated it. Prolog, OTOH, is as minimalistic and pragmatic as ever, with many implementations around, based on an ISO standard even. There's a slowly evolving initiative to come up with an extended standard library across Prolog implementations (eg. Prolog Commons). Though as you might know, Prolog doesn't solve hard problems in itself; rather, it gives you a Turing-complete language based on backtracking, negation-as-failure, closed-world reasoning, and extra-logical mechanisms as primitives to build more interesting reasoning or planning/optimization problems on.
- PaulHoule 10y ago
  The "semantic web" stuff has not really disappeared, but it hasn't thrived either. OWL2 points to a world where a production rules or logic language is fronted by a macro language that lets you say something like "x is a transitive property" rather than write "x(a,b) and x(b,c) implies x(a,c)" In fact, many RDFS and OWL implementations work exactly that way. RDFS/OWL is designed to support data integration in the sense of "this predicate is an alias of that predicate" but aren't revolutionary because it lacks the ability to do things like "CentigradeTemperature(x,T) implies KelvinTemperature(x,T+273.15)" which ordinary production rules engines do easily. Many semantic web tools have such a production rules engine hidden inside (Jena/GraphDB/...) but production rules have resisted standardization. Note that "Datalog", a subset of Prolog that is purely logical and doesn't have cuts and all of that awful stuff, has caught on, but there has never been a "Turbo Datalog" or "Common Datalog" because frequently Datalog-equivalent functionality gets build into semantic web tools, Datomic, or systems that don't claim any formal compatibility with anything else. RDF suffered a lot because of RDF/XML where it was really unclear where RDF ended and XML started. Today there is Turtle and JSON-LD, both of which are pretty ergonomic. One issue I see is that many companies have their own "semantic web"-like technologies which are their own secret sauce and competitive advantage (thus not shared) whereas there seems to be a huge amount of fear and loathing of the W3C standards process, especially amoung people who are veterans of it.

saosebastiao 10y ago

The trendy way to approach the problem these days is to throw massive amounts of data at a deep learning model and let the model try to learn the rules. And my irony detector can't help but squalk out the famous minsky koan: "What are you doing?", asked Minsky. "I am training a randomly wired neural net to play Tic-tac-toe", Sussman replied. "Why is the net wired randomly?", asked Minsky. "I do not want it to have any preconceptions of how to play", Sussman said. Minsky then shut his eyes. "Why do you close your eyes?" Sussman asked his teacher. "So that the room will be empty." At that moment, Sussman was enlightened. Machine Learning has its use cases, that's for sure, but I can't help but laugh at the person who eschews a well understood model of a well understood system by an experienced and trained expert human in favor of a magic black box that at its best might converge on the expert's understanding after enough time and data. There is no shame in building off of what is already known and understood.

trapperkeeper79 10y ago
Any tips for introductory/intermediate material on the topic? In undergrad, we used Russel and Norvig. Surely there are more advanced books or courses.
- saosebastiao 10y ago
  On expert systems specifically no. With the AI winter of the early 90's, not only did expert systems research die, but so did publisher interest in the topic. "Expert Systems" is effectively a dead field of study. That being said, the field of Operations Research, at least philosophically speaking, has picked up where AI researchers dropped it off. They've fully embraced the idea that human experts can model many systems extremely well, and have built incredible tools to do so: mathematical modeling and optimization, constraint programming, boolsat, graphical models, etc. Effectively speaking, if you think expert systems are cool, the next logical step is to delve into constraint programming, which is a sort of evolution of logic programming. I'd recommend the Minizinc Tutorial for a practical introduction with a nice DSL. Constraint Processing by Rina Dechter is a great intro with a more academic bent. I'd say mathematical modeling has been Ops Research's greatest success, and I'd definitely recommend Model Building with Mathematical Programming by Paul Williams. I'd also say that Baysians have embraced these ideas (that of building upon human expertise) within the field of Machine Learning far more than other ML researchers have. My recommendations here are probably less helpful...I've only ever toyed with Bayesian learning models, but never employed them professionally. But I would recommend Doing Bayesian Data Analysis by Kruschke. It was very helpful as an introductory material.
  - tannhaeuser 10y ago
    Constraint logic programming indeed evolved from logic programming, but it encompasses such diverse topics as finite-domain propagation, integer interval propagation, SAT solvers, linear and restricted polynomial optimization, discrete planning and constraint satisfaction strategy meta languages that the only commonality seems the more or less Prolog-like syntactical presentation. These vastly different formalisms were cast into terms such as CP(x) (meaning constraint programming over x, for x the reals or other domains) but this doesn't give you a solution strategy (the solution strategies being as varied as math itself).
  - PaulHoule 10y ago
    Today we have production rules systems such as iLog and Drools which are head and shoulders better than OPS5 and other "expert system shells" from the golden age of A.I. These are in widespread use for a few applications. Probably every bank has at least one iLog instance running for enforcing business rules. Another one is "complex event processing", where a RETE engine makes it easy to aggregate small events into larger events. Also, a few efforts, such as Inform7 and Clara have tried to push the boundaries of programming for non-experts and real-life applications. It is interesting, however, that the technology is not further applied, and a deep analysis of that could be worthwhile. For instance, RETE networks can eat "callback hell" situations for lunch, much like the complex event processing case. Instead, however, we are seeing one awful Javascript framework after another, and coroutines pushed as a very narrow answer to the problems on the server. Note that all forms of A.I. have ties to optimization. For instance, usually when you train a neural net you are minimizing some kind of an error function. Drools (and iLog) both have optimization frameworks, etc. The recent discussion of "superintelligence" has been marked by both a lack of imagination and any awareness of previous work on the subject. For instance, Rules engines are a fairly direct answer to "AI Safety" and "AI Ethics" problems that there is so much handwringing about. Most areas that require computers to be "creative" amount to some kind of multi-objective optimization, and even if rules can't make a system good, they can at least prevent the worst abuses.
    - saosebastiao 10y ago
- agumonkey 10y ago
  Some prolog books have chapters on how to implement ExpSys on it. I don't know much about this field, but in case you have nothing better .. ps: this book https://www.amazon.ca/Prolog-Programming-Artificial-Intelligence-4th/dp/0321417461 https://www.amazon.ca/Prolog-Programming-Artificial-Intellig...
empath75 10y ago
But I think experience has already shown that machine learning models blow away expert systems in a lot of domains.
- ch4s3 10y ago
  > in a lot of domains Right, but not some domains where expert systems are not only simpler to build but perform better. The can also be audited and evaluated for correctness.
- giardini 10y ago
  empath75 says:> "But I think experience has already shown that machine learning models blow away expert systems in a lot of domains." I don't think there have been many head-to-head comparisons. Academic researchers simply ceased using "expert system" in their research grant applications and began to use "neural network". For real-world applications your choice of model would likely depends on the data available (e.g., human expert vs historical data on electronic media). I think this Quora posting characterizes well the status of expert systems vis-a-vis neural networks today: "Artificial Intelligence: Are Expert Systems outdated?" https://www.quora.com/Artificial-Intelligence-Are-Expert-Systems-outdated https://www.quora.com/Artificial-Intelligence-Are-Expert-Sys...

ansible 10y ago

I'm not hostile to them per-se, but I don't see much usefulness in systems like this one these days. For some kinds of video game agents, traditional logic programming might be fine. But for anything that I would consider using in the real world, I would want to use probabilistic knowledge representation and reasoning. Even for many kinds of video games you'd want that instead. Like in a first person shooter where the agents have limited knowledge of the world state, you want to be able to reason about the other player's position and status without cheating, so that the agent can be more realistic and fair.

TuringTest 10y ago
I work for a living in a company that creates products to calculate timetables and schedules for transport companies. The schedules and timetables need to be exactly, defined according to a large number of requirements (hard constraints and weighted preferences). While these exact schedules could be created through probabilistic means, the systematic approach of logic search provides more consistent results than a stochastic search, and the imperative programming style allows finer control over the search algorithm than what would be easy to achieve in a pure logic language.
- naasking 10y ago
  http://okmij.org/ftp/kakuritu/logic-programming.html http://okmij.org/ftp/kakuritu/logic-programming.html

tannhaeuser 10y ago

Some of what logic programming represented became absorbed into "semantic web" technologies (RDF, SPARQL, OWL/OWL2), and went down with it, though I think actually OWL2 (description logic) isn't half bad and has lots of vertical use in eg. bibliography systems, taxonomic meta knowledge bases for medical and other research support systems, and backends for graph databases like OpenGraph. I remember RDF being used for metadata in Linux desktop search software, and folks hated it. Prolog, OTOH, is as minimalistic and pragmatic as ever, with many implementations around, based on an ISO standard even. There's a slowly evolving initiative to come up with an extended standard library across Prolog implementations (eg. Prolog Commons). Though as you might know, Prolog doesn't solve hard problems in itself; rather, it gives you a Turing-complete language based on backtracking, negation-as-failure, closed-world reasoning, and extra-logical mechanisms as primitives to build more interesting reasoning or planning/optimization problems on.

PaulHoule 10y ago
The "semantic web" stuff has not really disappeared, but it hasn't thrived either. OWL2 points to a world where a production rules or logic language is fronted by a macro language that lets you say something like "x is a transitive property" rather than write "x(a,b) and x(b,c) implies x(a,c)" In fact, many RDFS and OWL implementations work exactly that way. RDFS/OWL is designed to support data integration in the sense of "this predicate is an alias of that predicate" but aren't revolutionary because it lacks the ability to do things like "CentigradeTemperature(x,T) implies KelvinTemperature(x,T+273.15)" which ordinary production rules engines do easily. Many semantic web tools have such a production rules engine hidden inside (Jena/GraphDB/...) but production rules have resisted standardization. Note that "Datalog", a subset of Prolog that is purely logical and doesn't have cuts and all of that awful stuff, has caught on, but there has never been a "Turbo Datalog" or "Common Datalog" because frequently Datalog-equivalent functionality gets build into semantic web tools, Datomic, or systems that don't claim any formal compatibility with anything else. RDF suffered a lot because of RDF/XML where it was really unclear where RDF ended and XML started. Today there is Turtle and JSON-LD, both of which are pretty ergonomic. One issue I see is that many companies have their own "semantic web"-like technologies which are their own secret sauce and competitive advantage (thus not shared) whereas there seems to be a huge amount of fear and loathing of the W3C standards process, especially amoung people who are veterans of it.