4 ms·

Logic programming offers a good foundation for anything that people call "rule engines." Within logic programming, there is some variation on the degree of decl

Logic programming offers a good foundation for anything that people call "rule engines." Within logic programming, there is some variation on the degree of declarativeness. Datalog is arguably the minimal core logic programming, similar to what the lambda calculus achieves for functional programming. Unfortunately, it has been forgotten outside of database and query processing realm. A resurgence has happened in recent years, as PL researchers and also industry have discovered the virtues of datalog (e.g. Flix, DataFun). My own attempt at making this more widely known is here https://github.com/google/mangle https://github.com/google/mangle, a language from the datalog family and its implementation as a go library. As the example shows: plain "rules" (or: plain datalog) is rarely enough to capture everything that one wants to express: the question then is, how to combine a pure declarative "kernel" with more general purpose programming (e.g. mapping a list). PROLOG offered one answer, already in the 1980s, but I fully reject it: the fact that the writing a program in the wrong order with negation and recursion makes it non-terminating is not something we'd want everyone to deal with. Datalog with stratified recursion is somewhat better, as "layers of rules" is a concept that is easy to understand. In mainstream programming languages, the possibility of writing non-terminating programs also exists, but is rarely an issue. That is why I believe a good combination of declarative and general-purpose has to make it really easy to recognize which parts of a program are in the declarative, terminating, safe kernel and which parts require more attention from the programmer.

alecco 3y ago
The problem for Datalog adoption is there are dozens of incompatible Datalog engines. And most (all?) are not good or well maintained. It's sad.
- burakemir 3y ago
  I think it is helpful to see datalog as a formal, conceptual kernel (or "toy programming language" in the famous Alice Book "Foundations of Databases"). When we look at the functional programming languages, we do not usually see is as a dozen of incompatible implementations of lambda calculus. The kind of standardization that happened for SQL and PROLOG certainly helped spread its use, but it is a very differently world today. Developers can easily do their own take on DBs' data warehouse by serving from memory or existing DBs or files. If you do not see it as a programming language, but a way to think about computation, then we are in the ballpark of "rule engines:" there are of course innumerable implementations of things that are called rule engines. Like the post, "rules" make knowledge explicit, but we wouldn't even think about the possibility that all the folks who wrote or use these rule engine implementations use datalog syntax. It is more the semantics, structuring the problem as facts and rules, that counts. Of course having a common syntax helps and matters in getting the message out: that there is a good foundation. But how to add aggregation or user-defined functions is not settled (it is also not settled for SQL, many vendor-specific extensions, syntaxes...) and I think today's business world does not provide much incentives for agreeing and standardizing. Maybe academia will be able to help over time, by teaching newer generations who will then pick standard syntax for their next PL because they are familiar. For academia to be interested in an applied PL topic, it has to be reasonably formal, derivable from first principles and teachable.
z5h 3y ago
Were you able to make use of the fixpoint optimization ideas in Datafun? I thought it was a great idea but the project seems to have stalled.
- burakemir 3y ago
  Mangle uses seminaive evaluation which is a standard, not very fancy but incremental way of computing fixpoint. I believe Datafun uses the same but there it requires more thinking since it interacts with other language features. I need to reread the paper. There are really many low level ways to optimize incremental fixpoint computation in practice since the key step involves a look up whether a fact has already been computed. However, having evaluation access the FactStore through an interface means a developer can hook up various implementations and that flexibility can matter more than the last bit of performance.
cced 3y ago
Would it be a good candidate for modelling flows w.r.t economics? For instance, country 1 has currency pinned to country 2, import/exports, etc.
- burakemir 3y ago
  You mean datalog or Mangle? Mangle adds a bunch of things that make various things easy to model in the sense that you have entities and connections and data and query that. Maybe one could call this a "knowledge hypergraph". Datalog is very basic and everyone needs aggregations and structured data so Mangle also supports aggregations, structured data and also some function calls. When you use these it is clearly no longer datalog, but it is easy to see what part is datalog and what part is "more." To qualify this a little: Mangle does not provide much for mutability (neither the "spec" which is a bit implicit, nor the implementation) so if you want to make a real DB with inserts or an RPC interface you have to code that yourself. I find having a readable source file and running queries is good for playing around and also may cover many use cases for small DBs with static or slow changing data, or configuration.
riku_iki 3y ago
it looks like authors of the project are not employed by Google, how this project managed to be under google at github?..

alecco 3y ago

The problem for Datalog adoption is there are dozens of incompatible Datalog engines. And most (all?) are not good or well maintained. It's sad.

burakemir 3y ago
I think it is helpful to see datalog as a formal, conceptual kernel (or "toy programming language" in the famous Alice Book "Foundations of Databases"). When we look at the functional programming languages, we do not usually see is as a dozen of incompatible implementations of lambda calculus. The kind of standardization that happened for SQL and PROLOG certainly helped spread its use, but it is a very differently world today. Developers can easily do their own take on DBs' data warehouse by serving from memory or existing DBs or files. If you do not see it as a programming language, but a way to think about computation, then we are in the ballpark of "rule engines:" there are of course innumerable implementations of things that are called rule engines. Like the post, "rules" make knowledge explicit, but we wouldn't even think about the possibility that all the folks who wrote or use these rule engine implementations use datalog syntax. It is more the semantics, structuring the problem as facts and rules, that counts. Of course having a common syntax helps and matters in getting the message out: that there is a good foundation. But how to add aggregation or user-defined functions is not settled (it is also not settled for SQL, many vendor-specific extensions, syntaxes...) and I think today's business world does not provide much incentives for agreeing and standardizing. Maybe academia will be able to help over time, by teaching newer generations who will then pick standard syntax for their next PL because they are familiar. For academia to be interested in an applied PL topic, it has to be reasonably formal, derivable from first principles and teachable.

z5h 3y ago

Were you able to make use of the fixpoint optimization ideas in Datafun? I thought it was a great idea but the project seems to have stalled.

burakemir 3y ago
Mangle uses seminaive evaluation which is a standard, not very fancy but incremental way of computing fixpoint. I believe Datafun uses the same but there it requires more thinking since it interacts with other language features. I need to reread the paper. There are really many low level ways to optimize incremental fixpoint computation in practice since the key step involves a look up whether a fact has already been computed. However, having evaluation access the FactStore through an interface means a developer can hook up various implementations and that flexibility can matter more than the last bit of performance.

cced 3y ago

Would it be a good candidate for modelling flows w.r.t economics? For instance, country 1 has currency pinned to country 2, import/exports, etc.

burakemir 3y ago
You mean datalog or Mangle? Mangle adds a bunch of things that make various things easy to model in the sense that you have entities and connections and data and query that. Maybe one could call this a "knowledge hypergraph". Datalog is very basic and everyone needs aggregations and structured data so Mangle also supports aggregations, structured data and also some function calls. When you use these it is clearly no longer datalog, but it is easy to see what part is datalog and what part is "more." To qualify this a little: Mangle does not provide much for mutability (neither the "spec" which is a bit implicit, nor the implementation) so if you want to make a real DB with inserts or an RPC interface you have to code that yourself. I find having a readable source file and running queries is good for playing around and also may cover many use cases for small DBs with static or slow changing data, or configuration.

riku_iki 3y ago

it looks like authors of the project are not employed by Google, how this project managed to be under google at github?..