8 ms·

PostgreSQL-Prolog: A Prolog library to connect to PostgreSQL databases

99 points by triska 5y ago

hirundo 5y ago
The Datalog query language is a subset of Prolog. I wonder how hard it would be to translate Datalog into SQL and query PostgreSQL with that. XTDB (aka Crux), which uses Datalog, supports some limited SQL querying. How limited would it be to go in the other direction?
- dgb23 5y ago
  If you’re ok with typecasting then it’s relatively straight forward. I think postgres has some convenient utility around that.
- maweki 5y ago
  With recursive common table expressions it is really straightforward. You just have to manually stratify your datalog program to find out the order for the CTE. And then you just write down all the rules. edit: As long as queries aren't recursive in a difficult manner, like co-recursive or have multiple recursive calls in the same rule.
koeng 5y ago
This is extremely cool and I will be testing it out. I’m building large systems for handling biological data, and have wanted to use prolog for that use, but couldn’t fit all of my data in RAM. This could help me a bunch.
- triska 5y ago
  Importantly for this use case, Scryer Prolog represents lists of characters very compactly internally, using a sequence of raw bytes! It is the first Prolog system to use this efficient representation. This helps considerably when processing huge amounts of data, for example when parsing large text corpora with Prolog's built-in grammar mechanism, definite clause grammars (DCGs).
  - tannhaeuser 5y ago
    Sure about that? It is/was pretty common to keep strings as long as possible, and convert to list of chars lazily only when absolutely necessary.
    - triska 5y ago
      The key innovation of Scryer Prolog in this respect is that the system internally, i.e., within Rust, represents each string as a sequence of raw bytes (UTF-8 encoded, 0-terminated), and to Prolog programs, the string appears as if it were a list of characters (i.e., atoms of length 1). So, no conversion is necessary at all: The compact internal representation is used whenever possible, and Prolog programs only see lists of characters all of the time, making common predicates such as append/3 and length/2, and most importantly DCGs, automatically usable for reasoning about strings. For example, the string "abc" takes only 4 bytes in this representation, and the unification "abc" = [a,b,c] succeeds. Indeed, we have: ?- write_canonical("abc"). '.'(a,'.'(b,'.'(c,[]))) true. On 64-bit systems, a conventional internal representation of "abc" as the compound term .(a, .(b, .(c, []))) takes 8 bytes per functor (each '.'/2 takes 8 bytes, i.e., 1 cell in the WAM), 8 bytes per character (as a pointer to the atom table, again 1 cell in the WAM), and 8 bytes for each tail. All Prolog systems before Scryer Prolog use 16 to 24 times as much memory to represent a list of characters. As of recent, Trealla Prolog also uses the efficient representation, at least for fully instantiated strings. Scryer Prolog can also represent partial strings such as [a,b,c|Ls] with the efficient encoding.
      - tannhaeuser 5y ago
        > All Prolog systems before Scryer Prolog use 16 to 24 times as much memory to represent a list of characters. That's not correct. As I said, it's a pretty basic optimization to store strings as native char buffers, and then only convert to explicit list representation if a variable bound to a string in that way is used in eg unification against a (partial) list, or subjected to a list (or other non-string) builtin. Definitely not new with Scryer Prolog; maybe true of F/OSS Prologs though I doubt it.
- still_grokking 5y ago
  A little bit off-topic, but still related: I could imagine the Flix language being a good fit for that use case. https://flix.dev/ https://flix.dev/ It has first-class Datalog constraints as a language feature! As being a FP language it's of course good at modeling data. At the same time it's fast enough for heavy tasks: It runs on the JVM, having also access to its rich ecosystem.
infogulch 5y ago
Wow pretty cool. The next thing it needs is query params so you can avoid sql injection / unreliable user-maintained escaping rules.
deleted 5y ago
[deleted]
EamonnMR 5y ago
Very cool. I really think that prolog's future hinges on its interop with sql.
TedDoesntTalk 5y ago
Last I used prolog was about 1990 (more than 30 years ago). I’m glad to see it still evolves!
void_mint 5y ago
I love Prolog and I love Postgres.
maweki 5y ago
Prolog and side-effects are a really unfortunate pairing. What I would love is when backtracking over a side-effect, that effect would be taken back. With transactions that might even be possible. As it stands, quering (pseudocode) "connect(C), insert(C, data), false" has no solution but side-effects. SLD-resolution probably just does not work with side-effects.
- deleted 5y ago
  [deleted]
- Twisol 5y ago
  In some sense, whenever there's a choice (or a checkpoint) between two branches, each branch should morally get its own copy of the world; and effects should only leak out when both branches incur the same observable effect. (This is motivated by the logical law `(A v B) ^ (A -> C) ^ (B -> C) -> C`.) Of course, that's hard to do when interacting with external systems!
  - maweki 5y ago
    Do you know of any work on this? I have a prototype implementation for a side-effects prolog library with a similar semantics and I'm working on the corresponding paper.
    - Twisol 5y ago
      Not directly, no -- it's at least part of my own musings on logical systems. Probably the closest thing in my mind is Kuper et al.'s LVars, and I've long wanted to extend its join-semilattices to meets, but I left academia for a rather demanding day job ^_^; I do think consensus algorithms have something deeply in common here, as CALM (Consistency As Logical Monotonicity) seems to cover the space of join-semilattice structures pretty crisply, and CALM is all about when you don't need consensus. But again, just my own musings.
dreamcompiler 5y ago
This appears to merely be a Prolog wrapper around SQL queries which still have to be written explicitly. Prolog itself is a dramatically more expressive query language than SQL; using Prolog instead of SQL would be a big win (IMHO) but this library doesn't appear to do that.
fraber 5y ago
The following post is from October 2020, but I remember having used an interface to PostgreSQL more than 10 years ago: https://swi-prolog.discourse.group/t/swi-prolog-connecting-to-postgresql-via-odbc/2404 https://swi-prolog.discourse.group/t/swi-prolog-connecting-t... For those who don't get it: side-effects allow for the power of Prolog, just don't use them if you don't want them...

99 points by triska 5y ago

The Datalog query language is a subset of Prolog. I wonder how hard it would be to translate Datalog into SQL and query PostgreSQL with that. XTDB (aka Crux), which uses Datalog, supports some limited SQL querying. How limited would it be to go in the other direction?

dgb23 5y ago
If you’re ok with typecasting then it’s relatively straight forward. I think postgres has some convenient utility around that.
maweki 5y ago
With recursive common table expressions it is really straightforward. You just have to manually stratify your datalog program to find out the order for the CTE. And then you just write down all the rules. edit: As long as queries aren't recursive in a difficult manner, like co-recursive or have multiple recursive calls in the same rule.

koeng 5y ago

This is extremely cool and I will be testing it out. I’m building large systems for handling biological data, and have wanted to use prolog for that use, but couldn’t fit all of my data in RAM. This could help me a bunch.

triska 5y ago
Importantly for this use case, Scryer Prolog represents lists of characters very compactly internally, using a sequence of raw bytes! It is the first Prolog system to use this efficient representation. This helps considerably when processing huge amounts of data, for example when parsing large text corpora with Prolog's built-in grammar mechanism, definite clause grammars (DCGs).
- tannhaeuser 5y ago
  Sure about that? It is/was pretty common to keep strings as long as possible, and convert to list of chars lazily only when absolutely necessary.
  - triska 5y ago
    The key innovation of Scryer Prolog in this respect is that the system internally, i.e., within Rust, represents each string as a sequence of raw bytes (UTF-8 encoded, 0-terminated), and to Prolog programs, the string appears as if it were a list of characters (i.e., atoms of length 1). So, no conversion is necessary at all: The compact internal representation is used whenever possible, and Prolog programs only see lists of characters all of the time, making common predicates such as append/3 and length/2, and most importantly DCGs, automatically usable for reasoning about strings. For example, the string "abc" takes only 4 bytes in this representation, and the unification "abc" = [a,b,c] succeeds. Indeed, we have: ?- write_canonical("abc"). '.'(a,'.'(b,'.'(c,[]))) true. On 64-bit systems, a conventional internal representation of "abc" as the compound term .(a, .(b, .(c, []))) takes 8 bytes per functor (each '.'/2 takes 8 bytes, i.e., 1 cell in the WAM), 8 bytes per character (as a pointer to the atom table, again 1 cell in the WAM), and 8 bytes for each tail. All Prolog systems before Scryer Prolog use 16 to 24 times as much memory to represent a list of characters. As of recent, Trealla Prolog also uses the efficient representation, at least for fully instantiated strings. Scryer Prolog can also represent partial strings such as [a,b,c|Ls] with the efficient encoding.
    - tannhaeuser 5y ago
      > All Prolog systems before Scryer Prolog use 16 to 24 times as much memory to represent a list of characters. That's not correct. As I said, it's a pretty basic optimization to store strings as native char buffers, and then only convert to explicit list representation if a variable bound to a string in that way is used in eg unification against a (partial) list, or subjected to a list (or other non-string) builtin. Definitely not new with Scryer Prolog; maybe true of F/OSS Prologs though I doubt it.
still_grokking 5y ago
A little bit off-topic, but still related: I could imagine the Flix language being a good fit for that use case. https://flix.dev/ https://flix.dev/ It has first-class Datalog constraints as a language feature! As being a FP language it's of course good at modeling data. At the same time it's fast enough for heavy tasks: It runs on the JVM, having also access to its rich ecosystem.

infogulch 5y ago

Wow pretty cool. The next thing it needs is query params so you can avoid sql injection / unreliable user-maintained escaping rules.

deleted 5y ago

[deleted]

EamonnMR 5y ago

Very cool. I really think that prolog's future hinges on its interop with sql.

TedDoesntTalk 5y ago

Last I used prolog was about 1990 (more than 30 years ago). I’m glad to see it still evolves!

void_mint 5y ago

I love Prolog and I love Postgres.

maweki 5y ago

Prolog and side-effects are a really unfortunate pairing. What I would love is when backtracking over a side-effect, that effect would be taken back. With transactions that might even be possible. As it stands, quering (pseudocode) "connect(C), insert(C, data), false" has no solution but side-effects. SLD-resolution probably just does not work with side-effects.

deleted 5y ago
[deleted]
Twisol 5y ago
In some sense, whenever there's a choice (or a checkpoint) between two branches, each branch should morally get its own copy of the world; and effects should only leak out when both branches incur the same observable effect. (This is motivated by the logical law `(A v B) ^ (A -> C) ^ (B -> C) -> C`.) Of course, that's hard to do when interacting with external systems!
- maweki 5y ago
  Do you know of any work on this? I have a prototype implementation for a side-effects prolog library with a similar semantics and I'm working on the corresponding paper.
  - Twisol 5y ago
    Not directly, no -- it's at least part of my own musings on logical systems. Probably the closest thing in my mind is Kuper et al.'s LVars, and I've long wanted to extend its join-semilattices to meets, but I left academia for a rather demanding day job ^_^; I do think consensus algorithms have something deeply in common here, as CALM (Consistency As Logical Monotonicity) seems to cover the space of join-semilattice structures pretty crisply, and CALM is all about when you don't need consensus. But again, just my own musings.

dreamcompiler 5y ago

This appears to merely be a Prolog wrapper around SQL queries which still have to be written explicitly. Prolog itself is a dramatically more expressive query language than SQL; using Prolog instead of SQL would be a big win (IMHO) but this library doesn't appear to do that.

fraber 5y ago

The following post is from October 2020, but I remember having used an interface to PostgreSQL more than 10 years ago: https://swi-prolog.discourse.group/t/swi-prolog-connecting-to-postgresql-via-odbc/2404 https://swi-prolog.discourse.group/t/swi-prolog-connecting-t... For those who don't get it: side-effects allow for the power of Prolog, just don't use them if you don't want them...