5 ms·
Learn Datalog Today
- dventimi 3y ago[flagged]
- SJC_Hacker 3y agoI learned a bit of Datalog back in university, too many years ago. It was impressive how powerful the query language is. You could do in a single line what required several lines of SQL, and far more intuitively. But ... the problem is how many DBs support it, and how useful of a skill it is to know.
- FreeFull 3y agoIt's a shame that there doesn't seem to be any decent open-source implementation of Datalog. If you go for full Prolog instead of Datalog, there are several (Scryer Prolog being my personal favourite).
- cmrdporcupine 3y agohttps://en.wikipedia.org/wiki/Souffl%C3%A9_(programming_language) https://en.wikipedia.org/wiki/Souffl%C3%A9_(programming_lang... https://github.com/souffle-lang/souffle https://github.com/souffle-lang/souffle
- nezaj 3y agoHere’s a blog post showing you how to roll your own in ~100 lines of JS https://www.instantdb.com/essays/datalogjs https://www.instantdb.com/essays/datalogjs
- namingisharder 3y agoExcept that is not remotely Datalog. Would folks please stop publishing nonsense about simple relational algebra implementations claiming they are Datalog, even though there is no recursion. This is like seeing an article about how to implement context free parsing and finding one about implementing regular expressions with DFAs.
- kjqgqkejbfefn 3y ago1. Datomic - While not open-source, it has an open-source version called Datomic Free, which is a distributed database designed to enable scalable, flexible, and intelligent data storage and queries. Datomic's query language is closely inspired by Datalog. 2. DataScript - An open-source in-memory database and query engine for Clojure, ClojureScript, and JavaScript that is heavily influenced by Datalog and Datomic. 3. Crux (now XTDB) - A bitemporal database with Datalog-inspired querying capabilities. It is designed for efficient querying of historical data and offers ACID transactions. 4. Racket's miniKanren - While not strictly a database, miniKanren is an open-source logic programming extension to the Racket language, which is inspired by Datalog and can be used to manipulate and query data in a manner similar to Prolog. 5. LogicBlox - An open-source platform that combines a database system, a Datalog-based modeling language, and application server facilities. It allows developers to build complex, data-intensive applications. 6. Soufflé - A Datalog-inspired language that is designed for static analysis problems. It can be viewed as a database query language with a focus on performance, allowing for parallel execution of queries. 7. Dedalus - A Datalog-like temporal logic language used to express complex distributed systems. It is primarily a research tool but has informed the design of other Datalog-inspired systems. 8. Flora-2 - An open-source object-oriented knowledge representation and reasoning system that integrates a variant of Datalog with objects and frames. Top 3 are from the Clojure ecosystem. Additionnaly in this same space there is Datalevin & Datahike among many others
- persnickety 3y agoCozo uses Datalog for queries, and has several backends, including SQLite
- cmrdporcupine 3y agoCozo is very attractive. I just wish there was a native Rust DSL API for it, so it could be embedded in Rust programs without using datalog queries in strings.
- soraki_soladead 3y ago
- Jonovono 3y agoCozoDB: https://github.com/cozodb/cozo https://github.com/cozodb/cozo
- manu3000 3y agoyou can use Datalig within Flix https://flix.dev/ https://flix.dev/
- refset 3y agoFor comparison, I previously translated that cart parts scheduling example on the Flix homepage to Datomic-style Datalog syntax: https://gist.github.com/refset/21b3fc1dec9a6928943073809e13356d https://gist.github.com/refset/21b3fc1dec9a6928943073809e133...
- dagipflihax0r 3y agoMangle https://github.com/google/mangle https://github.com/google/mangle is an open-source implementation in golang, it was an explicit goal to make it easy to learn. Meaning: it is easy to recognize the pure datalog part, the syntax is following the good old course material. It was discussed here: https://news.ycombinator.com/item?id=33756800 https://news.ycombinator.com/item?id=33756800
- grepexdev 3y agoI thought that syntax looked familiar! Looks like Logseq uses Datalog for advanced queries. https://hub.logseq.com/features/av5LyiLi5xS7EFQXy4h4K8/getting-started-with-advanced-queries/8xwSRJNVKFJhGSvJUxs5B2 https://hub.logseq.com/features/av5LyiLi5xS7EFQXy4h4K8/getti...
- packetlost 3y agoMore specifically, Logseq uses DataScript, a Datomic-inspired Datalog engine for ClojureScript.
- achileas 3y agoThey do! IIRC they were inspired by Roam’s use of it with Clojurescript
- cldwalker 3y agoLogseq does! We at Logseq also believe in giving back so we sponsored learndatalogtoday.org when it went down - https://github.com/sponsors/jonase https://github.com/sponsors/jonase . I'd encourage others to do the same if they appreciate the author for making datalog more accessible
- brendanyounger 3y agoI wish people would stop referring to Datomic as datalog. Datomic is many things, but only the query format (Horn clauses with unification of variables, similar to prolog) has anything to do with datalog. Real datalog is far more interesting since it implicitly encodes recursion allowing you to chain rules. Rule A derives new facts, which rule B uses to derive new facts, which rules A and C use to derive new facts, and so on. Datomic has a notion of rules which are mostly syntax sugar and do not support this sort of recursive reasoning. Why is that a big deal? When rules are run automatically, you can build live, reactive systems, not just a database that sits around waiting for you to query it. Hellerstein's work at UC Berkeley (https://dsf.berkeley.edu/papers/sigrec10-declimperative.pdf https://dsf.berkeley.edu/papers/sigrec10-declimperative.pdf) explores this in some detail.
- 6gvONxR4sf7o 3y agoSounds cool. What's the complexity of running this kind of recursive reasoning? Reasonable? Can you suggest any tools to not have to implement it ourselves?
- brendanyounger 3y agoSouffle and Cozo mentioned below already implement the whole of "traditional" datalog. Percival (https://github.com/ekzhang/percival https://github.com/ekzhang/percival) has some very nice examples showing how you can interactively write and test rules on top of a datalog interpreter. Bud (http://bloom-lang.net/bud/ http://bloom-lang.net/bud/) is Hellerstein's proof of concept playground. It has bit-rotted in the past few years, but the examples are readable even if you can't easily get it working. The complexity can be quite good. You can syntactically determine when you've written linear recursion (equivalent to a for loop) vs not. Otherwise, the complexity is what you'd expect from incremental view maintenance in a normal SQL database. Which is to say O(n^k) with k being the number of relations joined, but usually much, much less with appropriate indexes and skew in the data. All the usual tricks concerning data normalization and indexes from databases apply.
- refset 3y agoRDFox offers a rather impressive sounding Datalog inferencing engine: https://www.oxfordsemantic.tech/rdfox https://www.oxfordsemantic.tech/rdfox > We present a novel approach to parallel materialisation (i.e., fixpoint computation) of datalog programs in centralised, main-memory, multi-core RDF systems. Our approach comprises an algorithm that evenly distributes the workload to cores, and an RDF indexing data structure that supports efficient, ‘mostly’ lock-free parallel updates. > Materialisation is PTIME-complete in data complexity and is thus believed to be inherently sequential. Nevertheless, many practical parallelisation techniques have been developed [...] There have been several papers and patents describing their approach, e.g. http://www.cs.ox.ac.uk/dan.olteanu/papers/mnpho-aaai14.pdf http://www.cs.ox.ac.uk/dan.olteanu/papers/mnpho-aaai14.pdf
- dang 3y agoRelated: Learn Datalog Today - https://news.ycombinator.com/item?id=27173890 https://news.ycombinator.com/item?id=27173890 - May 2021 (34 comments) Learn Datalog - https://news.ycombinator.com/item?id=19154997 https://news.ycombinator.com/item?id=19154997 - Feb 2019 (1 comment) Learn Datalog Today - https://news.ycombinator.com/item?id=17109105 https://news.ycombinator.com/item?id=17109105 - May 2018 (2 comments) Learn Datalog Today - https://news.ycombinator.com/item?id=14434457 https://news.ycombinator.com/item?id=14434457 - May 2017 (10 comments) Learn Datalog Today Ported to DataScript and Clojure (JVM) - https://news.ycombinator.com/item?id=13037199 https://news.ycombinator.com/item?id=13037199 - Nov 2016 (1 comment) Learn Datalog Today – An interactive Datomic query tutorial - https://news.ycombinator.com/item?id=6171722 https://news.ycombinator.com/item?id=6171722 - Aug 2013 (7 comments)
- Clever321 3y agoDatalog feels so much more intuitive than SQL or any other query language I've used. I'm able to write concise, complex expressions pretty easily. In a SQL-based system, there seems to be a (low) complexity metric where it's easier to write/debug/maintain what was supposed to be a 'declarative' SQL query in a functional/imperative language instead. It feels like datalog is the next evolution of a declarative query language, one that is much more declarative than SQL itself. In the "day of datomic" videos, there is a segment where Stu debugs a slow query. He does the debugging without even looking at the data model, only by rearranging the clauses. It is really, really impressive, and I can't imagine having that capability in SQL.
- brendanyounger 3y agoI greatly respect what Stu and Rich have done to make Datomic. However, they made an explicit design decision to not include a query optimizer and execute the clauses as they were written. This is usually fine since the author has some idea of what the best order is, but there are O(2^k) different permutations of clauses so doing it by hand will fail at some point (if you want the optimal ordering).
- refset 3y agoI asked Rich about his thoughts on query optimizers last year (not in the context of Datomic specifically) and his only reservation was around the practical implications for the operational experience. Specifically, that database systems should always provide the means for developers to control exactly how/when/whether existing (cached) execution plans get re-optimized, otherwise query optimizers can actually be a source of greater problems than they solve, particularly for applications with extremely rapid changes in data.
- account-5 3y agoFor the idiot in the thread, why would I use datalog (which I've never heard of before) over SQL? Having looked quickly at it just now it seems (Wikipedia article) similar to Web Ontology Language (OWL), though I believe datalog may have been around long before owl.
- brendanyounger 3y agoOn a syntax level, parsing, generating, and templating datalog is _much_ simpler than doing the same to SQL. DBT would never exist if every SQL database accepted datalog queries and SQL injection attacks would be rare to non-existent. The more interesting answer is to think of datalog as making it easy to encode nearly all of your application logic as a bunch of self-referencing, incrementally updated, materialized views. Some examples: # view of Users table for currently logged in user LoggedInUserView(name, email, id) :- Users(id: payload["userId"], name, email), Cookies(name: "login", payload). # view of Users for admin AdminUserView(name, email, id) :- Users(id, name, email), Cookies(name: "login", payload), payload["isAdmin"] = true. # posts a user can see PostsView(title, content, id) :- Posts(title, content, public: true). PostsView(title, content, id) :- Posts(title, content, author: payload["userId"]), Cookies(name: "login", payload). And then you write your UI code to explicitly reference these derived views rather than manually wrapping an API around querying the Posts table and doing the filtering. The examples above can be neatly replicated in Supabase or Postgraphile (the OG of auto-generated GraphQL over Postgres), but you can do a lot more with datalog as a language. The Hellerstein paper mentioned above is a good starting place.
- 7thaccount 3y agoI can't really understand this without seeing the equivalent SQL as I don't understand datalog. Is this SQL query similar to the first datalog query you listed? I apologize for how HN formatted the below and my lack of understanding for how to get around it. Select u.name, u.email, u.id From Users u Join Cookies c on u.name = c.name
- brendanyounger 3y ago
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- philzook 3y agoOne of the easiest to get started on Datalog in my opinion is really clingo https://potassco.org/clingo/ https://potassco.org/clingo/ , which can be pip installed and has python bindings. Answer Set Programming goes beyond datalog, but it holds datalog semantics as a sublanguage. It is unfortunate this is not well advertised. pip install clingo #! clingo edge(1,2). edge(2,3). path(X,Y) :- edge(X,Y). path(X,Z) :- edge(X,Y), path(Y,Z). Datalog has some really cool use cases for program analysis (more or less relations represent an overapproximation of possible values variables can have). The declarative constrained nature of the language enables incrementalization, composition of analysis, and connection back to other formal aspects like types. Souffle is very good, logica has a nice install story. https://colab.research.google.com/drive/1KJ6xKSwpw5FWWkvUOyBCpAcU_jVZV7Gs?usp=sharing https://colab.research.google.com/drive/1KJ6xKSwpw5FWWkvUOyB... . Many other interesting datalog systems. - https://www.philipzucker.com/notes/Languages/datalog/ https://www.philipzucker.com/notes/Languages/datalog/ - https://www.philipzucker.com/notes/Logic/answer-set-programming/ https://www.philipzucker.com/notes/Logic/answer-set-programm...