9 ms·
Logica, a novel open-source logic programming language
- tofflos 6y agoI recently gave Rego a shot but had a difficult time grokking it. It's also inspired from Datalog. How would you say Logica compares to Rego? > It supports modules and imports, it can be used from an interactive Python notebook and it even makes testing your queries natural and easy. I don't see any examples of how to do tests in the announcement. Consider adding some.
- tannhaeuser 6y agoNice to see Datalog being validated by a big name, though I don't see what's modern about Logica in particular, or why one should use it over plain Datalog (as syntactical Prolog subset) when the available backends are restricted to SQL rewriters or locked-in to BigQuery. Will have to look into the model for aggregate queries which I guess is a selling point for Logica (as is modularization/composition with optimization), and a weak point since typically neither in Datalog (the decidable logic fragment) nor portable. Edit: also I find the title a bit grandiose since this isn't about Logic Programming in general, but only database querying
- devit 6y agoIt seems that it's open source (Apache 2.0) and can generate SQL for PostgreSQL and SQLite in addition to BigQuery.
- anentropic 6y agoOh that's good, there was no mention of anything other than BigQuery in the front-matter but looks like it's in there: https://github.com/EvgSkv/logica/blob/main/compiler/dialects.py https://github.com/EvgSkv/logica/blob/main/compiler/dialects...
- deleted 6y ago[deleted]
- tannhaeuser 6y agoYeah but that reduces Logica outside Google to SQL rewriting. When the weak point of SQL isn't so much the syntax but the scalability and expressiveness limitations for big data and document data (fixation on strong ACID/CAP guarantees, schemas). SQL syntax has strong points, too; one being that it's not just a query but also update/batch language with ACID semantics; another being that it's standardized with a range of mature options available. Consider also the practical side: using Datalog as merely "prettier SQL" still doesn't allow you to dynamically define data properties or go schema-less as in RDF or other logic/deductive graph databases. Whenever you want a new column, you must execute DDLs (ALTER TABLE ADD COLUMN) also leading to forced commits, overly broad permissions, chaotic backup procedures and/or code artefacts containing the dreaded SELECT * syntax. Also, parsing Datalog queries, reformulating into SQL, then re-parsing SQL in the DB engine isn't the most efficient thing. Basically, the workflows and use cases for SQL RDBMSs and Datalog/graph databases are not the same, and if you're using one on top of the other, you're getting the intersection of possibilities but the union of problems, as is well known from O/R mappers );
- YeGoblynQueenne 6y ago>> Whenever you want a new column, you must execute DDLs (ALTER TABLE ADD COLUMN) also leading to forced commits, overly broad permissions, chaotic backup procedures and/or code artefacts containing the dreaded SELECT * syntax. I don't understand what you mean here. With datalog, if you have a predicate person(Name, Age, Height) and you want to add an argument (a "column") for income, you can simply create a new predicate person(Name, Age, Height, Income). Or, if you want to avoid duplication, you can write a rule to combine the information in two (or more) predicates: person(Name, Age, Height, Income):- person(Name, Age, Height) ,person(Name, Income). You don't need to remove the old predicate. That's actually one case where Datalog works better than SQL, that only allows "rows" i.e. "facts" (in Datalog parlance) but not "rules" that establish relations _between tables_. That's true for Datalog, based on what I know about Prolog (not a Datalog expert!). I don't know how it works in Logica, but from reading the article above I think the semantics would be similar. >> Basically, the workflows and use cases for SQL RDBMSs and Datalog/graph databases are not the same, and if you're using one on top of the other, you're getting the intersection of possibilities but the union of problems, as is well known from O/R mappers ); That's funny. But I don't think it applies here. SQL and datalog are both relational. The difference is that Datalog lets you define relations over tables ("rules"), not just relations over data ("facts"/"rows"). Essentially, SQL is one half of datalog's relational semantics - only information without reasoning. Datalog adds reasoning on top, but the reasoning is still, well, relational (facts, rules and queries are all relations). There's no impedence mismatch here, as in trying to fit relational data into a non-relational program.
- cpdean 6y ago> ...or why one should use it over plain Datalog... I have been looking for examples of how to use a practical implementation of Datalog for years and the closest I've come to is actually miniKanren instead. Could you point me to codebases that productively use Datalog internally?
- RBerenguel 6y agoWhat example with miniKanren have you found? This is an area I have a passing interest but never find the time to delve deep enough to find anything shiny enough
- cpdean 6y agoThe most interesting open source example I have found is a hy-lang minikanren that implements a tree-rewriter which lets you encode equivalent implementations of code snippets. It's a code-linter that will automatically simplify code you write via higher-level rules written in mk https://github.com/algernon/hydiomatic https://github.com/algernon/hydiomatic I've heard informally there are many companies relying on clojure/core.logic to rewrite complicated business rules for the sake of filtering and constraint solving problems in applications, but I do not know of any open source examples to reference. edit: i accidentally linked to a dependency of the project i meant to link to. originally linked to the mk implementation: https://github.com/algernon/adderall https://github.com/algernon/adderall
- RBerenguel 6y agoThanks!
- hcarvalhoalves 6y agoDatomic: https://docs.datomic.com/cloud/query/query-data-reference.html https://docs.datomic.com/cloud/query/query-data-reference.ht... Datascript: https://github.com/tonsky/datascript https://github.com/tonsky/datascript Crux: https://opencrux.com/main/index.html https://opencrux.com/main/index.html
- samuell 6y agoMy hope would be mainly that this can get datalog into mainstream use, and soon get more (and more mature) libraries created by the community. That is in itself very exciting to me though. Would be pretty awesome if we could have logica (or something similar) for dataframes (including pandas), and so could build pipelines of transformations-via-queries on those. (If there is anything like this already implemented, I'm all ears!).
- harperlee 6y agoNot a great introduction to the language, IMHO. There is not a clear use of logic to automatically reason about anything, just query composition. It seems the language is much more powerful than what this introduction makes it to be! > English words (...) often capitalized to keep the old-fashioned COBOL spirit of the 70s alive! I like logic programming a lot but a convention not technologically enforced is a poor reason to argue for a language change. When arguing about SQL limited abstraction capabilities that space would have been better spent talking about CTE limitations, for example. Also: > To make things worse, SQL code is rarely tested, because “testing SQL queries” sounds rather esoteric to most engineers, at best So nonexisting best practices require a language change, apparently. It was also not showcased how this can be done in Logica, beyond the table mocking that could be done with a "with xxx as (values a, b, c) select (query to be tested)" approach in sql.
- da39a3ee 6y ago> So nonexisting best practices require a language change, apparently. It was also not showcased how this can be done in Logica. Look for the section containing the text "As a final example, let us mock the comments table, in a unittest of a query." That demonstrates mocking and is explicitly pointing towards testing. The article is only a high-level intro document.
- harperlee 6y agoSorry, I was editing my post in parallel. I see what you are pointing to. I'm sure the language has been thought out for much more time than my reading of the post - I'm just complaining about this "presentation" post. It's not clear from the example what the language provides over just redefining the table with mock values for testing. I´m sure Logica has more than what's stated in here, it's just not a good example (IMHO).
- deleted 6y ago[deleted]
- mikkom 6y agoI have a hard time understanding what problem this solves versus SQL? This really should have demonstration of some of the actual use cases where this shines vs SQL. Also: SQL-92 has values clause which makes the example provided a little bit silly, you could just use values (2),(3),(5) Another example they gave is a 5-line (excluding imported code) mocking code with a comment "compare that to what you would have to do to achieve the same using bare SQL". Okay.. select * from (values (1, 'hello'), (2, 'logic'), (3, 'programming')) as mocktable(user_id, comment);
- da39a3ee 6y agoThat's explained carefully in the first 5 paragraphs, especially paras 1, 3, 4, and 5.
- mikkom 6y agoYes I read it. Lots of abstract talk about modularity and how SQL is so bad because people tend to use all caps without demonstrating how and what this whole new programming language can do better. Note that I'm not claiming it can't do but I would be interested for the authors to point out what the actual benefits are. All I see from their examples that clauses in Logica are much longer than SQL counterparts and that they are importing modules which (I assume) re-define already defined schemas which brings all kinds of different dependency problems that I'm not going to go in here..
- zaphar 6y agoThe document assumes you have used SQL in anger before and run into it's lack of modularity. If you haven't run into those things then you'll miss the point. But if you have then this looks really interesting.
- gweinberg 6y agoYeah, the quip about many people preferring to write SQL keywords in all caps as if that were somehow a defect of SQL struck me as being astonishingly asinine.
- js8 6y agoI have yet to check it out, but it's very cool when people try to rethink SQL, and also revive Datalog. I (morally) support this effort. Myself, I would like to see data query/manipulation language as a total functional language, possibly based on the idea of categorical data transformations: https://www.categoricaldata.net/ https://www.categoricaldata.net/ Also - bit of a rant - if you're creating a new programming language, consider making syntax and semantics separate in the specification. Lots of people get hung up on arguing about language syntax but it's really semantics differences that are important for compatibility. Lot of new languages comes up only to fix syntactic problems with existing languages but create small semantic differences in the process, making automated translation from and to existing languages difficult. I wish we could move to a world where syntax and semantics in programming languages are discussed separately from each other.
- chriswarbo 6y ago> Also - bit of a rant - if you're creating a new programming language, consider making syntax and semantics separate in the specification. I agree. Implementations should accept a stable, machine-friendly format (doesn't matter which; JSON, s-expressions, or even XML would do). If they also accept a human-friendly format, there should be a standard/built-in translation from human format -> machine format (optionally the other way too). This way, we can always convert random real-world code (scraped from GitHub, or whatever) into a language-agnostic format (yes Python has an `ast` module; that doesn't help a Python linter written in something else, like Go); tools can manipulate this format without having to care about the surface syntax (e.g. linting/doc-gen/static-analysis/versioning/diffing/refactoring/macros/etc.); the output of such tools can always be fed back into the main implementation to compile/run/type-check/syntax-check/etc.
- thechao 6y agoI agree with the GP. It frustrates me when the surface syntax and the semantics are not split apart. The constraints of old compiler technology (pre-3rd-millennium) continue to dictate the overall architecture of compilers. LLVM is a tiny baby step in the right direction. Note, I'm not advocating for a single solution, here. I'm advocating that language authors should always think in terms of a front, middle, and back-end: front is surface syntax (their preferred one?); middle is the semantics, with a prescribed API; and, the backend is the implementation side — nicely abstracted by a 2nd API. That way it gives those of us stuck in not-your-language a fighting chance to integrate Your Cool Thing™.
- deleted 6y ago[deleted]
- BenoitP 6y agoNice to see some new initiatives in this domain (or old ideas resurface), but there is a long way towards mainstream adoption IMHO: * My business clients and I speak SQL together. I don't see them learning a new language. I don't have the authority nor any will to force them to. * I can spin up a container for testing business rules logic (and often share the results back to the client: here is what the impact of updating rule A is, rows of type W will be affected in this way). Even though SQL has ceremony/verbosity, I'd rather see the standard be evolved. My clients and I could pick it up more easily. ---- That's great for BigQuery though. You can't spin up a BigQuery docker container anyway, and testing with another schema/project is risky while you have interns around.
- pgt 6y agoDatomic-flavoured Datalog: http://www.learndatalogtoday.org/ http://www.learndatalogtoday.org/
- xvilka 6y agoThere's Datomic[1] though it's proprietary. Regarding the Prolog, I hope they will take a look at newly emerging "GHC of Prolog" in Rust - Scryer[2]. [1] https://www.datomic.com/ https://www.datomic.com/ [2] https://github.com/mthom/scryer-prolog https://github.com/mthom/scryer-prolog
- cabite 6y agoAlso there are ongoing efforts in the Clojure community to make datomic queries reactive. Pluging this into a react-like architecture can lead to writing code that is 100% declarative when dealing with updating the UI according to the DB or vice-versa
- harperlee 6y agoWhat do you mean with "GHC of Prolog"? I don´t know a lot about the Haskell ecosystem so I dont know what that implies. Edit: Never mind, the Scryer Prolog github page states it as follows: Scryer Prolog aims to become to ISO Prolog what GHC is to Haskell: an open source industrial strength production environment that is also a testbed for bleeding edge research in logic and constraint programming, which is itself written in a high-level language.
- psibi 6y agoRelated question: Is there a good resource to learn logic programming (using Prolog or something like that) for an experienced programmer ?
- namelosw 6y agoI recommend taking a look at 'Learn Datalog Today'[0] first. Although it's just Datalog (in S-expression form) which is a subset of Prolog but makes people get the gist of it very quickly. For Prolog me too wondering if there's a great source. But I have read 'the Reasoned Schemer', it used a simple Scheme-based logical programming language for teaching purposes and it's very educative and entertaining. [0] http://www.learndatalogtoday.org/ http://www.learndatalogtoday.org/
- harperlee 6y agoThe Reasoned Schemer is a highly praised book for that. It teaches minikanren and builds it from scratch. I personally did not like a lot its "socratic" style. The best way to learn it, in my opinion, is to implement microkanren, which is micro by design for teaching purposes. It is small enough to fit in your head, understand what's unification, and play with it. Then you can jump into other implementations. If you like clojure, you can use core.logic, although documentation is not abundant. More prolog-related, The Power of Prolog https://www.metalevel.at/prolog https://www.metalevel.at/prolog has been praised here several times.
- tannhaeuser 6y agohttps://news.ycombinator.com/item?id=18188003 https://news.ycombinator.com/item?id=18188003 My recommendation would be to learn the real thing, not an almost, sort-of Prolog that's actually a LISP dialect in disguise.
- billfruit 6y agoWithout any hesitation I would recommend the Ivan Bratko book, it is so densely filled with knowledge. Prolog is vastly different from other programming systems and some of the concepts take a bit of exposition before they sink in, and this book very much strives to explain quite a lot of mysterious things. Prolog Programming for Artificial Intelligence by Ivan Bratko.
- trurl 6y agoIt seems misleading to call this "Datalog". The GitHub repo even says "among database theoreticians Datalog and SQL are known to be equivalent", which is absolutely wrong without qualification. Some flavors of SQL will have recursive extensions so that they could be considered equivalent, but that is not true in general. I can't find any mention of recursion on the original blog post or the GitHub page. Without recursion it isn't Datalog.
- layer8 6y agoSQL 99 has recursion via recursive CTEs, so the claim is probably valid.
- trurl 6y agoFair enough, but to a database theoretician SQL means "non-recursive conjunctive queries". And I skimmed the tutorial and there is not a single example of using Logica to compute something recursively. Transitive closure is the canonical example for showing off Datalog.
- molham 6y agoSQL 99 only requires support for linear recursion, i.e. "each FROM has at most one reference to a recursively-defined relation". Datalog doesn't have such a restriction
- zoshi 6y agoA major difference between Datalog and SQL is that Datalog uses set semantics, whereas SQL uses bag semantics. For those not aware, that means facts in datalog are unique. SQL’s equivalent to facts (records) are not unique, and relations can contain duplicates.
- bob1029 6y ago> The main flaw of SQL, however, lies in its very limited support for abstraction. This is objectively bullshit. What SQL database engine doesn't provide views, stored procedures and/or user-defined function support? Failure to construct higher-order abstractions in SQL is a failure of the engineer to understand the problem domain, not a failure of the tool. SQL is capable of operating at any level of abstraction you wish for it to. It is all engineering from there.
- radiowave 6y agoNone of those things (views, stored procs, functions) are part of the query language. They all involve making somewhat permanent alterations to the structure of the database, and are entirely capable of conflicting with other objects in the database. So if your query needs these things, you're now into "release management" territory. In terms of lack of linguistic abstraction, try writing a query that abstracts over what table it's to be run against, allowing choice of source data to be parameterized, without either resorting to shenanigans like passing strings of SQL fragments around to be eval'd during query execution, or writing the query in some other system such that it can then be compiled into SQL for execution.
- deleted 6y ago[deleted]
- jerf 6y agoTo give a concrete example of what we're talking about, I wrote this a couple of months ago in a similar discussion: https://news.ycombinator.com/item?id=26320573 https://news.ycombinator.com/item?id=26320573 The problem is that "Compose" function is essentially impossible to write. I say essentially because it's probably something that could theoretically be written, but not with any reasonable amount of effort. It would literally be on par with the difficulty of implementing a DB from scratch that implements the Logica language as its base query language.
- PaulHoule 6y agoIt astonishes me that it took so long for Datalog to be legitimized as a query language. It's almost as if people saw OWL 2 DL, didn't believe what it had accomplished, and didn't try to make anything better.
- JulianMorrison 6y agoSQL queries aren't just esoteric, they have highly opaque performance implications. Two ways of doing an SQL query that might look mathematically equivalent to a human could result in orders of magnitude speed difference due to one of them using the proper index and the other having to do a sequential scan, or various other performance issues like creating temporary data. So this is going to run into the same issues as any SQL code generator (compare Hibernate for example): you need to know what query it will output. And you need DBA skills to know what that query means in terms of performance. Neither of those steps can be skipped. Nor is unit testing necessarily helpful when using small n. Issues of poor scaling don't show in tests unless the data is large.
- HelloNurse 6y agoCompared to Prolog, which is sensitive to declaration/search order and can assert enough new facts to not terminate, the risk of inefficient but always correct queries is a marked improvement.
- theon144 6y agoThis is just pure speculation on my part, but: What about optimizations? It seems like it should be possible to construct a SQL query that doesn't hit the pain points (e.g. avoids queries that do not use indexes). Although from my experiences with other ORM frameworks, that probably isn't an easy problem. Even then though, since it looks like it somewhat aims to replace SQL even in the database-construction step, that might help in this regard, by constructing a more optimal representation of the data (which doesn't seem to be tabular)? Unit testing I am similarly skeptical about though. The article does mention it being "rather esoteric [sounding] at best", I would actually agree with that expression, haha. I don't think I've ever written or even seen, in my 8 years as a developer, a 100-line SQL query that was not at least partly generated (and hence required testing as a unit, and not just the code around it). I suppose Google operates at a different scale, but still.
- timmg 6y agoIn the examples I saw, I never saw one that did a "join". I guess I'm surprised that they positioned this as a replacement for SQL but gave examples like detecting if something is prime. Might be useful, but I didn't get a good sense for it by reading the article or nearby docs.
- hansvm 6y agoJoins show up in the tutorial.
- dan-robertson 6y agoThe linked page has an example of a join between the `comments` table and the MagicNumber relation. A join of A and B looks like C(x,z) :- A(x,y), B(y,z)
- samuell 6y agoFantastic to see a logic programming/datalog/prolog-like from a big actor like Google. Perhaps this can make more such tools become mainstream. Only I wish we had such a language for a more generic streaming / data processing framework, such as materialize [1]. I was very optimistic about that for some time, as the guy behind the technology, Frank McSherry, wrote some datalog tooling as well [2]. [1] https://materialize.com/ https://materialize.com/ [2] https://github.com/vmware/differential-datalog https://github.com/vmware/differential-datalog
- hardwaregeek 6y agoMaybe I'm missing something, but why don't query languages allow for sum types and pattern matching? Why is it so hard to express in SQL a table that can contain either this schema or that schema?
- ben509 6y agoThe relational model describes a relational algebra, which in turn specifies a "relation" (often called a table) to be a set of tuples (rows), and "relational operators" that accept and return relations. And the relational algebra is complete in the sense that it can express all queries expressible by predicate logic + types and changes through a small set of operations. The relational model adds constraints, state, and a mechanism for first-class derived relations (updateable views). And while you can stick anything with a well-defined equality in a relation, including other relations, the point of the relational algebra is to describe structure using relations, thus making it all accessible to relational operators. In a properly normalized database, all structure can be manipulated through a common set of operations. So you could, e.g. create a relation with a single JSON attribute and call it a day. But now, in addition to the relational operators, you need a whole mess of JSON operators to query it. Thus, while you could have a sum type, you don't need this because you can put the various summands into separate relations. For instance, the simple case of booleans: Persons(key id: int, name: str, is_tall: bool) ... noramlizes to ... Persons(key id: int, name: str) TallPersons(key id: int) Or for an Either: Persons(key id: int, name: str, zing: either<int, str>) ... noramlizes to ... Persons(key id: int, name: str) LeftPersons(key id: int, zing: int) RightPersons(key id: int, zing: str) AssertEmpty: LeftPersons{key} & RightPersons{key} What you really want your database to do is to let you enter that first "Persons" table with the sum type. That should logically be a derived table that is backed by the normalized tables. Then, you'd get the simplicity of entering Persons.insert(key=5, name='bob', zing=Left(5)), but that's simply an updateable view. It will really update the base tables Persons / LeftPersons with simple atomic values.
- hardwaregeek 6y agoThat's a very good explanation, thanks. However I'd argue that just because you can express something in a language, doesn't mean it's the optimal way to express it. Any turing complete language can express sum types, but there's something to be said about having explicit features within the language for sum types. The SQL vs NoSQL debate reminds me of the static types vs dynamic types debate. Static type systems offer a more robust mental model and verification, but at the cost of rigidity and forcing the programmer to think in the type system's mental model. Dynamic types are therefore appealing because they can adapt to the programmer's mental model, even if that model is seriously flawed. One change in philosophy that has helped static type system is the realization that you can design static type systems that while not conceptually pure, get closer to the dynamic mental models. These systems, such as TypeScript or Go, make static types easier to swallow. I wonder if the same could be said for SQL/relational systems. If you could design a relational system that's a touch more ergonomic, that doesn't require learning a mental model that feels a little foreign, maybe the NoSQL options will be a lot less appealing.
- aseerdbnarng 6y agoI am trying and find concrete examples of where this is superior to mainstream SQL. Right now the documentation and examples are targeted at higher-level users than myself and it is not clear (to me that is) what it is trying to solve. Anyone have an ELI5-type link?
- usgroup 6y agoLogica compiles to SQL. It is semantically equivalent but not syntactically so the gain has to be in differences in the grammar. I see that one can create predicates (functions) with parameters as a means for code re-use. I could also see that having implications for testability. That's interesting. I could see how one could build a DSL with Logica to make fairly tricky queries easier. That's interesting. Has anyone used this? If so could you explain how are SQL functions called? Do they have to specifically exist in Logica or are they just assumed to exist in SQL? (I'm thinking about geographic functions in particular for example. Are window functions also possible in Logica?
- Syzygies 6y agoThe C-like syntax is a bit depressing. There is a syntax debate I respect. While I prefer austere syntax, deeper thinkers like Bill Joy note that programmer productivity increases with more information on screen at once. Syntax that improves both code density and clarity is a good thing. I love Haskell and Ruby in actual use, even if I want to prefer Lisp without parentheses (an easy preprocessor if one thinks it through). I cannot respect perpetuating C syntax just to attract users who would otherwise be challenged (that Apollo 13 astronaut who "never trained in the LEM"). Rob Pike once gave the only justification I can understand: Code used to need to survive communicating through channels that mangled whitespace. That is no longer the case, and modern editors all support syntax highlighting. (We've reached the point where one should develop an editor language server in parallel with any new language.) If your editor can figure out your language's grammar, and then you can with the editor's help, then one achieves greater code density and clarity at once. Some people do love terminals, but most people use graphical user interfaces. Why is language design stuck in terminal pre-history? There is no excuse in 2021 for lots of stray punctuation that's just ground glass in programmers' eyes.
- gnulinux 6y ago> There is a syntax debate I respect. While I prefer austere syntax, deeper thinkers like Bill Joy note that programmer productivity increases with more information on screen at once. Syntax that improves both code density and clarity is a good thing. I love Haskell and Ruby in actual use, even if I want to prefer Lisp without parentheses (an easy preprocessor if one thinks it through). I strongly prefer verbose type systems, in particular, some lightweight type inference is good as long as it's not full fledged HM type inference (like Haskell, Rust etc). The problem with HM type inference is although it's extremely powerful and makes the code look cleaner, it hides important data from programmer, which ultimately causes 2 bugs: * variables being inferred to have types slightly different than ehat programmer expects. E.g. I expected foo to be A(B(C)) -> D(C) turns out it's actually A(B(X)) -> D(C) which also type checks. * Errors can be harder to read.
- Syzygies 6y agoYes. My Haskell code includes all signatures, even for functions defined inside functions.
- opk 6y agoI'm surprised there's no trademark issues with the name. I worked for Logica, the company for many years. At some point later on they got bought by CGI but you'd have thought they still held the rights to the name. CGI is a stupid name - a three-letter acronym that conflicts with more than one unrelated IT term. Seems logica.com still redirects.
- molham 6y agoI don't believe you can trademark the name of a computer language