4 ms·

CQL: Categorical Databases

111 points by noworriesnate 5mo ago

flying_sheep 5mo ago
Thanks for the sharing. It looks interesting but I did not dive deep into it. Just wonder how is it different from SQL trigger which can also ensure integrities?
- js8 5mo ago
  It's not much really, CDBs are based on foreign key relationships as a fundamental building block, rather than on relation. The difference is more in theory than in practice.
srean 5mo ago
There was a good blog post on how the category theoretic ideas behind this applies to data frames What Category Theory Teaches Us About DataFrames https://mchav.github.io/what-category-theory-teaches-us-about-dataframes/ https://mchav.github.io/what-category-theory-teaches-us-abou... Discussed on HN at (67 comments) https://news.ycombinator.com/item?id=47561426 https://news.ycombinator.com/item?id=47561426
randomNumber7 5mo ago
Since Codd's paper showed that the relational model dominates other approaches (for data storage) I would expect a paper that shows categorical database are not affected by this and what benefit they have.
- js8 5mo ago
  My (amateur) take. CDB model (based on functions) has three advantages over RDB model (based on relations): 1. Easier modelling sum types (inheritance) due to duality. 2. Better handling of null due to labelled null. 3. Better foundation of elementary types (they're just another table ids). (Column stores often do that already, if your question is about storage.)
  - adrian_b 5mo ago
    While the relational model is claimed to be based on relations, the vast majority of the "relations" used in practice are functions, not general relations. A general relation exists only between the columns of a table that are included in a multi-column primary key. All columns that are not part of the primary key are functions of the primary key. Most tables used in practice use a single column as the primary key, which is frequently just a number or a UUID. Most databases contain only tables that are functions, without any table that contains general relations. The most frequently used kinds of joins are just function compositions.
    - thesz 5mo ago
      While it is true on the schema level, it is not true at the query level. The most frequently used kind of join is a general relation. [1] https://github.com/agirish/tpcds/blob/master/query1.sql https://github.com/agirish/tpcds/blob/master/query1.sql Query 1 from TPC-DS creates a multi-column relation by using GROUP BY. Which relation is then partially constrained by different columns from different tables.
bob1029 5mo ago
> CQL is not a database management system: it neither stores nor updates data. The same could be said for SQL. How does CQL differ from SQL? If I squint my eyes just a tiny amount, these ideas become really difficult to separate. I was always under the impression that the relational model is based upon many concepts studied in category theory. To my mind, all of the following things are overlapping parts of the exact same monster: Set theory Category theory Graph theory Type theory Discrete mathematics Relational algebra Relational calculus Relational modeling An actual sql schema
- khana 5mo ago
  [dead]
- js8 5mo ago
  > How does CQL differ from SQL? SQL is like Java, CQL is like Haskell. SQL has been around and used in production. CQL is a research language, possibly cleaner foundation but YMMV. The math fields you list are connected, but whether they are the same monster - again it's kinda like claiming all programming languages and implementations are the same (Turing-complete?) monster.
  - randomNumber7 5mo ago
    SQL is not an imperative programming language.
    - adrian_b 5mo ago
      DROP TABLE ? Most of SQL is not imperative, but it certainly also includes some imperative commands. Inserting a new row into an existing table is an imperative command, which may be the most frequently used of the SQL features, in certain applications concerned with recording transactions. Only the subset of SQL that is used for queries can be said to not be an imperative programming language.
      - da_chicken 5mo ago
mattsouth 5mo ago
Not to be confused with https://cql.hl7.org/ https://cql.hl7.org/ which is the CQL I know of.
- gengstrand 5mo ago
  For me, CQL will always mean Cassandra Query Language https://cassandra.apache.org/doc/4.0/cassandra/cql/ https://cassandra.apache.org/doc/4.0/cassandra/cql/
- gourabmi 5mo ago
  This was my first thought too!
deleted 5mo ago
[deleted]
mattstir 5mo ago
> Reduce risk of failure through artificial intelligence. CQL contains an embedded automated theorem prover that guarantees the correctness of CQL programs. Man, it's a rough environment right now marketing-wise. I don't know if they're contractually obligated to say the funny magic words, but the term AI is nearly entirely meaningless at this point. Akin to saying "behold my mighty calculator app: it prevents divisions by zero through artificial intelligence!"
- PaulHoule 5mo ago
  It's what we used to call AI back in the 1970s and 1980s which has advanced a whole lot with little awareness. That is, people thought 10,000 rules was a lot of rules in 1998 and now you can work with 10,000,000 rules. And theorem provers, SAT and SMT all got vastly better.
- ianhorn 5mo ago
  Huh, I read the pitch differently. As "reduce risk of (failure through artificial intelligence)," not as "(reduce risk of failure) through artificial intelligence." Maybe that's my bias since that's what I'm working on, but it's a big benefit to have stronger compiler guarantees of correctness so that an LLM can't screw things up as much. No BSing that it works when the compiler requires proof.
  - layer8 5mo ago
    I had to read it twice to come to that conclusion. Maybe the prima facie ambiguity is intentional.
iron_fever 5mo ago
I know very little about databases, but I work heavily with category theory, so fwiw: I think the main benefit is composition. The edge over SQL shows up when you combine schema mappings - a mapping is a functor, so when you migrate data along it the constraints come with it by construction, and you don't end up writing ETL and hoping integrity held. As best as I can tell (but i really dont know much about databases) it's probably a narrow advantage - storage and everyday queries still go to Codd's model - but for stitching schemas together it seems like it could work. I'm using similar math for automated formal verification, where this approach is what makes it tractable.
- solomonb 5mo ago
  I haven't used CQL but this is how the advantages have been described to me as well.
Syzygies 5mo ago
We have not yet converged on a best (or even adequate) way to present a structured mountain of information to AI, not already in its training corpus. AI agents fielded by major AI players still fail at the basic task of providing immediate and correct support for use of the current versions of their products. If a programming language is too new to have adequate representation in the training corpus, there isn't an accepted standard way to provide a reference manual targeting AI agents. Even the best way to include documentation in a large project so new AI agents can take over is controversial. A pile of linked markdown files really isn't an answer, less structured than a codebase itself, that AI is good at navigating. Other HN posts have discussed using SQL as a backbone for the AI "mind mapping" support we need for AI more critically than for ourselves. I was hoping that CQL could be an answer to this. Perhaps, but not its current primary goal.
joshsh 5mo ago
If this sort of thing is interesting to you -- using type theory and category theory to add compositionality to programming and databases -- also check out CQL's cousin, the graph programming language Hydra: https://github.com/CategoricalData/hydra https://github.com/CategoricalData/hydra.
- snthpy 5mo ago
  Interesting, thanks!
deterministic 5mo ago
> that has revolutionized several areas of computer science That is quite a claim. I will argue that Categorical Databases most definitely haven't. Any areas where it is true?

111 points by noworriesnate 5mo ago

Thanks for the sharing. It looks interesting but I did not dive deep into it. Just wonder how is it different from SQL trigger which can also ensure integrities?

js8 5mo ago
It's not much really, CDBs are based on foreign key relationships as a fundamental building block, rather than on relation. The difference is more in theory than in practice.

srean 5mo ago

There was a good blog post on how the category theoretic ideas behind this applies to data frames What Category Theory Teaches Us About DataFrames https://mchav.github.io/what-category-theory-teaches-us-about-dataframes/ https://mchav.github.io/what-category-theory-teaches-us-abou... Discussed on HN at (67 comments) https://news.ycombinator.com/item?id=47561426 https://news.ycombinator.com/item?id=47561426

randomNumber7 5mo ago

Since Codd's paper showed that the relational model dominates other approaches (for data storage) I would expect a paper that shows categorical database are not affected by this and what benefit they have.

js8 5mo ago
My (amateur) take. CDB model (based on functions) has three advantages over RDB model (based on relations): 1. Easier modelling sum types (inheritance) due to duality. 2. Better handling of null due to labelled null. 3. Better foundation of elementary types (they're just another table ids). (Column stores often do that already, if your question is about storage.)
- adrian_b 5mo ago
  While the relational model is claimed to be based on relations, the vast majority of the "relations" used in practice are functions, not general relations. A general relation exists only between the columns of a table that are included in a multi-column primary key. All columns that are not part of the primary key are functions of the primary key. Most tables used in practice use a single column as the primary key, which is frequently just a number or a UUID. Most databases contain only tables that are functions, without any table that contains general relations. The most frequently used kinds of joins are just function compositions.
  - thesz 5mo ago
    While it is true on the schema level, it is not true at the query level. The most frequently used kind of join is a general relation. [1] https://github.com/agirish/tpcds/blob/master/query1.sql https://github.com/agirish/tpcds/blob/master/query1.sql Query 1 from TPC-DS creates a multi-column relation by using GROUP BY. Which relation is then partially constrained by different columns from different tables.

bob1029 5mo ago

> CQL is not a database management system: it neither stores nor updates data. The same could be said for SQL. How does CQL differ from SQL? If I squint my eyes just a tiny amount, these ideas become really difficult to separate. I was always under the impression that the relational model is based upon many concepts studied in category theory. To my mind, all of the following things are overlapping parts of the exact same monster: Set theory Category theory Graph theory Type theory Discrete mathematics Relational algebra Relational calculus Relational modeling An actual sql schema

khana 5mo ago
[dead]
js8 5mo ago
> How does CQL differ from SQL? SQL is like Java, CQL is like Haskell. SQL has been around and used in production. CQL is a research language, possibly cleaner foundation but YMMV. The math fields you list are connected, but whether they are the same monster - again it's kinda like claiming all programming languages and implementations are the same (Turing-complete?) monster.
- randomNumber7 5mo ago
  SQL is not an imperative programming language.
  - adrian_b 5mo ago
    DROP TABLE ? Most of SQL is not imperative, but it certainly also includes some imperative commands. Inserting a new row into an existing table is an imperative command, which may be the most frequently used of the SQL features, in certain applications concerned with recording transactions. Only the subset of SQL that is used for queries can be said to not be an imperative programming language.
    - da_chicken 5mo ago

mattsouth 5mo ago

Not to be confused with https://cql.hl7.org/ https://cql.hl7.org/ which is the CQL I know of.

gengstrand 5mo ago
For me, CQL will always mean Cassandra Query Language https://cassandra.apache.org/doc/4.0/cassandra/cql/ https://cassandra.apache.org/doc/4.0/cassandra/cql/
gourabmi 5mo ago
This was my first thought too!

deleted 5mo ago

[deleted]

mattstir 5mo ago

> Reduce risk of failure through artificial intelligence. CQL contains an embedded automated theorem prover that guarantees the correctness of CQL programs. Man, it's a rough environment right now marketing-wise. I don't know if they're contractually obligated to say the funny magic words, but the term AI is nearly entirely meaningless at this point. Akin to saying "behold my mighty calculator app: it prevents divisions by zero through artificial intelligence!"

PaulHoule 5mo ago
It's what we used to call AI back in the 1970s and 1980s which has advanced a whole lot with little awareness. That is, people thought 10,000 rules was a lot of rules in 1998 and now you can work with 10,000,000 rules. And theorem provers, SAT and SMT all got vastly better.
ianhorn 5mo ago
Huh, I read the pitch differently. As "reduce risk of (failure through artificial intelligence)," not as "(reduce risk of failure) through artificial intelligence." Maybe that's my bias since that's what I'm working on, but it's a big benefit to have stronger compiler guarantees of correctness so that an LLM can't screw things up as much. No BSing that it works when the compiler requires proof.
- layer8 5mo ago
  I had to read it twice to come to that conclusion. Maybe the prima facie ambiguity is intentional.

iron_fever 5mo ago

I know very little about databases, but I work heavily with category theory, so fwiw: I think the main benefit is composition. The edge over SQL shows up when you combine schema mappings - a mapping is a functor, so when you migrate data along it the constraints come with it by construction, and you don't end up writing ETL and hoping integrity held. As best as I can tell (but i really dont know much about databases) it's probably a narrow advantage - storage and everyday queries still go to Codd's model - but for stitching schemas together it seems like it could work. I'm using similar math for automated formal verification, where this approach is what makes it tractable.

solomonb 5mo ago
I haven't used CQL but this is how the advantages have been described to me as well.

Syzygies 5mo ago

We have not yet converged on a best (or even adequate) way to present a structured mountain of information to AI, not already in its training corpus. AI agents fielded by major AI players still fail at the basic task of providing immediate and correct support for use of the current versions of their products. If a programming language is too new to have adequate representation in the training corpus, there isn't an accepted standard way to provide a reference manual targeting AI agents. Even the best way to include documentation in a large project so new AI agents can take over is controversial. A pile of linked markdown files really isn't an answer, less structured than a codebase itself, that AI is good at navigating. Other HN posts have discussed using SQL as a backbone for the AI "mind mapping" support we need for AI more critically than for ourselves. I was hoping that CQL could be an answer to this. Perhaps, but not its current primary goal.

joshsh 5mo ago

If this sort of thing is interesting to you -- using type theory and category theory to add compositionality to programming and databases -- also check out CQL's cousin, the graph programming language Hydra: https://github.com/CategoricalData/hydra https://github.com/CategoricalData/hydra.

snthpy 5mo ago
Interesting, thanks!

deterministic 5mo ago

> that has revolutionized several areas of computer science That is quite a claim. I will argue that Categorical Databases most definitely haven't. Any areas where it is true?