6 ms·

What is a graph database? A miserable little pile of joins. Though to be serious: what do you expect a graph database to provide that sqlite cannot / does not

by Groxx 5y ago

What is a graph database? A miserable little pile of joins. Though to be serious: what do you expect a graph database to provide that sqlite cannot / does not do efficiently?

patates 5y ago
Treating links as an entity. In a RDBMS, it is not possible to map a bidirectional relation semantically. It needs to have a direction (you can define it in both directions but then you have 2 relations). You then also need to duplicate link properties (in the join table) or normalize to yet another join table. That has been my pet peeve for a while, and can make it hard to define navigation. Totally not a deal-breaker though. I'd still use sqlite because I ### love it :)
- Groxx 5y ago
  why would a direction be necessary? relation_id │ relatee_id ────────────┼─────────── 1 │ x 1 │ y voila, directionless relationships. you can even do N-way, just insert more rows with the same relation-id.
  - patates 5y ago
    To quote myself: > or normalize to yet another join table This is also semantically incorrect. There needs to be exactly 2 rows there. You cannot in a declarative way, prevent that if you use this approach. It's also a known dilemma, so we don't have to rediscover the wheel here.
    - noctune 5y ago
      What if you have something like (relatee_a_id, relatee_b_id, [rest of relation fields ...]) with the constraint that relatee_a_id < relatee_b_id? Granted, it might not be very nice to query.
      - patates 5y ago
        Well, that is what I actually use most of the time, but to nitpick, it still is not declarative, and semantically a and b have no meaning. Just to be clear, I'm saying these are in 99% of the cases not deal breakers but they can be if all you want to do is map graphs (and follow chains through many members), as there is also a significant performance cost to all this abstraction. What you described is, IIRC, also what some file-based graph DBs do in the background with sqlite and abstract away from you (I've seen a couple custom ones in some projects). Left with a RDBMS, you just need to do it yourself and analyze how much it costs.
    - Groxx 5y ago
      Custom indexes or insert triggers can both guarantee that, both in a declarative way (in that they prevent normal "blind" inserts. I'm not entirely sure what you mean by declarative here tbh). Also I have no idea why a separate table is a negative / constraint of some kind. It's a natural way to model it.
      - patates 5y ago
        > no idea why a separate table is a negative When you are following a chain of relations (a vector), you end up losing a lot of performance. > not entirely sure what you mean by declarative Guaranteed by the database semantics, not imperative code even when it runs on the DB. I don't want to come off sounding like I think it's a bad idea to map graphs on RDBMSs, it's just inconvenient sometimes. Usually not enough to complicate your setup, adding another database, but rarely it is. I'm not sure though about the suggested use case here, I really don't know it enough to make a suggestion. In such cases I always pick sqlite or postgres (depending on the client model) though. You can't be too wrong with those, and you are most of the times right :) Not to mention the crazy extensibility of pg and the stuff you can do with it.
        Groxx 5y ago
        Imperative code run on the DB (i.e. triggers) seems entirely in-scope. It's an incredibly powerful tool provided natively by the DB, intended to be used for exactly this kind of custom constraint. Whether it's begin/end with imperative code or modeled some other way declarative seems absolutely, 100% irrelevant to me - it's the perfect implementation detail, it could have been done with `create unique index max 2 (x, y) on mytable` and you'd never know. If you meant to prevent "all inserts are now [run this stored procedure / multi-statement operation / transaction with N steps]", then yeah - I agree that runs counter to "normal" use in most cases. It's certainly dramatically more error prone. Triggers don't need this though. re chain of relations: I haven't poked too hard in this direction in SQLite, but I do totally believe its query planner isn't too sophisticated. And though you can get pretty good data locality with fancy design around the DB, you certainly won't get good locality just by inserting a bunch of data and running queries.
    - harperlee 5y ago
      A directionless link or relation does not require exactly 2 rows here, that’s overconstraining the definition. A simple example would be to link 3 brothers with the same relation: the worse solution would be dividing it in 3 different binary relations; the reality is that each brother is part of the “brother relation” set by way of sharing parents.
      - galaxyLogic 5y ago
        Brothers yes. But what about children? child_of is a directional relation, right? Semantic web I assume is full of such directional relations. Bidirecitonal relations are the exception. Things like "causes", "indicates", "enables", "prevents", ...
- deleted 5y ago
  [deleted]
dng88 5y ago
wheels 5y ago
You can reasonably model graphs in a column-oriented database, but traditional SQL models tend to be horrible performance-wise, because most graph algorithms need fast traversal of edges, and doing a lot of recursive lookups in SQL is impractically slow. It's not that you can't model it, it's just that performance is terrible. For a graph database to be efficient, you need a high degree of locality for edge information (ideally a vector you can simply read out). (Note: I've actually written a graph database from scratch, for exactly these reasons.)
- ahevia 5y ago
  Do you have examples of Graph DBs that could embedded in a static site as easily as SQLite?
- oever 5y ago
  Modelling transitive edges can be done in Sqlite with recursive common table expressions. The performance for that is probably less than for graphs databases, but Sqlite has many other advantages over graph databases.
  - wheels 5y ago
    Uhm, that's basically what I said? Aside from the "other advantages" -- which, yeah, sure. Socket sets have other advantages than hammers, but they're not very good at pounding in nails. You can do it, but it'll suck at it. Also, in this case, "probably less" is "multiple orders of magnitude slower".
    - oever 5y ago
      The locality that you want for performance is available as row values in sqlite. Sqlite seems to have the parts needed to build a graph db.
      - deleted 5y ago
        [deleted]
      - 5y ago
j-pb 5y ago
A whole pile of joins, that naturally arises if you want to combine data from a ton of domains. Triple stores are essentially relational databases in 6th normal form. But relational databases like SQLite don't have good join algorithms to deal with this (they do pairwise joins instead of the worst case optimal ones like Leapfrog-Triejoin or Tetris). They also lack good interfaces for so many joins, you want something more declarative like Datalog/SparQL/GraphQL, than to explicitly write out every join.

by Groxx 5y ago

What is a graph database? A miserable little pile of joins. Though to be serious: what do you expect a graph database to provide that sqlite cannot / does not do efficiently?

patates 5y ago

Treating links as an entity. In a RDBMS, it is not possible to map a bidirectional relation semantically. It needs to have a direction (you can define it in both directions but then you have 2 relations). You then also need to duplicate link properties (in the join table) or normalize to yet another join table. That has been my pet peeve for a while, and can make it hard to define navigation. Totally not a deal-breaker though. I'd still use sqlite because I ### love it :)

Groxx 5y ago
why would a direction be necessary? relation_id │ relatee_id ────────────┼─────────── 1 │ x 1 │ y voila, directionless relationships. you can even do N-way, just insert more rows with the same relation-id.
- patates 5y ago
  To quote myself: > or normalize to yet another join table This is also semantically incorrect. There needs to be exactly 2 rows there. You cannot in a declarative way, prevent that if you use this approach. It's also a known dilemma, so we don't have to rediscover the wheel here.
  - noctune 5y ago
    What if you have something like (relatee_a_id, relatee_b_id, [rest of relation fields ...]) with the constraint that relatee_a_id < relatee_b_id? Granted, it might not be very nice to query.
    - patates 5y ago
      Well, that is what I actually use most of the time, but to nitpick, it still is not declarative, and semantically a and b have no meaning. Just to be clear, I'm saying these are in 99% of the cases not deal breakers but they can be if all you want to do is map graphs (and follow chains through many members), as there is also a significant performance cost to all this abstraction. What you described is, IIRC, also what some file-based graph DBs do in the background with sqlite and abstract away from you (I've seen a couple custom ones in some projects). Left with a RDBMS, you just need to do it yourself and analyze how much it costs.
  - Groxx 5y ago
    Custom indexes or insert triggers can both guarantee that, both in a declarative way (in that they prevent normal "blind" inserts. I'm not entirely sure what you mean by declarative here tbh). Also I have no idea why a separate table is a negative / constraint of some kind. It's a natural way to model it.
    - patates 5y ago
      > no idea why a separate table is a negative When you are following a chain of relations (a vector), you end up losing a lot of performance. > not entirely sure what you mean by declarative Guaranteed by the database semantics, not imperative code even when it runs on the DB. I don't want to come off sounding like I think it's a bad idea to map graphs on RDBMSs, it's just inconvenient sometimes. Usually not enough to complicate your setup, adding another database, but rarely it is. I'm not sure though about the suggested use case here, I really don't know it enough to make a suggestion. In such cases I always pick sqlite or postgres (depending on the client model) though. You can't be too wrong with those, and you are most of the times right :) Not to mention the crazy extensibility of pg and the stuff you can do with it.
      - Groxx 5y ago
        Imperative code run on the DB (i.e. triggers) seems entirely in-scope. It's an incredibly powerful tool provided natively by the DB, intended to be used for exactly this kind of custom constraint. Whether it's begin/end with imperative code or modeled some other way declarative seems absolutely, 100% irrelevant to me - it's the perfect implementation detail, it could have been done with `create unique index max 2 (x, y) on mytable` and you'd never know. If you meant to prevent "all inserts are now [run this stored procedure / multi-statement operation / transaction with N steps]", then yeah - I agree that runs counter to "normal" use in most cases. It's certainly dramatically more error prone. Triggers don't need this though. re chain of relations: I haven't poked too hard in this direction in SQLite, but I do totally believe its query planner isn't too sophisticated. And though you can get pretty good data locality with fancy design around the DB, you certainly won't get good locality just by inserting a bunch of data and running queries.
  - harperlee 5y ago
    A directionless link or relation does not require exactly 2 rows here, that’s overconstraining the definition. A simple example would be to link 3 brothers with the same relation: the worse solution would be dividing it in 3 different binary relations; the reality is that each brother is part of the “brother relation” set by way of sharing parents.
    - galaxyLogic 5y ago
      Brothers yes. But what about children? child_of is a directional relation, right? Semantic web I assume is full of such directional relations. Bidirecitonal relations are the exception. Things like "causes", "indicates", "enables", "prevents", ...
deleted 5y ago
[deleted]

dng88 5y ago

wheels 5y ago

You can reasonably model graphs in a column-oriented database, but traditional SQL models tend to be horrible performance-wise, because most graph algorithms need fast traversal of edges, and doing a lot of recursive lookups in SQL is impractically slow. It's not that you can't model it, it's just that performance is terrible. For a graph database to be efficient, you need a high degree of locality for edge information (ideally a vector you can simply read out). (Note: I've actually written a graph database from scratch, for exactly these reasons.)

ahevia 5y ago
Do you have examples of Graph DBs that could embedded in a static site as easily as SQLite?
oever 5y ago
Modelling transitive edges can be done in Sqlite with recursive common table expressions. The performance for that is probably less than for graphs databases, but Sqlite has many other advantages over graph databases.
- wheels 5y ago
  Uhm, that's basically what I said? Aside from the "other advantages" -- which, yeah, sure. Socket sets have other advantages than hammers, but they're not very good at pounding in nails. You can do it, but it'll suck at it. Also, in this case, "probably less" is "multiple orders of magnitude slower".
  - oever 5y ago
    The locality that you want for performance is available as row values in sqlite. Sqlite seems to have the parts needed to build a graph db.
    - deleted 5y ago
      [deleted]
    - 5y ago

j-pb 5y ago

A whole pile of joins, that naturally arises if you want to combine data from a ton of domains. Triple stores are essentially relational databases in 6th normal form. But relational databases like SQLite don't have good join algorithms to deal with this (they do pairwise joins instead of the worst case optimal ones like Leapfrog-Triejoin or Tetris). They also lack good interfaces for so many joins, you want something more declarative like Datalog/SparQL/GraphQL, than to explicitly write out every join.