4 ms·

Embedded KV stores are as comparable to SQLite as Cassandra is to Postgres (i.e. lacking joins or other data modelling essentials), but I agree that embedding a

by refset 6y ago

Embedded KV stores are as comparable to SQLite as Cassandra is to Postgres (i.e. lacking joins or other data modelling essentials), but I agree that embedding a database directly inside your app is interesting and opens up a world of possibilities for efficiently blending N+1 code and queries. It gets even more interesting to take it a step further and embed a networked-replica of a database à la Datomic Peers.

dfox 6y ago
It was probably meant as a learning exercise. And for me it is that implementation details of the KV/heap layer with some meaningful transactional properties seem more interesting than how to build SQL/whatever database on top of that, which in comparison looks like mostly obvious but tedious SMOP.
- refset 6y ago
  True, I was being somewhat pedantic. Whenever I see the word "db" I can't help but think "DBMS". It's frustrating that "DBMS" is such a mouthful that nobody bothers using it anymore, and that KV store projects choose to (ab)use the DB suffix. > implementation details of the KV/heap layer with some meaningful transactional properties seem more interesting There is undoubtedly a real beauty in LMDB's data structures, and it is impressive to see how Rockset have re-engineered RocksDB to become cloud-native …but my own feeling is that the SQL/whatever, distributed consistency and federation layers of a DBMS encompass some seriously hard and fascinating problems. What if the web could behave like a cohesive offline-first database for all of human knowledge after all? Now I think about it, "interesting" has a pretty loose definition too!
jerzyt 6y ago
I haven't used SQLite until a few weeks ago because I've always had an Oracle or Postgres running and it was an easy choice. Recently, I needed a self contained solution for a Python project, and I wanted the SQL support with minimum installation and system dependencies. I tried SQLite - I'm absolutely amazed how well it worked for the task. I don't know how SQLite scales for large data sets, but for a small tasks it's fantastic, and can be trivially ported to something like Postgres.

by refset 6y ago

dfox 6y ago

It was probably meant as a learning exercise. And for me it is that implementation details of the KV/heap layer with some meaningful transactional properties seem more interesting than how to build SQL/whatever database on top of that, which in comparison looks like mostly obvious but tedious SMOP.

refset 6y ago
True, I was being somewhat pedantic. Whenever I see the word "db" I can't help but think "DBMS". It's frustrating that "DBMS" is such a mouthful that nobody bothers using it anymore, and that KV store projects choose to (ab)use the DB suffix. > implementation details of the KV/heap layer with some meaningful transactional properties seem more interesting There is undoubtedly a real beauty in LMDB's data structures, and it is impressive to see how Rockset have re-engineered RocksDB to become cloud-native …but my own feeling is that the SQL/whatever, distributed consistency and federation layers of a DBMS encompass some seriously hard and fascinating problems. What if the web could behave like a cohesive offline-first database for all of human knowledge after all? Now I think about it, "interesting" has a pretty loose definition too!

jerzyt 6y ago

I haven't used SQLite until a few weeks ago because I've always had an Oracle or Postgres running and it was an easy choice. Recently, I needed a self contained solution for a Python project, and I wanted the SQL support with minimum installation and system dependencies. I tried SQLite - I'm absolutely amazed how well it worked for the task. I don't know how SQLite scales for large data sets, but for a small tasks it's fantastic, and can be trivially ported to something like Postgres.