4 ms·

SQLite version 1 was implemented on top of a gdbm backend. They ran SQL queries on top of a key-value store. I wonder if Tokyo Tyrant, which has the goal of b

by lsb 17y ago

SQLite version 1 was implemented on top of a gdbm backend. They ran SQL queries on top of a key-value store. I wonder if Tokyo Tyrant, which has the goal of being a modern implementation of dbm, would make a good backend.

strlen 17y ago
It would be interesting to see a SQL parser (a front-end, basically) on top of a distributed, HA key-value store. It could even be a pluggable MySQL engine (there already is a Memcached engine). Of course transactions and certain types of constraints would become impossible (as key-value stores do not provide for atomicity), but that could be an acceptable trade off.
- dasil003 17y ago
  If you're going to use SQL why would you use such a limited backend with a parity mismatch and a much greater set of limitations in place of many heavily-optimized-over-decades backends?
  - strlen 17y ago
    To gain the scalability/performance of a k/v store while having the capability to do complex queries (e.g. joins, range queries)
    - dasil003 17y ago
      I believe you would be much better off with a DSL that mapped to a set of fast operations on a k/v store. SQL databases are heavily heavily optimized, and far more performant than k/v stores for most non-trivial queries. k/v stores don't somehow give magical performance. What they do is limit you to a small set of primitives that helps with scalability and maps more naturally for applications that are document-oriented and don't slice and dice data as much.
- lsb 17y ago
  K/V stores don't provide ACID properties themselves, but you could bolt them on pretty easily. For instance: Atomicity: save a journal of the pre and post values of any rows, and on commit take all of your staged changes and write them and then delete the journal. If you're ever going to hit the DB again and you see a journal file that wasn't cleaned up, roll back to whatever pre values there are. Isolation: use a semaphore to cause the start of a writing transaction to block all reads and writes, use a semaphore to cause the start of a reading transaction to block all writes, and escalate from reading transaction to writing transaction when necessary.
  - silentbicycle 17y ago
    Transactions are actually incredibly tricky to get right, particularly in a portable manner. Not exactly the sort of thing one can implement in a weekend. For example: http://sqlite.org/atomiccommit.html http://sqlite.org/atomiccommit.html Also, it will slow down the database a bit (particularly if added to the db as an afterthought!), which seems to really irritate people who would rather have potentially corrupt information as soon as possible rather than automatic guarantees of internal consistency.
    - nettdata 17y ago
      And let's not even start talking about consistent reads.

by lsb 17y ago

strlen 17y ago

It would be interesting to see a SQL parser (a front-end, basically) on top of a distributed, HA key-value store. It could even be a pluggable MySQL engine (there already is a Memcached engine). Of course transactions and certain types of constraints would become impossible (as key-value stores do not provide for atomicity), but that could be an acceptable trade off.

dasil003 17y ago
If you're going to use SQL why would you use such a limited backend with a parity mismatch and a much greater set of limitations in place of many heavily-optimized-over-decades backends?
- strlen 17y ago
  To gain the scalability/performance of a k/v store while having the capability to do complex queries (e.g. joins, range queries)
  - dasil003 17y ago
    I believe you would be much better off with a DSL that mapped to a set of fast operations on a k/v store. SQL databases are heavily heavily optimized, and far more performant than k/v stores for most non-trivial queries. k/v stores don't somehow give magical performance. What they do is limit you to a small set of primitives that helps with scalability and maps more naturally for applications that are document-oriented and don't slice and dice data as much.
lsb 17y ago
K/V stores don't provide ACID properties themselves, but you could bolt them on pretty easily. For instance: Atomicity: save a journal of the pre and post values of any rows, and on commit take all of your staged changes and write them and then delete the journal. If you're ever going to hit the DB again and you see a journal file that wasn't cleaned up, roll back to whatever pre values there are. Isolation: use a semaphore to cause the start of a writing transaction to block all reads and writes, use a semaphore to cause the start of a reading transaction to block all writes, and escalate from reading transaction to writing transaction when necessary.
- silentbicycle 17y ago
  Transactions are actually incredibly tricky to get right, particularly in a portable manner. Not exactly the sort of thing one can implement in a weekend. For example: http://sqlite.org/atomiccommit.html http://sqlite.org/atomiccommit.html Also, it will slow down the database a bit (particularly if added to the db as an afterthought!), which seems to really irritate people who would rather have potentially corrupt information as soon as possible rather than automatic guarantees of internal consistency.
  - nettdata 17y ago
    And let's not even start talking about consistent reads.