5 ms·

Shameless plug of my mvSQLite [1] project here! It's basically another distributed SQLite (that is API-compatible with libsqlite3), but with support for everyth

Shameless plug of my mvSQLite [1] project here! It's basically another distributed SQLite (that is API-compatible with libsqlite3), but with support for everything expected from a proper distributed database: synchronous replication, strictly serializable transactions, + scalable reads and writes w/ multiple concurrent writers. [1] https://github.com/losfair/mvsqlite https://github.com/losfair/mvsqlite

ok_dad 4y ago
> Distributed, MVCC SQLite that runs on top of FoundationDB. FYI to anyone here, FoundationDB is fucking awesome for something like this. Question @losfair: Did you find the Rust bindings for FDB to be very good? The Go bindings are OK, but are pretty out-of-date with some cool new features on the HEAD of the FDB source repo.
- losfair 4y ago
  I found foundationdb-rs pretty good & robust. Haven't tried any FDB 7.x features (tenants, etc.) though.
endisneigh 4y ago
mvSQLite looks great, though I’m curious how you’d implement a schema migration given the locking properties of SQLite combined with the transaction limits of fdb. I imagine you’d get an FDB transaction time limit error preventing any schema migrations with non trivial amounts of data.
- losfair 4y ago
  We found a way around FDB transaction time limit. A mvSQLite transaction is not directly mapped to a FDB transaction (details in readme and wiki)
  - endisneigh 4y ago
    I’ve read it - there’s still a time limit and so long schema migrations still would be an issue. Even without FoundationDB long schema migrations are problems.
    - losfair 4y ago
      Online DDL is now a WIP feature. This will allow to convert DB into read-only mode & run arbitrarily large schema migration concurrently (first stage), and eventually fully concurrent DDL by replaying logs.
      - endisneigh 4y ago
        That sounds very promising. Excellent work with the project!
darthShadow 4y ago
I really wanted to give this a try but the lack of WAL support has prevented me from using it. With the recent addition of WAL support[1] in litefs, would it be possible to add the same to mvsqlite too? [1] https://github.com/superfly/litefs/pull/120 https://github.com/superfly/litefs/pull/120
- losfair 4y ago
  What are you trying to achieve with WAL mode? Is it some kind of application compatibility issue? The entire SQLite journaling mechanism is not used by mvSQLite (you can set journal_mode=off safely - although SQLite won't be happy to do explicit rollback in this case)
  - darthShadow 4y ago
    Yeah, I am trying to plugin mvsqlite into a binary-only app which is using WAL mode. Changing to any other journaling mode, just causes the app to not start at all.

ok_dad 4y ago

> Distributed, MVCC SQLite that runs on top of FoundationDB. FYI to anyone here, FoundationDB is fucking awesome for something like this. Question @losfair: Did you find the Rust bindings for FDB to be very good? The Go bindings are OK, but are pretty out-of-date with some cool new features on the HEAD of the FDB source repo.

losfair 4y ago
I found foundationdb-rs pretty good & robust. Haven't tried any FDB 7.x features (tenants, etc.) though.

endisneigh 4y ago

mvSQLite looks great, though I’m curious how you’d implement a schema migration given the locking properties of SQLite combined with the transaction limits of fdb. I imagine you’d get an FDB transaction time limit error preventing any schema migrations with non trivial amounts of data.

losfair 4y ago
We found a way around FDB transaction time limit. A mvSQLite transaction is not directly mapped to a FDB transaction (details in readme and wiki)
- endisneigh 4y ago
  I’ve read it - there’s still a time limit and so long schema migrations still would be an issue. Even without FoundationDB long schema migrations are problems.
  - losfair 4y ago
    Online DDL is now a WIP feature. This will allow to convert DB into read-only mode & run arbitrarily large schema migration concurrently (first stage), and eventually fully concurrent DDL by replaying logs.
    - endisneigh 4y ago
      That sounds very promising. Excellent work with the project!

darthShadow 4y ago

I really wanted to give this a try but the lack of WAL support has prevented me from using it. With the recent addition of WAL support[1] in litefs, would it be possible to add the same to mvsqlite too? [1] https://github.com/superfly/litefs/pull/120 https://github.com/superfly/litefs/pull/120

losfair 4y ago
What are you trying to achieve with WAL mode? Is it some kind of application compatibility issue? The entire SQLite journaling mechanism is not used by mvSQLite (you can set journal_mode=off safely - although SQLite won't be happy to do explicit rollback in this case)
- darthShadow 4y ago
  Yeah, I am trying to plugin mvsqlite into a binary-only app which is using WAL mode. Changing to any other journaling mode, just causes the app to not start at all.