4 ms·

anyone running a large scale saas job with rqlite/dqlite?

jansommer 4y ago
I think it's a deal breaker that rqlite only supports deterministic statements [0] and the way dqlite works is by having to run inside your application, meaning that you no longer can scale to zero. Ideally someone would implement replication as a separate process, like litestream but with better guarantees like rqlite/dqlite, but until that happens, I don't think they're of much use in a SaaS, unless it's to be an in process database alternative to Postgres and the likes, where users only access is through something like a rest api. [0] https://www.sqlite.org/deterministic.html https://www.sqlite.org/deterministic.html
- benbjohnson 4y ago
  Litestream author here. I'm actually working on a next iteration of Litestream that'll have some configurable guarantees (e.g. synchronous replication, async replication w/ a maximum time window). I could possibly add distributed consensus (e.g. Raft) but I'm not sure how much benefit that would have over rqlite/dqlite.
  - jansommer 4y ago
    Sounds super cool! Really appreciatw the work you're doing. What I'm imagining in regards to Raft is this: 5 or more servers, each running unikernels in tiny virtual machines, with a JavaScript interpreter or similar to run untrusted user code. Each unikernel is then booted on every request, perhaps with a cool down period before being shut down. This would work similar to GCP, Azure etc. and performance would be great because of how fast unikernels boot. The unikernels would also be compiled with SQLite, with each database per customer stored on the host, and with restrictions on the virtual machine to only allow access to the database associated with the caller. Whenever untrusted code is executed, it can insert/update/read/delete from the database on the host. If the host then handled replication, that means I can just shut down the vm when the code has been run, and start a clean one for the next customer. A guarantee that the data have been safely replicated would be essential here, since another server might handle the next request, with the expectation that the data is in a new state.
- otoolep 4y ago
  rqlite author here. I'm actually working to address that. But it does require parsing the SQLite statement before it's sent to the Raft log, and replacing the non-deterministic value with the value evaluated on the leader. It's all solvable (in principle).

I think it's a deal breaker that rqlite only supports deterministic statements [0] and the way dqlite works is by having to run inside your application, meaning that you no longer can scale to zero. Ideally someone would implement replication as a separate process, like litestream but with better guarantees like rqlite/dqlite, but until that happens, I don't think they're of much use in a SaaS, unless it's to be an in process database alternative to Postgres and the likes, where users only access is through something like a rest api. [0] https://www.sqlite.org/deterministic.html https://www.sqlite.org/deterministic.html

benbjohnson 4y ago
Litestream author here. I'm actually working on a next iteration of Litestream that'll have some configurable guarantees (e.g. synchronous replication, async replication w/ a maximum time window). I could possibly add distributed consensus (e.g. Raft) but I'm not sure how much benefit that would have over rqlite/dqlite.
- jansommer 4y ago
  Sounds super cool! Really appreciatw the work you're doing. What I'm imagining in regards to Raft is this: 5 or more servers, each running unikernels in tiny virtual machines, with a JavaScript interpreter or similar to run untrusted user code. Each unikernel is then booted on every request, perhaps with a cool down period before being shut down. This would work similar to GCP, Azure etc. and performance would be great because of how fast unikernels boot. The unikernels would also be compiled with SQLite, with each database per customer stored on the host, and with restrictions on the virtual machine to only allow access to the database associated with the caller. Whenever untrusted code is executed, it can insert/update/read/delete from the database on the host. If the host then handled replication, that means I can just shut down the vm when the code has been run, and start a clean one for the next customer. A guarantee that the data have been safely replicated would be essential here, since another server might handle the next request, with the expectation that the data is in a new state.
otoolep 4y ago
rqlite author here. I'm actually working to address that. But it does require parsing the SQLite statement before it's sent to the Raft log, and replacing the non-deterministic value with the value evaluated on the leader. It's all solvable (in principle).