free-ekanayaka

6 ms·

1.
by free-ekanayaka 7y ago
It's not a buzz term. It's really fully async disk I/O. Dqlite does not use SQLite's stock vfs implementation for writing to disk, as it's an entirely different model (based on raft).
2.
by free-ekanayaka 7y ago
Everything you said is very accurate (dqlite author here).
3.
by free-ekanayaka 7y ago
Give us time, we'll get there :)
4.
by free-ekanayaka 7y ago
Depends on how fast is your disk, what file system and kernel you use, and how low is your network latency. Difficult to predict. But it's basically as fast as it can get given 1) hardware constraints 2) raft consensus. If you want lig
5.
by free-ekanayaka 7y ago
Not that I know, but it's on my todo list (dqlite author here).
6.
by free-ekanayaka 7y ago
The support of async disk I/O in Linux differs depending on kernel version and file system type. But it is possible to get 100% async I/O with the is_submit(), and dqlite will leverage that if detected. There is now a new async I&
7.
by free-ekanayaka 7y ago
https://github.com/canonical/dqlite/blob/master/doc/faq.md#w...
8.
by free-ekanayaka 7y ago
It has been around for 2 years and half, but we released v1.0.0 just yesterday. I added answers to your other questions to the FAQ: https://github.com/canonical/dqlite/blob/master/doc/faq.md#w... h
9.
by free-ekanayaka 7y ago
Oh I had no idea somebody was using it in the wild! It has been unstable until now, we just released v1.0.0 yesterday. So no more public API breakage from now on.
10.
by free-ekanayaka 9y ago
Yes, dqlite is currently used for clustering support in LXD 3.0: https://github.com/lxc/lxd
11.
by free-ekanayaka 9y ago
If you read the raft paper, or just watch presentations of it, you'll see that all the issues you mentioned are covered: - faulty node replacement: you can take any node off (or any node can crash) at any time. As long as there are eno
12.
by free-ekanayaka 9y ago
"slower" means this: when you perform a SQLite write, SQLite writes a new page on disk to the its write-ahead log. dqlite needs to replicate that page write across all nodes. A page is typically 4kb, so when you do a write on the
13.
by free-ekanayaka 9y ago
Unit and fuzzy-based test coverage that includes all possible edge cases is being put in place. I'd like to write a Jepsen test suite too. Hardware does not matter that much, since SQLite is pretty hardened for handling any possible ha
14.
by free-ekanayaka 9y ago
If you require SQL, sharding and concurrent writes, yes do use CockroachDB or other equivalent solution. If you don't have those requirements, dqlite or rqlite might get the job done with less moving parts and operational overhead. It&
15.
by free-ekanayaka 9y ago
I'm working towards a 1.0 release. At that point documentation will be more complete and the plumbing needed to use it in an app should be also reduced. I think the best explanation of why you'd use it over PG or MySQL is the same
16.
by free-ekanayaka 9y ago
BedrockDB requires to operate a separate process, whereas you can embed dqlite in your Go application (pretty much in the SQLite philosophy). Also, afaik BedrockDB patches upstream SQLite with some more intrusive changes than dqlite (e.g. f
17.
by free-ekanayaka 9y ago
If your application is: 1) Distributed (e.g. n equal processes running on n machines) 2) Needs some shared state across nodes 3) Would like that state to have SQL semantics (relations, transactions, etc.) 4) Not to heavy on writes to this s
18.
by free-ekanayaka 9y ago
I implemented dqlite in Go for convenience (a production-ready raft library was available, github.com/hashicorp/raft). It'd be interesting to implement something similar in C or event batter in rust, so you can link the libra
19.
by free-ekanayaka 9y ago
Hello, author of dqlite here. Things are still in flux, but yes, I plan to publish benchmarks before making a 1.0 release, as well as improving documentations and introduce some more abstractions to make it easier to use. The reason it migh