3 ms·
With rqlite, the docs indicate that a single node setup should run ok. Is that supposed to still apply when a small cluster (eg three nodes) is reduced to a si
by justinclift 4y ago
With rqlite, the docs indicate that a single node setup should run ok.
Is that supposed to still apply when a small cluster (eg three nodes) is reduced to a single node? eg during node failures, maintenance, node failures during maintenance ;), etc.
When I was trying rqlite last week - eg prototyping a new SQLite "cloud" offering for DBHub.io - a remaining single node would consistently refuse to elect itself as the leader. Using default settings for everything.
Tried it out several times, though I didn't give it more than a minute or two to get itself happy.
Anyway, had to go with an initial non-RAFT approach for the prototype, but will likely investigate rqlite again in the future if I was just doing something dumb / fixable with tweaking. :)
- otoolep 4y agoA single node should work perfectly fine when it's the only node in the cluster i.e. you're running a single-node cluster. If you couldn't bring up a single node from scratch, you're doing something wrong. :-) Show me your launch command, or join the rqlite Slack channel for help (https://www.philipotoole.com/join-rqlite-slack https://www.philipotoole.com/join-rqlite-slack) However if you create a 3-node cluster (or add 2 nodes to an existing single-node cluster) then at least a quorum of nodes (N/2 +1) must be up and running for the cluster to be available. That's the way Raft works -- Cockroach, etcd, Consul -- all of them will work the same way. So in your case a 3-node cluster with 2 nodes subsequently going down (or deliberately taken offline) is a cluster which is offline. There is extensive documentation available on clustering, which you should review: https://rqlite.io/docs/clustering/ https://rqlite.io/docs/clustering/
- justinclift 4y agoOh, the "getting a single node cluster" up and running was easy, as you said. It's just it wasn't clear from the initial docs I saw that a 3 -> 1 cluster would be considered any differently. Clearly, an error on my part. :) I'll circle back to rqlite in a while, after I get this existing prototype code rolled out and see what people make of it. Hopefully it turns out to be useful for people.
- otoolep 4y agoYeah, going from 3->1 simply by taking down nodes won't work -- that's breaking quorum. If you really want to do this, you can explicitly override the configuration stored in Raft. Check out these directions: https://rqlite.io/docs/clustering/general-guidelines/#recovering-a-cluster-that-has-permanently-lost-quorum https://rqlite.io/docs/clustering/general-guidelines/#recove... While this talks about "dealing with failure" using the config file described above basically allows you to force the cluster into any configuration you want (including a single-node cluster).
- otoolep 4y agoI added a FAQ entry which explains a little bit more what you can do: https://rqlite.io/docs/faq/#how-can-i-change-my-multi-node-cluster-to-a-single-node-system https://rqlite.io/docs/faq/#how-can-i-change-my-multi-node-c...
- justinclift 4y agoThanks, that should help anyone else following the same thought pattern I had. :)