3 ms·
I'll try to reply to the different parts of your comment: 1) "Clearly afterthought". Sorry no actual argument. 2) Actively re-write the configuration: because
by antirez 12y ago
I'll try to reply to the different parts of your comment:
1) "Clearly afterthought". Sorry no actual argument.
2) Actively re-write the configuration: because of this (and the fact the configuration is fsynced) the basic Sentinel guarantees hold even in case the machine running the Sentinel processes crashes. Sentinel (and Redis) provide a full API for runtime reconfiguration, so I believe there are many ways to create automatic deployments. For people using puppet, it is possible to use the include files, it is not perfect but works most of the time. However this is a design decision, not a shortcoming IMHO.
3) Redis Sentinel has one of the faster failovers you can find around, I believe you tried a very old version. Example setting the down-after-milliseconds to 2000 (2 second timeout to consider the master failing, but note that this does not change anything, you can set it to 60 seconds, the point is, after the 60 seconds, how fast it is):
$ date; redis-cli -p 9000 debug segfault
Tue Sep 23 18:16:43 CEST 2014
Error: Server closed the connection
Slave logs: 2359:M 23 Sep 18:16:46.235 * MASTER MODE enabled (user request)
The slave was elected after 3 seconds, so the failover happened in 1 second.
$ 2719:X 23 Sep 18:16:45.993 # +odown master mymaster 127.0.0.1 9000
... snip more logs here ...
$ 2719:X 23 Sep 18:16:47.014 # +promoted-slave slave 127.0.0.1:9001
As you can see at ~46 the odown was reached, and at ~47 there was already the slave promoted to master (acknowledged via INFO output processing, not just sent).
4) Sentinel data loss has nothing to do with the speed of the failover (which is super fast), but mostly with partitions. The distributed system that you obtain summing Sentinel + Redis data stores is an eventually consistent system where the merge function is using the data set of the master with the greatest configEpoch (the latest promoted by the majority of Sentinels). This, and the fact that Redis uses asynchronous replication, means that isolated masters + clients, can process writes that will disappear. However Redis has configuration options (well documented in the Sentinel official doc) in order to bound the window of lost writes in a minority partition. The options I'm talking about allow isolated masters to stop accepting writes after some time no acknowledge is received by slaves.