3 ms·

Just to be clear: your complaint is etcd won't let you force the cluster into an inconsistent state to mask your poor infrastructure decisions and therefore etc

Just to be clear: your complaint is etcd won't let you force the cluster into an inconsistent state to mask your poor infrastructure decisions and therefore etcd is a "dangerous choice"?

vidarh 10y ago
The "poor infrastructure decision" would be to assume that because there's a network split it is automatically going to be unsafe to, for example, keep scheduling containers in each data centre, even the minority, without knowing whether or not constraints have been put in place to ensure this is safe to do. How many separate data centres do you want me to host in? Two is certainly insufficient, as if the data centre with the majority of Etcd nodes now falls of the network, you're SOL. Three? Think you can't have two fall of face of the earth the first time? Been there. Redundant connections doesn't always help if there are major external connectivity issues and you don't have multi-million dollar budgets for connectivity alone. Most small to mid size companies do not have the budget to run a system that is sufficiently redundant to be able to guarantee that they won't sooner or later be in a situation where either 1) the most viable partition is the minority, or 2) the Etcd cluster is centralised enough that too large parts of the cluster can loose access to it. When fighting large scale failures is the last time where you want to have to fight your tools to be able to convince them that you know what you're doing. For those kind of situations, Etcd's lack of support for more fine-grained control of consistency or for intentionally breaking apart the cluster and letting them continue to operate independently when you know what you're doing basically makes it unsuitable for running multi-data centre clusters for anything important. Which basically means any tools that assumes a single underlying Etcd cluster is appropriate as storage, for, say, a cluster scheduler (cough) becomes an inappropriate tool for those kind of setups.

Just to be clear: your complaint is etcd won't let you force the cluster into an inconsistent state to mask your poor infrastructure decisions and therefore etcd is a "dangerous choice"?

vidarh 10y ago

The "poor infrastructure decision" would be to assume that because there's a network split it is automatically going to be unsafe to, for example, keep scheduling containers in each data centre, even the minority, without knowing whether or not constraints have been put in place to ensure this is safe to do. How many separate data centres do you want me to host in? Two is certainly insufficient, as if the data centre with the majority of Etcd nodes now falls of the network, you're SOL. Three? Think you can't have two fall of face of the earth the first time? Been there. Redundant connections doesn't always help if there are major external connectivity issues and you don't have multi-million dollar budgets for connectivity alone. Most small to mid size companies do not have the budget to run a system that is sufficiently redundant to be able to guarantee that they won't sooner or later be in a situation where either 1) the most viable partition is the minority, or 2) the Etcd cluster is centralised enough that too large parts of the cluster can loose access to it. When fighting large scale failures is the last time where you want to have to fight your tools to be able to convince them that you know what you're doing. For those kind of situations, Etcd's lack of support for more fine-grained control of consistency or for intentionally breaking apart the cluster and letting them continue to operate independently when you know what you're doing basically makes it unsuitable for running multi-data centre clusters for anything important. Which basically means any tools that assumes a single underlying Etcd cluster is appropriate as storage, for, say, a cluster scheduler (cough) becomes an inappropriate tool for those kind of setups.