4 ms·
I have been burn by Consul countless time and advise to not actively using it. Google around company has down time with consult https://www.protocol.com/newsl
by kureikain 5y ago
I have been burn by Consul countless time and advise to not actively using it.
Google around company has down time with consult
https://www.protocol.com/newsletters/protocol-enterprise/roblox-hashicorp-consul-intel-earnings?rebelltitem=1#rebelltitem1 https://www.protocol.com/newsletters/protocol-enterprise/rob...
https://danveloper.medium.com/on-infrastructure-at-scale-a-cascading-failure-of-distributed-systems-7cff2a3cd2df https://danveloper.medium.com/on-infrastructure-at-scale-a-c...
I had use consul extensively in 2016-2017 with consul template.
The problem with consul is it's very easy to reject connection. Accidently have a node with too old or too new version join and entire cluster will be down. Which can happen easily in an autoscaling env.
Another issue is again, agent takes a lot of time to be cleanedup and removed, say in autoscaling vm where node come and go all the time you ended up with so many dead node on consul ui. Again, this is 2017 so It might changed now.
I have been replace consul with something suprisingly simple, and I think in retrospective that's all I want.
A local process run a key value API on top of a local database(can be SQLite or anything up to your taste. Its replace its state with a central server.
If the central server is down, the local agent still has its state to continue serving stale data.
- wink 5y agoCan't speak for current experience, but we've used it ca early 2016 - late 2017 on ~100 VMs on GCP and it performed flawlessly. No autoscaling though.
- liveoneggs 5y agoyou autoscale the consensus system?
- viraptor 5y agoI take it as a general concept of asg, not just scaling. For example I have all instances in some autoscaling group to enable transparent replacements and make rolling updates easier. (Update the requested image, then just start killing the old ones)
- liveoneggs 5y agoin the case of the parent comment I think he means, literally, having them scale up and down. I mostly have experience with zookeeper and I treat them very very carefully.