2 ms·
This, from your page, is worth highlighting: > because we wanted a simple, highly available service, not a strongly consistent one. I absolutely agree with t
by vidarh 11y ago
This, from your page, is worth highlighting:
> because we wanted a simple, highly available service, not a strongly consistent one.
I absolutely agree with this design choice.
This is the big flaw for me with how a lot of setups use Etcd, Consul, Zookeeper and the like.
A lot of the time they're being used because we need high-availability, not at all for their consistency guarantees, and their consistency guarantees has great potential to reduce the availability.
I've not run Zookeeper in production, but both with Etcd and Consul I've had situations where the clusters have gotten into totally broken states because the cluster membership has changed in bad ways. It makes sense for the guarantees they are providing. It does not make sense if you don't need the consistency guarantees.
The way I see it, lost service registrations are a non-issue as you generally want to continuously renew them combined with a TTL to avoid stale registrations, so the occasionally lost one due to e.g. a part of the service directory cluster failing at a bad time is no big deal. Conversely, if a service stays in the service directory when it shouldn't, the worst that happens is that you run into an unhealthy service when you try to issue requests, but that will happen during normal operation while using services with consistency guarantees anyway, since the service may have failed between health checks anyway, which means you still need to handle retries.
I'd rather use a service-discovery solution with fairly significant consistency issues that keeps replying "no matter what" than one which focuses on consistency but may stop answering.