Google Spanner (and one other system I worked with) had this neat trick up its sleeve: Utilizing comparatively lighter witness replicas, that only participate i

Google Spanner (and one other system I worked with) had this neat trick up its sleeve: Utilizing comparatively lighter witness replicas, that only participate in a quorums and keep a time-truncated (recent) un-materialized log necessary but answer no queries. Witnesses could, however, be used to bring other full nodes up-to-speed. This meant, availability of the cluster overall generally went up, assuming co-ordinated failure is rare and there's enough (hardware and software) heterogeneity in the cluster [0]. Regarding consistency in distributed systems, there's a lot to untangle here that even experts like Pat Helland are left lamenting the confusion both in the academia and in the engineering circles [1] [0] http://www2.cs.uh.edu/~paris/MYPAPERS/Icdcs86.pdf http://www2.cs.uh.edu/~paris/MYPAPERS/Icdcs86.pdf [1] https://pathelland.substack.com/p/dont-get-stuck-in-the-con-game-v2 https://pathelland.substack.com/p/dont-get-stuck-in-the-con-...