7 ms·
Generic robust replicated log would be nice to have. For two products I've implemented leaderless oplog to back application level replication to sync not only d
by bvrmn 3y ago
Generic robust replicated log would be nice to have. For two products I've implemented leaderless oplog to back application level replication to sync not only database changes but also configuration files and other needed data. Works like a charm.
- HammadB 3y agoCurious - what was the specific leaderless replication strategy?
- bvrmn 3y agoEach operation has uniq id. Each node has own log with autoincremented op counter. According to topology rules. Each node tries to fetch new parts of remote logs with new ids from other nodes and apply to local state. Domain allows to express most operations in order independent manner. And for rest it's possible to use last op wins.
- EdwardDiego 3y agoHave you evaluated Bookkeeper? I haven't used it much, but keen on your thoughts on it especially on the robustness.
- esafak 3y agoIt underlies Apache Pulsar. https://pulsar.apache.org/docs/3.1.x/administration-zk-bk/ https://pulsar.apache.org/docs/3.1.x/administration-zk-bk/
- EdwardDiego 3y agoIt does :) That was why I asking. The times I evaluated Pulsar (and thus BookKeeper) I found myself unconvinced of BK's reliability, just because of wrong docs, comments in the user facing code that were wrong etc. But I never used it for real, with real data volume and throughput, so maybe the actual implementation was solid. The concepts in BK definitely made a lot of sense. You need bear in mind that Pulsar is the FOSS version of Twitter's homebrewed PubSub, with the same people leading it now (Streamnative) who led it in Twitter. And Twitter replaced it with Kafka. Which added to my caution. So yeah, thought I'd ask if they'd evaluated BK, because it's relevant to the reliability of Pulsar. If BK is ZK levels of solid, then that's awesome.