3 ms·
An easy example of this is the implementation in Apache Cassandra In Cassandra, we use MTs during active anti-entropy repair, where we have N replicas of some
by jjirsa 7y ago
An easy example of this is the implementation in Apache Cassandra
In Cassandra, we use MTs during active anti-entropy repair, where we have N replicas of some data and want to make sure they’re in sync
The repair command will repair a range of data - in real terms, it’s repairing all keys between two tokens, where the token is the murmur3 hash of the key. The whole cluster covers 2^64, but a typical repair will repair perhaps 2^40 or smaller.
When you’re repairing 2^40 keys, you don’t want to compare each of them one by one - if they match, you’re sending a ton of data across the network for no reason. Instead, if each replica builds a needle tree representing the data for that range, you can not only tell quickly if they’re in sync (comparing the roots), but you can descend to identify the minimal set of data to stream between replicas.
The Cassandra implementation isn’t a textbook implementation of the concept, but it’s easy to think about.