4 ms·

> you'd only backfill the small amount of data from the keyspace "stolen" by the new node I think this is the part I'm not so sure about. Say I have 100 stats

> you'd only backfill the small amount of data from the keyspace "stolen" by the new node I think this is the part I'm not so sure about. Say I have 100 stats, and of course each stat is per forum, per day (going back from 1 day to ... 5 years?). How do I know what keys were just "stolen"? Do I have my new-node code hash every possible key (all stats for all forums for all hours for all time) to see which might go to that node? And then it reverses that key to know what it "means" to backfill it? (I need to do that followup post as the way our data 'flows' in is applicable here)

sigil 16y ago
> How do I know what keys were just "stolen"? Do I have my new-node code hash every possible key (all stats for all forums for all hours for all time) to see which might go to that node? And then it reverses that key to know what it "means" to backfill it? Right, you'd have to iterate through all zset elements on the existing node, applying the consistent hash function to decide whether or not the element will be stolen by the new node. If the element itself doesn't contain user id (or whatever you shard on) all bets are off.