4 ms·
I can't describe the exact use case, but suffice to say I'd like to incrementally replicate (in one direction) data, but I don't desire to track what has change
by atombender 3y ago
I can't describe the exact use case, but suffice to say I'd like to incrementally replicate (in one direction) data, but I don't desire to track what has changed or having to index the data.
I'm attracted to the simplicity behind Merkle trees where you can use hashes as a much smaller proxy for the upstream data, and even describe use the root-level hash as a "signature" representing the state of the dataset.
A naive, non-Merkle-y solution would be to associate each datum with a modification timestamp, and simply grab any data where "updated_at > $last_timestamp"; this would require an index to filter efficiently, however. Another simple solution would be a log that tracked every insert, and a process could then tail this log; this would require all the bookeeping this entails.
Thanks, I will check out those links.