4 ms·
> Now my functions just downloaded the file from S3, if it was newer than the local copy if you have strong consistency requirements, this doesn't work. synchr
by rmbyrro 2y ago
> Now my functions just downloaded the file from S3, if it was newer than the local copy
if you have strong consistency requirements, this doesn't work. synchronizing clocks reliably between different servers is surprisingly hard. you might end up working with stale data. might work for use cases that can accept eventual consistency.
- 66yatman 2y agoThis shouldn't depend on clocks, just tracking Etag is more consistency proof.
- Spivak 2y agoI don't really know if that matters for this use case. Just by the very nature of source_data -> processing -> dest_data taking nonzero time anything consuming dest_data must already be tolerant of some amount of lag. And how it's coded guarantees you can never observe dest_data going new -> old -> new.
- jedberg 2y agoOne of the announcements from AWS this year at Re:invent is that they now can guarantee that the instances clocks are synced within microseconds of each other. Close enough that you can rely on it for distributed timekeeping.
- Dylan16807 2y agoIf you have strong consistency requirements, then it doesn't work by the very nature of making multiple copies of the database. Even if the clocks are perfect. (Though the clocks are probably close enough that it doesn't matter.)
- efitz 2y agoI versioned the databases and my use case didn’t require strong consistency.
- iskela 2y agoWouldnt e-tag version numbers also work? Or just havkng .jsom with version metadata next to the db blob? No need to sync clocks. Just GET the small db-ver.json and compare version details?