2 ms·
Yeah, you should generate an always-increasing sequence number in the source database. Timestamps cannot be used to "select ... where T > ?", since at the time
by lobster_johnson 12y ago
Yeah, you should generate an always-increasing sequence number in the source database.
Timestamps cannot be used to "select ... where T > ?", since at the time of polling another record may come in with the same timestamp as the newest one, and this will skip records; "where T >= ?" won't work either unless you can de-duplicate, and that requires local state.
- stickfigure 12y agoUnfortunately you don't get monotonic sequence numbers in distributed databases. Timestamps actually work very well for most applications, at the cost of extra reindexing proportional to how closely you can manage clock skew. Keep in mind that you are always subtracting a reasonable max skew value and getting extra records. All operations on the secondary store must be upserts.