4 ms·
How to Get Stronger Consistency Out of a Datastore (2019)
- pbalau 5y agoI'm not sure about the first part of the solution. Why isn't it enough for the "post process" consumer to wait for events on a "persisted events" topic? And have snuba copy there a message only after it was persisted (and available on all replicas)? /edit: actually, why does the "post process" consumer even cares about events not persisted yet?
- deleted 5y ago[deleted]
- aparsons 5y agoSequencing in distributed systems is difficult to handle correctly. When faced with such a situation, it’s often a matter of philosophy rather than technical brilliance - are you willing to accept that failure is inevitable even with a neat “waiter” implementation? What happens if the waiter waits forever? Is shaving seconds tangibly benefitting customers (for Sentry it might) and worth the operational overhead? Framing these problems slightly differently - is 10 second latency close enough to “real time”? - unlocks solutions like an online batch processor that reads out of your DB (if it doesn’t support NOTIFY) on an indexed incremental identifier.