4 ms·
It the system is moving fast this is still somewhat complicated to implement robustly because by the time the "catchup" request to t0 returns, more time has pas
by aboodman 5y ago
It the system is moving fast this is still somewhat complicated to implement robustly because by the time the "catchup" request to t0 returns, more time has passed and more events have happened, so you still can't resume consuming the webhooks.
To be correct with such a system you have to be prepared to queue the incoming webhook events, do the catchup query, then replay the queued events.
- MrStonedOne 5y agoonly if the order of events matters.
- fiddlerwoaroof 5y agoYeah, if the events can be handled idempotently and this handling properly accounts for time, you don’t have to stop processing webhooks while filling in the gaps.
- aboodman 5y agoOK but, it usually matters. If the events represent changes to some object, those changes almost always have to be handled in order if you are to arrive at the same end state as the source. edit: Also, idempotent is not the right term here. Idempotent just would mean the event could be handled by the receiver multiple times w/o changing the meaning. If you need the events to be applicable out of order, then you need them to be commutative. This is a much more difficult property to ensure and in practice, I am guessing, almost non-existent in deployed webhook APIs.
- fiddlerwoaroof 5y agoI think that’s right re. idempotency. However, I think if each webhook is a statement of the state of an object at time t (using some monotonic definition of time, like a Lamport clock), commutativity is trivial: compare the time in the webhook event to the time in your local database, and only update your database if the webhook event is newer.