4 ms·
That solves the problem of items added concurrently (by deduplicating them) but that doesn't solve the problem of removed items.
by ngrilly 6y ago
That solves the problem of items added concurrently (by deduplicating them) but that doesn't solve the problem of removed items.
- dec0dedab0de 6y agoThe only thing that completely solves for removed items is constantly polling, or setting up a webhook/callback of some kind. Padding the offset would solve for the problem mentioned where deleting an item would mean some non deleted items are not included in the paging results because they got moved up a page after that page was requested, but before the next page was requested. For example If I request 100 items at a time but set my offset to be 90 more than what i have received so far, i can expect my response to have duplicates, if it does not then i know my offset was not padded enough and i can request from a different offset. Of course you would adjust the numbers based on knowledge of the data. Edit: If you used the ID of the last item instead of an offset, then you could get errors if your last item is in fact the one that was deleted.
- ngrilly 6y agoYour padding + deduplication solution is nice. > Edit: If you used the ID of the last item instead of an offset, then you could get errors if your last item is in fact the one that was deleted. If the items are sorted by ID, then we use the ID of the last item x. But for example if they are sorted by timestamp, then we use the timestamp of the last item (and maybe the ID to break ties). Then it doesn't matter if the last item is still there or not. We are only considering values before or after depending on the sort order.