3 ms·

An API that uses the Link header can return a set of ready-made links so the API consumer doesn't have to construct links themselves. This is especially importa

An API that uses the Link header can return a set of ready-made links so the API consumer doesn't have to construct links themselves. This is especially important when pagination is cursor based. In the header is nice because then there’s no need to parse the payload to get the next page. But better still is to avoid cursor based pagination. Instead give me a cheap endpoint to get the total number of results and the configured max results per page and have constructable urls. (E.g. “?page=4” or “?offset=500”). This way generating all the urls can be a completely separate process from pulling the results.

mtsr 7y ago
That's indeed the nicer way of paginating, but it breaks if the underlying resultset changes between requests. Which is exactly when cursor based pagination is generally used.
- bradleyjg 7y ago
  If the changes in the resultset are additive it's no problem as long as they are sorted in such a way as new results go to the end (which the api should at least make an option if possible). Updates to data within results may be a problem because you can end up with a dataset that has a view of the world that doesn't represent any particular time, but in many cases are safe. Deletions screw everything up and should be avoided if possible. The general solution to this problem is to allow as part of the query some particular time that you want the results to reflect the state of the world as of, but that's obviously going to be expensive to serve.
caseysoftware 7y ago
+1 to what your other response said You have to remember the goal of pagination: to move through a collection of results sequentially. If your underlying page is constantly changing (as the other response noted), then you have NO way to know what should be your next intended offset/page to move either or back. A simple sort with "results always go here" seems like a good approach but now you're packing additional understanding into using your API which is totally out of band with it. Or using a different sort blows it up rending that approach useless. Cursors are the only approach that actually accomplishes the goal.
- bradleyjg 7y ago
  If the underlying data is constantly changing, how do cursors solve that problem? The only guarantee I get asking for the next page after a given item is that it won't contain the last item I've already seen. There's no other inherent guarantees. The page could contain all items I've already seen, early pages (in this new version of the underlying dataset) could have items I've never seen, and so on. It's as arbitrary as page numbers but without the corresponding convenience.
  - deleted 7y ago
    [deleted]
ptman 7y ago
How about keyset pagination instead of limit & offset? https://use-the-index-luke.com/no-offset https://use-the-index-luke.com/no-offset
- bradleyjg 7y ago
  I don’t like it because it introduces a linear dependency for every step on the ingest on the all the prior steps. It’s convenient for the server but not the client.

mtsr 7y ago

That's indeed the nicer way of paginating, but it breaks if the underlying resultset changes between requests. Which is exactly when cursor based pagination is generally used.

bradleyjg 7y ago
If the changes in the resultset are additive it's no problem as long as they are sorted in such a way as new results go to the end (which the api should at least make an option if possible). Updates to data within results may be a problem because you can end up with a dataset that has a view of the world that doesn't represent any particular time, but in many cases are safe. Deletions screw everything up and should be avoided if possible. The general solution to this problem is to allow as part of the query some particular time that you want the results to reflect the state of the world as of, but that's obviously going to be expensive to serve.

caseysoftware 7y ago

+1 to what your other response said You have to remember the goal of pagination: to move through a collection of results sequentially. If your underlying page is constantly changing (as the other response noted), then you have NO way to know what should be your next intended offset/page to move either or back. A simple sort with "results always go here" seems like a good approach but now you're packing additional understanding into using your API which is totally out of band with it. Or using a different sort blows it up rending that approach useless. Cursors are the only approach that actually accomplishes the goal.

bradleyjg 7y ago
If the underlying data is constantly changing, how do cursors solve that problem? The only guarantee I get asking for the next page after a given item is that it won't contain the last item I've already seen. There's no other inherent guarantees. The page could contain all items I've already seen, early pages (in this new version of the underlying dataset) could have items I've never seen, and so on. It's as arbitrary as page numbers but without the corresponding convenience.
- deleted 7y ago
  [deleted]

ptman 7y ago

How about keyset pagination instead of limit & offset? https://use-the-index-luke.com/no-offset https://use-the-index-luke.com/no-offset

bradleyjg 7y ago
I don’t like it because it introduces a linear dependency for every step on the ingest on the all the prior steps. It’s convenient for the server but not the client.