5 ms·

> it uses offsets for pagination... understood to be bad practice by today’s standards. Although convenient to use, offsets are difficult to keep performant in

> it uses offsets for pagination... understood to be bad practice by today’s standards. Although convenient to use, offsets are difficult to keep performant in the backend This is funny. Using offsets is known to be bad practice because.... it’s hard to do. Look I’m just a UI guy so what do I know. But this argument gets old because I’m sorry, but people want a paginated list and to know how many pages are in the list. Clicking “next page” 10 times instead of clicking to page 10 is bullshit, and users know it.

alexchamberlain 6y ago
I can't find one right now, but I feel like there must be an algorithm that can identify the key that can be used for each page, yet cheaper than a full sort (ie cheaper than offset pagination in generality). Of course, a skip list would work for a secondary index.
- juancn 6y ago
  This is the best one I've been able to come up with: https://engineering.medallia.com/blog/posts/sorting-and-paging-on-distributed-data/ https://engineering.medallia.com/blog/posts/sorting-and-pagi...
juancn 6y ago
The hardest part is to be able to parallelize the sorts in the backend and keep the working sets reasonably sized. If you ask for "first 50 items after key X", you just need to keep a priority queue of size 50 on each BE node and merge them before returning them (I'm assuming a distributed backend). It doesn't matter on which page you are. But if you specify "first 50 items after element N" it gets really tricky, each BE shard needs to sort the first N elements, and it can use some trickery to avoid doing a naive merge (see: https://engineering.medallia.com/blog/posts/sorting-and-paging-on-distributed-data/ https://engineering.medallia.com/blog/posts/sorting-and-pagi... ). You can at most save some transfer over the network.
yxhuvud 6y ago
No, what is bullshit is having the option to go to page 10 in the first place. If the user does that then the UI is already broken. What is needed is good filter abilities.
- draw_down 6y ago
  Filtering is an orthogonal concern to this, I'm not advocating against filtering. Sometimes you don't know what you're looking for, and you will go insane if you're not sure if the thing you want is on page 8 or 12 if you have to keep clicking "next/prev" to get between them. Another way to put what you're saying is essentially that the user should never have more than one page of results. Putting it that way kind of shows that it's not really a solution
Rafert 6y ago
How do people even figure out they need to be on page 10? Sounds like a filter of some kind (e.g. before a date) would be better, except they trained themselves to use x amount of pages to approximate. Either way, 10 pages isn't so bad but tens of thousands can become troublesome as explained on https://shopify.engineering/pagination-relative-cursors https://shopify.engineering/pagination-relative-cursors
- PurplePanda 6y ago
  Maybe they've been there before and happen to remember "page 10". Maybe someone else has been there before and told them "Go to page ten". Maybe you know that there are 20 pages, and are looking to find the median, or are just informally playing around, looking at the distribution. Same as you'd do with an interesting page of a book. I don't think I'd stop using page numbers in dead tree books if they somehow came with filters.

alexchamberlain 6y ago

I can't find one right now, but I feel like there must be an algorithm that can identify the key that can be used for each page, yet cheaper than a full sort (ie cheaper than offset pagination in generality). Of course, a skip list would work for a secondary index.

juancn 6y ago
This is the best one I've been able to come up with: https://engineering.medallia.com/blog/posts/sorting-and-paging-on-distributed-data/ https://engineering.medallia.com/blog/posts/sorting-and-pagi...

juancn 6y ago

The hardest part is to be able to parallelize the sorts in the backend and keep the working sets reasonably sized. If you ask for "first 50 items after key X", you just need to keep a priority queue of size 50 on each BE node and merge them before returning them (I'm assuming a distributed backend). It doesn't matter on which page you are. But if you specify "first 50 items after element N" it gets really tricky, each BE shard needs to sort the first N elements, and it can use some trickery to avoid doing a naive merge (see: https://engineering.medallia.com/blog/posts/sorting-and-paging-on-distributed-data/ https://engineering.medallia.com/blog/posts/sorting-and-pagi... ). You can at most save some transfer over the network.

yxhuvud 6y ago

No, what is bullshit is having the option to go to page 10 in the first place. If the user does that then the UI is already broken. What is needed is good filter abilities.

draw_down 6y ago
Filtering is an orthogonal concern to this, I'm not advocating against filtering. Sometimes you don't know what you're looking for, and you will go insane if you're not sure if the thing you want is on page 8 or 12 if you have to keep clicking "next/prev" to get between them. Another way to put what you're saying is essentially that the user should never have more than one page of results. Putting it that way kind of shows that it's not really a solution

Rafert 6y ago

How do people even figure out they need to be on page 10? Sounds like a filter of some kind (e.g. before a date) would be better, except they trained themselves to use x amount of pages to approximate. Either way, 10 pages isn't so bad but tens of thousands can become troublesome as explained on https://shopify.engineering/pagination-relative-cursors https://shopify.engineering/pagination-relative-cursors

PurplePanda 6y ago
Maybe they've been there before and happen to remember "page 10". Maybe someone else has been there before and told them "Go to page ten". Maybe you know that there are 20 pages, and are looking to find the median, or are just informally playing around, looking at the distribution. Same as you'd do with an interesting page of a book. I don't think I'd stop using page numbers in dead tree books if they somehow came with filters.