3 ms·
I think this article could have done a better job explaining the underlining data structures, which would have then given the readers much better intuition why
by branko_d 5y ago
I think this article could have done a better job explaining the underlining data structures, which would have then given the readers much better intuition why this can actually work, and when.
In a nutshell, a table can be:
- Heap-based: the rows are in the unordered "heap", with any number of indexes (B-trees) pointing to them physically (by holding the row's physical address).
- Or clustered (aka. index-organized): the rows are in a primary/clustered index (B-tree), with any number of secondary indexes (also B-trees) pointing to them logically (by holding the copy of the primary key).
Fetching a row through any index in a heap-based table, or any secondary index in a clustered table, requires a separate step: physical heap access in the former case, or a clustered index seek in the later.
So...
- if you are using a heap-based table,
- or using a clustered table where pagination order differs from the clustering order
...then it makes sense to avoid the price of fetching the entire row for those rows that you are going to discard anyway. You just scan the index that matches the pagination order until you reach the correct page, and only then start fetching the rows.
It's still "offset" pagination, but can be much faster in practice than the naive implementation (for later pages).
OTOH, if you are paginating in the clustering order, then there is no separate step for fetching the row, so this method would be unlikely to bring any benefit.
---
A DBMS with a decent query planner should be able to do this kind of optimization automatically, through "predicate pushdown". But it may still be a good idea not to rely on it and make things blindingly obvious to the planner by structuring the SQL as shown in the article.
---
BTW, "cursor pagination" is also known as "keyset pagination" or "seek method". Curiously, actual database cursors can be used for pagination, assuming very tight transactional coordination between client and server (and very nasty consequences if that's lacking), but that's different from the "cursor pagination" as mentioned in the article.