4 ms·

There is a cursor pagination spec available:https://relay.dev/graphql/connections.htm https://relay.dev/graphql/connections.htm which is used quite often.

cameronh90 4y ago
I'm aware (and that's what we use) but it still suffers the same issues. To use it with deep nested queries, you essentially need to use something like Apollo which can rewrite queries for enumeration where it doesn't re-query the things from the original query that it already has cached. However, what I'm referring to - and I'm not sure if it has a better name - is the type of cursor where you can create it, then enumerate it separately, across multiple requests. In pseudocode it would be something like: let c = db.users(name like 'c%').groups(memberCount > 50).members # request 1 while c.any(): c.fetch(max=50) # request 2+ Of course, for that to work, the entire state of the query needs to be encapsulated within that cursor or saved in some temporary session variable on the server. It also doesn't really fit into GraphQL's model right now. I seem to recall Hacker News actually used something similar for pagination back in the day, and if you left a page open for too long then tried to press next, you got an invalid continuation error or something to that effect. Not sure if that was something similar...
- yencabulator 4y ago
  And where this really gets tricky is when it's not a loop like that, but the next fetch happens only after user interaction, perhaps 5 hours later. To do that right, you need to encode everything the query is ordered by into a stateless continuation token, for example here that might be [user_name, group_id, member_id]. You can make that opaque to the client, but that's what you need to continue the search efficiently. Relay tries to do something like that, but it gets ridiculously complicated: https://relay.dev/docs/v12.0.0/guided-tour/list-data/advanced-pagination/ https://relay.dev/docs/v12.0.0/guided-tour/list-data/advance...

cameronh90 4y ago

I'm aware (and that's what we use) but it still suffers the same issues. To use it with deep nested queries, you essentially need to use something like Apollo which can rewrite queries for enumeration where it doesn't re-query the things from the original query that it already has cached. However, what I'm referring to - and I'm not sure if it has a better name - is the type of cursor where you can create it, then enumerate it separately, across multiple requests. In pseudocode it would be something like: let c = db.users(name like 'c%').groups(memberCount > 50).members # request 1 while c.any(): c.fetch(max=50) # request 2+ Of course, for that to work, the entire state of the query needs to be encapsulated within that cursor or saved in some temporary session variable on the server. It also doesn't really fit into GraphQL's model right now. I seem to recall Hacker News actually used something similar for pagination back in the day, and if you left a page open for too long then tried to press next, you got an invalid continuation error or something to that effect. Not sure if that was something similar...

yencabulator 4y ago
And where this really gets tricky is when it's not a loop like that, but the next fetch happens only after user interaction, perhaps 5 hours later. To do that right, you need to encode everything the query is ordered by into a stateless continuation token, for example here that might be [user_name, group_id, member_id]. You can make that opaque to the client, but that's what you need to continue the search efficiently. Relay tries to do something like that, but it gets ridiculously complicated: https://relay.dev/docs/v12.0.0/guided-tour/list-data/advanced-pagination/ https://relay.dev/docs/v12.0.0/guided-tour/list-data/advance...