42 ms·

http://restcookbook.com/Resources/pagination/ http://restcookbook.com/Resources/pagination/ "Even though it's tempting to create your own pagination scheme

http://restcookbook.com/Resources/pagination/ http://restcookbook.com/Resources/pagination/ "Even though it's tempting to create your own pagination scheme by "building" URLs, you should only use the information given by the API. This means, you cannot assume that the 3rd page can be found at /collection/3. If you do, your client will break as soon as the API changes its pagination behaviour." Is that a legitimate caveat? Are there clients that don't break when you change the API the client works with? Pretty smart clients.

dustingetz 14y ago
Consider a service powering a right click context menu. REST's HATEOAS constraint is a generalization of this idea - decoupling the API from the client. A lot of popular web APIs, e.g. Twitter, do not embrace HATEOAS, which is why API changes break clients. Which isn't really a big deal if you control all the clients.
- shawn-butler 14y ago
  I've never fully bought into this constraint, I guess. I can understand how it might be useful for machine-to-machine consumption of an API (which seems to be a sort of holy grail) or for an API that is meant to last for say a decade and be self-documenting between generations of programmers. But I guess it seems like neither of those are domains where anyone I know is really working; admittedly entirely anecdotal. API seem to change or get replaced by "the next best thing" on the order of months or years. And doing HATEOAS seems like making a c++ program const-correct. Always starts out well but pretty soon isn't viewed as worthwhile by powers that be. Do you have an example of a really good HATEOAS public API that is widely consumed?
  - w0utert 14y ago
    >> And doing HATEOAS seems like making a c++ program const-correct. I would argue that cons-correctness is probably one of the most important considerations of any public C++ API, so not really the best analogy ;-). Try to refactor some actively used C++ framework API that never bothered about const correctness some day, weeks of fun and bickering with your API clients guaranteed.
caseysoftware 14y ago
Ideally, it's done via Link Relations: http://www.iana.org/assignments/link-relations/link-relations.xml http://www.iana.org/assignments/link-relations/link-relation... The specific ones are first, last, next, and prev/previous. As long the name doesn't change, the URI behind the scenes should be able to change however you need. We do this in the Twilio API though we don't use the suggested Link Relations to describe the URIs: https://www.twilio.com/docs/api/rest/call#list-get-example-1 https://www.twilio.com/docs/api/rest/call#list-get-example-1
steveklabnik 14y ago
> Are there clients that don't break when you change the API the client works with? Pretty smart clients. Yes, you can write clients for GitHub's API that works that way, for example. Here, I whipped you up one: https://gist.github.com/steveklabnik/5282209/e48b100d95ff77fcfef4c02364e584725043c777 https://gist.github.com/steveklabnik/5282209/e48b100d95ff77f... This client will return you information about the first gist on the second page of the public list. It does this by fetching the list of gists from the root, then using the pagination to walk forward. GitHub can change all of this, and it'll still work. $ ruby fetch_gists.rb DATA: ID: 5282131 URL: https://api.github.com/gists/5282131 USER: sparksp Bonus: here's a slightly refactored version. The logic is more clear when you move things into helpers. https://gist.github.com/steveklabnik/5282209/52ed62366586838cdf987682dde7fd05eb1848e7 https://gist.github.com/steveklabnik/5282209/52ed62366586838...
- barkmadley 14y ago
  If I may, the goal seems a bit incorrectly specified for hypermedia. For instance, if github decided to change the number of results they return per page, then your script would not break, but it would have a different meaning. I would suggest a better goal to be "get the 11th public gist". Although that still isn't perfect since it doesn't mention how the collection would be sorted (I presume by date).
  - steveklabnik 14y ago
    Absolutely, this goal is bad. I was just trying to show off the mechanics, and that clients don't have to be complex.
buro9 14y ago
Two things: 1) Yes, you should only use info supplied by the API. This should mean that the API is helping to ensure the client never sends the user down a UX dead-end, and also that the client does not break when the API changes. But... 2) I don't like the thought of collections having a resource identifier for a page. As items are deleted from the collection, the resource identifiers now represent a modified resource and break caching (delete item 3 of a 5 item per page collection, and all resources in the collection - logical pages - have been modified implicitly). I much prefer using query string for this, as it is a query on a collection. But... the API should still be the thing that generates/builds these URLs, the client should never do this work. The client must use the provided first|prev|self|next|last URLs, otherwise the client may break in the future. An example of how I prefer pagination: "collectionName": { "total": 861, "limit": 10, "offset": 100, "pages": 87, "links": [ {"rel": "first", "href": "/api/things?limit=10"}, {"rel": "prev", "href": "/api/things?limit=10&offset=90"}, {"rel": "self", "href": "/api/things?limit=10&offset=100"}, {"rel": "next", "href": "/api/things?limit=10&offset=110"}, {"rel": "last", "href": "/api/things?limit=10&offset=860"}, ], "items": [ ... ] } Where 'limit' and 'offset' are provided by the query string (but default to 25 and 0 respectively) and the API takes care of generating valid links (like not including 'prev' if you are on the first page'). This way the client does what it should, just use the links provided. And whilst I'm here... one of the things I dislike about link relations is how they presently fail to mention the method you should use (let alone content-types acceptable by that end-point). In the example above every link is a GET, in the example on this page http://restcookbook.com/Basics/hateoas/ http://restcookbook.com/Basics/hateoas/ they're probably POST. In the link relation assignments http://www.iana.org/assignments/link-relations/link-relations.xml http://www.iana.org/assignments/link-relations/link-relation... 'edit' is probably a PUT. There's an 'edit', but not a 'delete', so it's not as if just sending more than one 'rel' might describe the method. Today the audience of an API is a developer, so it's fine to just point them at the documentation. But tomorrow it may well be a computer.
- bmelton 14y ago
  I do something similar, with the only exception being that I've never bothered calculating the number of pages (or links to other pages) on the server side, preferring to just let the client figure it out. Mainly, I've found that for dynamic data sets, the links could possibly change between the first page and the 87th, as well as the number of pages, voiding the practicality of the data set. Give them the number of records per page, the offset, the number of records, and let the clients determine the rest (ideally you'd have documentation showing how to paginate, whether it be ?page=1 or ?offset=20, or whatever). That looks very complete though.

dustingetz 14y ago

Consider a service powering a right click context menu. REST's HATEOAS constraint is a generalization of this idea - decoupling the API from the client. A lot of popular web APIs, e.g. Twitter, do not embrace HATEOAS, which is why API changes break clients. Which isn't really a big deal if you control all the clients.

shawn-butler 14y ago
I've never fully bought into this constraint, I guess. I can understand how it might be useful for machine-to-machine consumption of an API (which seems to be a sort of holy grail) or for an API that is meant to last for say a decade and be self-documenting between generations of programmers. But I guess it seems like neither of those are domains where anyone I know is really working; admittedly entirely anecdotal. API seem to change or get replaced by "the next best thing" on the order of months or years. And doing HATEOAS seems like making a c++ program const-correct. Always starts out well but pretty soon isn't viewed as worthwhile by powers that be. Do you have an example of a really good HATEOAS public API that is widely consumed?
- w0utert 14y ago
  >> And doing HATEOAS seems like making a c++ program const-correct. I would argue that cons-correctness is probably one of the most important considerations of any public C++ API, so not really the best analogy ;-). Try to refactor some actively used C++ framework API that never bothered about const correctness some day, weeks of fun and bickering with your API clients guaranteed.

caseysoftware 14y ago

Ideally, it's done via Link Relations: http://www.iana.org/assignments/link-relations/link-relations.xml http://www.iana.org/assignments/link-relations/link-relation... The specific ones are first, last, next, and prev/previous. As long the name doesn't change, the URI behind the scenes should be able to change however you need. We do this in the Twilio API though we don't use the suggested Link Relations to describe the URIs: https://www.twilio.com/docs/api/rest/call#list-get-example-1 https://www.twilio.com/docs/api/rest/call#list-get-example-1

steveklabnik 14y ago

> Are there clients that don't break when you change the API the client works with? Pretty smart clients. Yes, you can write clients for GitHub's API that works that way, for example. Here, I whipped you up one: https://gist.github.com/steveklabnik/5282209/e48b100d95ff77fcfef4c02364e584725043c777 https://gist.github.com/steveklabnik/5282209/e48b100d95ff77f... This client will return you information about the first gist on the second page of the public list. It does this by fetching the list of gists from the root, then using the pagination to walk forward. GitHub can change all of this, and it'll still work. $ ruby fetch_gists.rb DATA: ID: 5282131 URL: https://api.github.com/gists/5282131 USER: sparksp Bonus: here's a slightly refactored version. The logic is more clear when you move things into helpers. https://gist.github.com/steveklabnik/5282209/52ed62366586838cdf987682dde7fd05eb1848e7 https://gist.github.com/steveklabnik/5282209/52ed62366586838...

barkmadley 14y ago
If I may, the goal seems a bit incorrectly specified for hypermedia. For instance, if github decided to change the number of results they return per page, then your script would not break, but it would have a different meaning. I would suggest a better goal to be "get the 11th public gist". Although that still isn't perfect since it doesn't mention how the collection would be sorted (I presume by date).
- steveklabnik 14y ago
  Absolutely, this goal is bad. I was just trying to show off the mechanics, and that clients don't have to be complex.

buro9 14y ago

Two things: 1) Yes, you should only use info supplied by the API. This should mean that the API is helping to ensure the client never sends the user down a UX dead-end, and also that the client does not break when the API changes. But... 2) I don't like the thought of collections having a resource identifier for a page. As items are deleted from the collection, the resource identifiers now represent a modified resource and break caching (delete item 3 of a 5 item per page collection, and all resources in the collection - logical pages - have been modified implicitly). I much prefer using query string for this, as it is a query on a collection. But... the API should still be the thing that generates/builds these URLs, the client should never do this work. The client must use the provided first|prev|self|next|last URLs, otherwise the client may break in the future. An example of how I prefer pagination: "collectionName": { "total": 861, "limit": 10, "offset": 100, "pages": 87, "links": [ {"rel": "first", "href": "/api/things?limit=10"}, {"rel": "prev", "href": "/api/things?limit=10&offset=90"}, {"rel": "self", "href": "/api/things?limit=10&offset=100"}, {"rel": "next", "href": "/api/things?limit=10&offset=110"}, {"rel": "last", "href": "/api/things?limit=10&offset=860"}, ], "items": [ ... ] } Where 'limit' and 'offset' are provided by the query string (but default to 25 and 0 respectively) and the API takes care of generating valid links (like not including 'prev' if you are on the first page'). This way the client does what it should, just use the links provided. And whilst I'm here... one of the things I dislike about link relations is how they presently fail to mention the method you should use (let alone content-types acceptable by that end-point). In the example above every link is a GET, in the example on this page http://restcookbook.com/Basics/hateoas/ http://restcookbook.com/Basics/hateoas/ they're probably POST. In the link relation assignments http://www.iana.org/assignments/link-relations/link-relations.xml http://www.iana.org/assignments/link-relations/link-relation... 'edit' is probably a PUT. There's an 'edit', but not a 'delete', so it's not as if just sending more than one 'rel' might describe the method. Today the audience of an API is a developer, so it's fine to just point them at the documentation. But tomorrow it may well be a computer.

bmelton 14y ago
I do something similar, with the only exception being that I've never bothered calculating the number of pages (or links to other pages) on the server side, preferring to just let the client figure it out. Mainly, I've found that for dynamic data sets, the links could possibly change between the first page and the 87th, as well as the number of pages, voiding the practicality of the data set. Give them the number of records per page, the offset, the number of records, and let the clients determine the rest (ideally you'd have documentation showing how to paginate, whether it be ?page=1 or ?offset=20, or whatever). That looks very complete though.