3 ms·
Related: Do you think SDKs for these APIs should themselves require you to interact with the cursors? If I ask a SDK to get me 100 items but internally it retu
by verst 4y ago
Related: Do you think SDKs for these APIs should themselves require you to interact with the cursors?
If I ask a SDK to get me 100 items but internally it returns only 50 before needing to make another call with a marker or cursor It would be nice if the SDK had an option to handle this for me (possibly exposing cursors still for those who really want it).
- jahewson 4y agoWhat if it fails after 50?
- verst 4y agoMaybe return the items it found so far and provide the cursor it failed on? Have a way to optionally provide the cursor? I can think of some ways I'd implement this in an SDK. Probably for simplicity I'd have some sort of JSON status field indicating the request (traversing all cursors) completed. If it's not complete I still would look for the next cursor it returned and do the usual thing manually.
- progval 4y agoSDKs should typically return a lazy iterator in languages which support it, so caller do not even care about pagination and just iterate on it until they don't want any more data; while the SDK fetches new pages when needed. In pseudo-Python: def get_stuff(): url = "https://example.org/api.php" while url: response = requests.get(url) yield from response.json() url = parse_link(response.headers["link"]).next and you call it with: for item in get_stuff(): print(item)
- deleted 4y ago[deleted]
- scarface74 4y agoThe AWS Python API users “paginators” to allow you to iterate over results with a single for loop and it handles the pagination in the background. The underlying API - which is also exposes - uses and opaque “NextToken”