37 ms·

How to Design Better APIs

492 points by adrianomartins 5y ago

barefeg 5y ago
The PUT vs PATCH is debatable in different levels. One simple issue is how to resolve complex merges in a PATCH. For example if we patch a key that contains a list, what will be the expected result?
- ajcp 5y ago
  Even beyond PUT vs PATCH I found this statement to be rather naive: "From my experience, there barely exist any use cases in practice where a full update on a resource would make sense" Just in finance I can think of *dozens* of use cases around invoicing and purchasing *alone*. A lot of times thousands of resources may need just one field "corrected", but hundreds others "reset" to ensure the proper retriggering of routing, workflows, and allocations. That the resources need to exist as originals is incredibly important for these kinds of things.
pokoleo 5y ago
The error messages could be better yet. The example uses a different code per issue, for instance: "user/email_required". Most integrators will build their UI to highlight the input fields that contain an error. Making them parse the `code` field (or special-case each possible code) is pretty toilsome. // from blog post { "code": "user/email_required", "message": "The parameter [email] is required." } Make it parseable: // improved { "message": "An email is required.", "error": "missing_parameter", "parameter": "user.email" } In addition, I: * rewrote `message` to be an acceptable error message displayed to (non-technical) end-users * moved message to be the first field: some developer tools will truncate the JSON response body when presenting it in the stack trace. --- As an added bonus, structured data allows you to analyze `error` frequencies and improve frontend validation or write better error messages: https://twitter.com/VicVijayakumar/status/1495092161821429761 https://twitter.com/VicVijayakumar/status/149509216182142976...
- smoyer 5y ago
  It also doesn't hurt to repeat the HTTP status code in the JSON body - when you receive a response, the status code and entity body are coupled but even if the server logs the status code, they're often decoupled in the logging system - having both in one log entry is way easier!
- moltar 5y ago
  Agreed. But rather than reinvent, let’s just use JSON API standard? https://jsonapi.org/format/ https://jsonapi.org/format/ (Scroll to the very bottom)
  - pokoleo 5y ago
    The article is about REST API design, not JSON APIs! It’s a whole different ballpark.
- pan69 5y ago
  It might be to formal for your use-case, but there is a standard defined for error responses in RFC 7807: https://datatracker.ietf.org/doc/html/rfc7807 https://datatracker.ietf.org/doc/html/rfc7807
  - aeontech 5y ago
    Wow, I had never seen an API with errors at this level of detail… I feel lucky when they at least use sane status codes instead of always giving back 200 and a maybe-json-maybe-plaintext-maybe-empty body… I’d love to hear from anyone who has encountered APIs in the wild that actually implement this standard!
    - JimDabell 5y ago
      I use Problem Details in most APIs I build. It’s as simple to generate a Problem Detail as it is to generate ad-hoc errors, but you can re-use code.
    - abdusco 5y ago
knighthack 5y ago
A good checklist of things to mind when designing APIs. Sometimes you forget the good things.
- rbluethl 5y ago
  Thanks for the kind words! Appreciate you reading it.
pan69 5y ago
Some nice tips in here. However, tip 15, I strongly disagree with: > 15. Allow expanding resources I would suggest the opposite. A REST API should not return nested resources at all. Instead, and to stay with the example provided on the website, to obtain the "orders", the /users/:id/orders endpoint should be called. It might be tempting to return nested resources, because clients would only have to make a single call.Technically this is true but once the domain of your API starts to grow, you will find that the interface will become increasingly muddled. The suggestion provided (use query parameters) is basically a work-around. If you want to offer this to your clients, front your REST API with a GraphQL API instead. It is literally the problem that GraphQL solves. Keep your REST API clean and dumb and focused around CRUD.
- metadat 5y ago
  Dear PAM69, this is great advice if you want to end up with a slow-to-load, low-performing web application that your customers complain about and hate using. But at least it'll adhere to a specific notion of architectural "purity", right? /s Anytime clients need to make 15 async calls before the UI can be displayed, you're headed up the creek. Generally speaking, this is an anti-pattern. There are exceptions, but they're not the rule. It's better to weigh the tradeoffs in any given situation and make a decision about bundling sub-resource references based on what you're optimizing for. A few quick examples: * Dev speed: Unbundled * Quick-loading UI: Bundled * Sub-resources are computationally-expensive to query? Unbundled This has been my experience consistently throughout 20 years of web-tier development.
  - BulgarianIdiot 5y ago
    Well they said use GraphQL, which is a solid way to avoid REST's clumsiness once and for all.
    - nerdponx 5y ago
      Sort of, but is it any better if the GraphQL layer still has to make 15 requests in order to serve a single useful response?
drdrey 5y ago
Why would one prefer ISO 8601 dates over POSIX timestamps?
- paulryanrogers 5y ago
  Human readability most likely
- mulmboy 5y ago
  They can include a local timezone. Sometimes there's a big difference between 12am in UTC+0 and 3am in UTC+3 despite representing the same instant in time.
  - inopinatus 5y ago
    True enough, but I would still recommend that API responses normalize to UTC (Z suffix) in the general case and document as much, and if actually returning a timestamp with a specific timezone, document the intended meaning.
    - guhidalg 5y ago
      +1, if your application cares about time then you should just tell your users all dates will be normalized to UTC and they are responsible for displaying them in a preferred time zone.
      - codebje 5y ago
        Time zone rules change relatively frequently - future dates may be better stored in the appropriate time zone, optionally along with the UTC offset at the time of recording, so you don't report incorrect information when the rules do change on you. Past dates should always be in UTC, for the same reason - timezone rule changes are sometimes even retroactive.
- hn_throwaway_99 5y ago
  1. Human readability 2. The ISO date format is "big endian" by default, which makes it trivial to chunk and compare just specific date parts. Want all events that occurred on the same (UTC) date? Just do dateField.substring(0, 10). Everything in the same year? dateField.substring(0, 4).
cyb_ 5y ago
Google published a similar recommendations: https://google.aip.dev/ https://google.aip.dev/
- akavel 5y ago
  AIPs are very deep and comprehensive, I would say "similar" is a diplomatic understatement here :)
hardwaresofton 5y ago
A few things I see rarely discussed that I often do: - Always use a response envelope - HTTP status is not the same as your application status. I always include a "status" field in the response envelope (OP recommends standardizing errors but I think standardize all of it) - Always have an unique error code for every error your API can return (they should also be URL safe ("some-thing-went-wrong"), basically an enum of these should exist somewhere, the more specific the better. - Offer OpenAPI whenever you can. - There is no such thing as a publicly exposed "private" API. If it can be hit (and is not protected), it eventually will be. - Do blackbox testing of your API, E2E tests are the most important kind of test you could have. - (controversial) build in special test/debug endpoints -- these help with blackbox testing.
- metadat 5y ago
  Are you able to elaborate on what is a response envelope? Edit: Nevermind, see [0]. It's the simple and intuitive concept of encasing the payload in an consistently structured object which includes metadata, e.g. {"Error": null, Data: ...} [0] https://stackoverflow.com/questions/9989135/when-in-my-rest-api-should-i-use-an-envelope-if-i-use-it-in-one-place-should-i https://stackoverflow.com/questions/9989135/when-in-my-rest-... Makes intuitive sense, as it's then easier to develop a nice, generic REST client.
  - hardwaresofton 5y ago
    Here's an example straight from code I've rewritten at least 5 times because I'm allergic to saving myself time: export enum ErrorCode { InvalidEntity = 1, NotSupported = 2, UnexpectedServerError = 3, InvalidRequest = 4, Validation = 5, // ... } export enum ResponseStatus { Success = "success", Error = "error", } export class ResponseEnvelope<T> { public readonly status: ResponseStatus = ResponseStatus.Success; public readonly data: T | Pruned<T> | null = null; public readonly error?: { code: ErrorCode, message: string, details: object | string, status: ResponseStatus, }; constructor(opts?: Partial<ResponseEnvelope<T>>) { if (opts) { if (opts.status) { this.status = opts.status; } if (opts.data) { this.data = opts.data; } if (opts.error) { this.error = opts.error; } } if (!this.data) { return; } // Prune if the thing is prunable this.data = prune(this.data); } } Hopefully this isn't too hard to grok, it's Typescript. BTW, If you're cringing at the number scheme for the ErrorCode enum, don't worry I am too. This is why I prefer to use strings rather than numbers, and it basically just ends up being stuff like "invalid-entity", etc.
deleted 5y ago
[deleted]
BulgarianIdiot 5y ago
Mostly common-sense things, but I can't wait for the community to stop trying to use PUT, PATCH, DELETE and the like. There's a reason that in 2022 web forms only support GET and POST (and implicitly, HEAD).
- jahewson 5y ago
  What is the reason?
  - runarberg 5y ago
    I don’t know what OP thinks is the reason, but the actual reason is CORS. Web Devs (such as me) have asked for e.g. DELETE as an acceptable formmethod (e.g. https://github.com/whatwg/html/issues/3577 https://github.com/whatwg/html/issues/3577) however WHATWG always pushes back citing security concerns such as CORS. I suspect this is not what OP had in mind since it is trivial to send a DELETE request with a simple JavaScript: <button onclick="fetch('/api/resource/42', { method: 'DELETE' })"> Delete </button>
    - egberts1 5y ago
      Right. If CORS can be weakened in any simple way with that HTRP-DELETE method, then your database could simply disappeared via HTTP-DELETE method. Besides, webmasters’ HTTP DELETE method is a different domain scoping issue than the web developers’ HTML/JavaScript FORM deleteThis row-entry approach. I marvel at designers trying to flatten the scoping/nesting of abstractions without factoring apart the disparate error and protocol handling.
- jimbob45 5y ago
  Are you saying dump DELETE because you should instead logically delete it with an IsDeleted column passed in via POST?
  - djbusby 5y ago
    Or POST to '/resource/id/delete'? That's my least favourite pattern
polishdude20 5y ago
We don't use PATCH but use a PUT for partial objects. We have validator code at every endpoint and we validate both creates and updates. When a PUT comes in, the validator knows what can and can't be changed. Depending on your role, the validator lets you change certain things can be updated as well. A PATCH would need these too and now you have more code to deal with. Also, it requires the developer to now worry that they have all the fields for a complete object or not.
- BulgarianIdiot 5y ago
  Based on that description, you may be using PUT in conflict with its semantics (namely, idempotent way to replace an entire resource). This is one reason why I don't bother with these methods and stick to GET and POST.
  - Supermancho 5y ago
    > I don't bother with these methods and stick to GET and POST. Most people don't bother with them. If you need caching or want to be able to manipulate params in the browser/link to resources, use GET. This idea that you need additional verbs for web services is a classic case of in-theory vs in-practice. Introducing non-trivial complication for very tiny benefit is a strange tradeoff.
    - deleted 5y ago
      [deleted]
  - bcrosby95 5y ago
    PUT can be useful if you have a client that tries to handle errors. For example, our mobile application will automatically retry a PUT.
wokwokwok 5y ago
I’m gonna say it: Many rest apis are lazy and developer friendly, not consumer friendly. If you have related resources, let’s say, product and product options as two distinct endpoints: - /api/product - /api/options Then, and I want to be clear here, it is impossible for a client to perform an atomic operation on multiple distinct objects types. Let’s say the client needs to add a product with a single option or fail. You can create a product. You can create an option. You can add an option to a product. …but, at some point in time, a product will exist with no options on it. This kind of “pure” rest api is simply convenient to the developer because they push the problem of object consistency to the client. …but that’s not a client concern. If a product needs to be associated with an option on creation, then your api should offer that as an endpoint. It doesn’t meet the “standard” for a rest api? Too bad. Write APIs that do what the customer / consumer needs not lazy APIs that just make your life easier. I’ve worked with a lot of APIs and you know what I do not give the tiniest moment of care about? Consistency. I do. Not. Care. If POST /api/foo creates an object, or if the endpoint is /api/foo/create. Messy? Inconsistent? I don’t care. Just put it in the documentation and I’ll find it and use it. …but if your api forces object relational consistency onto me as an api consumer, you have no idea how much pain and hassle you’ve caused me having to implement my own set of fake transactions over your stupid api. Please, write APIs for consumers, not according to “the rules” of rest APIs. …and provide documentation. :)
- 1123581321 5y ago
  Agree that the tendency exists but don't see the tradeoff between transactional and messy in this example--you should be able to create a new product with options [...] in the same request, perhaps using /options to ascertain what's available before posting to /products, depending on the situation.
  - shoo 5y ago
    maybe a more illustrative example of where this can be messy is where a customer has placed an order for a bundle of products A B and C, and there's no "order bundle" API that can do that operation atomically, instead the client code trying to place the order has to call a bunch of random stateful create-product APIs to attempt to construct each product in the bundle, and deal with the mess if suddenly some bundle items error while others succeed. bonus points if some of the create-product APIs are not idempotent. "congratulations, we've provisioned the bundle A, C, C, C that you ordered! sorry about B!"
kumarvvr 5y ago
Sometimes, I feel that we ought to have a simple protocol, on top of HTTP, to simply do remote procedure calls and throw out all this HTTP verbs crap. Every request is a http POST, with or without any body and the data transfer is in binary. So that objects can be passed back and forth between client and server. Sure, there is gRPC, but it requires another API specification (the proto files). There I said it. HTTP Verbs constrained REST APIS are the worst thing ever. I hate them. They introduce un-necessary complexity, un-necessary granularity and they almost always stray away from the "REST principles". To hell with "Hypermedia" stuff. I find it such a joy to program in server rendered pages. No cognitive overhead of thinking in "REST". But, of course, all this is only where the client and server are developed by the same person / company. For publishing data and creating API for third party use, we have no serious, better alternative to REST.
- bruhboribhe 5y ago
  We do, it's called JSON-RPC.
  - kumarvvr 5y ago
    Whoa, completely missed that boat. Is it in active use? Wikipedia page says last spec update was in 2010. No other details online. I could not find any specific implementations.
    - bruhboribhe 5y ago
      Active use is really irrelevant if you only plan on using it inside a company, because you can implement a client and server within 30 mins to an hour, no external tools needed. The spec is clear and readable. It's excellent. We use it with a TypeScript codebase and just share the interfaces for the services in a monorepo. The spec is so simple it doesn't really need an update. If you want a more advanced implementation with type inference for Typescript you can use this: https://github.com/shekohex/jsonrpc-ts https://github.com/shekohex/jsonrpc-ts I'd still recommend implementing your own, my own implementation is based off the one above. The only caveat compared to GRPC is you lose out on field validation due to it not being protobuf/and the obvious json decode overhead Edit: if you want to know some current users of it, I believe the Ethereum protocol uses it heavily
dimitrios1 5y ago
For #2, better yet, prefer RFC 3339.
- rbluethl 5y ago
  Author here. I read that a couple of times already. RFC 3339 is what you should use - I should probably change that (it's what I actually meant). :) Thanks for pointing that out.
ramraj07 5y ago
To resurface the debate, is including related endpoint urls pre-filled in a “misc” field a good idea?
solatic 5y ago
a) Use standardized error codes, not standardized error messages. Clients are responsible for internationalization, which includes presenting error messages in the user's language. If you document a set of error codes as an enum, the client can present a user-friendly error message in the user's language based on the error code. If there are dynamic parts of the error message, i.e. "404: There is no user with ID 123456 in the system", then the user ID should be extracted into the error response body, so that it can be provided correctly to the user in the user's language. b) Pagination is the devil. The state of the server can change while the user is paginating, leading to fragile clients. Don't paginate your API. If you think you have a need to paginate, have one API call return a list of IDs, and have a separate API call return a list of resources for a given list of IDs, where the second API call accepts some maximum number of IDs as a query parameter. This ensures consistency from one API call to the next.
- CrimsonRain 5y ago
  b -> What happens if the "list of IDs" is 10k or 100k or 1M+?
  - solatic 5y ago
    There's typically business-logic ways to require a range to be specified, and then you specify a maximum range. For example, if the resource in question is audit events, you require the user to specify a time range, up to a maximum of a day, or seven days, or whatever performance / budget constraints allow for.
    - akvadrako 5y ago
      That's just like paging by day or whatever.
    - CrimsonRain 5y ago
      So if I specify a range and something changes between in that range while I'm interacting with those, sounds like same problem is back? Just same issue; different package :) If I do a search, a paginated search result can have an ID so I can paginate between the data without a new data messing up my pagination. But for normal entities, simple pagination is (mostly) more than enough. The solution you are describing is overkill and almost no benefit at all.
ceeker 5y ago
Can someone share how they handle versioning in their API when it comes to data model changes? For example `POST /users` now takes a required field `avatar_url` but it was not part of `v1`. Since this field is validated in the DB, merely having `v1` `v2` distinction at the API layer is not sufficient. So I was thinking we will have to either 1) disable DB validations and rely on app validations or 2) run two separate systems (e.g., one DB per version) and let people 'upgrade' to the new version (once you upgrade you cannot go back). Even though people refer to Stripe's API versioning blog, I don't recall any mention of actual data model changes and how it is actually managed
- ComputerGuru 5y ago
  I upvoted because I'm curious to hear what others are doing. We typically make sure to only make such breaking changes where either the now-required value or a sane filler value could be used. If it's the same API for the same purpose, it's usually not a stretch to assume the values for a new field are derived from some combination of an old field or else are primitive components of an old field such that they can be deduced and stubbed in your transition layer (or calculated/looked-up/whatever one-by-one as part of a bulk migration script during the transition). If your v2 is so drastic of a breaking upgrade that it bears no relationship to v1, I imagine your SOL and probably should have thought out your v1 or your v1-to-v2 story better, if only for the sake of the poor devs using your API (and you probably need separate tables at that point). For other fields like your example of `avatar_url` I would use a placeholder avatar for all legacy users (the grey anonymous snowman profile comes to mind).
  - ceeker 5y ago
    Thanks. This is a fair point. I made up the example only to illustrate the idea. Since Stripe is considered some sort of benchmark here I was curious to see how they tackle all the learnings they will have over time...I feel it is very hard to think through all the future cases especially when you are just about starting out with your product. For example, in financial services and insurance, regs change and what data we need to collect change and sometimes their dependency will change. I am curious what's companies that have grown substantially had to do to their APIs.
CipherThrowaway 5y ago
> 14. Use pagination Generally agree with everything else but strongly disagree with pagination envelopes and offset pagination. Allow keyset pagination using ordering and limiting query parameters.
tlarkworthy 5y ago
A good resource, some others I consult with https://cloud.google.com/apis/design https://cloud.google.com/apis/design https://opensource.zalando.com/restful-api-guidelines/ https://opensource.zalando.com/restful-api-guidelines/
trinovantes 5y ago
Do you guys use plural or singular terms in your API endpoints or both? /books /books/:id /book /book/:id It gets harder to keep consistent when there are a lot of nouns that have the same singular/plural form like "clothing"
- imnitishng 5y ago
  Both /books /book/:id
- systemvoltage 5y ago
  /clothes/:clothing_id The reason for using plural is because without the `:clothing_id`, `/clothes` endpoint would return a collection of clothing.
- akvadrako 5y ago
  I always prefer singular just because it's easier to spell if you know the type. It's stupid to have /geese return a Goose. If the APIs were in Esperanto it would be different.
- barefeg 5y ago
  We use the form that represents a collection of objects. For example books and clothing both refer to a collection of things so I take them as valid forms
sandreas 5y ago
While I agree with the most aspects of this very good blog post, there is a minor detail that I would like to note: While pagination is important, there is another possibility of pure size - it is using cursors, like mentioned in the JSONAPI specification[1] (containing many of the hints in the topics' post) and in this blog post[1] [1] https://jsonapi.org/format/#fetching-pagination https://jsonapi.org/format/#fetching-pagination [2] https://dev.to/jackmarchant/offset-and-cursor-pagination-explained-b89 https://dev.to/jackmarchant/offset-and-cursor-pagination-exp...
- rbluethl 5y ago
  Author here. Thanks for the nice words - I appreciate it. I've read that a couple of times in the comments already. Definitely something worth considering, Stripe does this as well. Thanks for your comment.
gigatexal 5y ago
lol this could have been a single line blog post: “Do whatever stripe does with their APIs.” I kid. There’s some good stuff in here.
- rbluethl 5y ago
  Author here. Haha! You are right. I admire Stripe's API and there's absolutely a lot of things in it that are my "benchmark" when it comes to good APIs. However, they return UNIX timestamps, for example, which I don't like that much (as stated in the post). They also use POST for updates, because, if I recall correctly, PUT/PATCH wasn't a thing yet when Stripe was built initially. Thanks for the comment.
janaagaard 5y ago
My recommendation: Use PUT everywhere instead of POST. PUT has to be idempotent, so if the request fails, the client can simply post it again. This solves issue like worrying about creating a second copy of an item if a POST timed out. Using PUT to create elements means that the client has to supply the ID, but this is easily solved by using GUIDs as IDs. Most languages have a package for create a unique GUIDs.
- systemvoltage 5y ago
  What are the use cases where idempotency is not needed or even harmful?
- gls2ro 5y ago
  I don't think we should strive to remove non-idempotent cases. If something is not idempotent does not mean it is bad. It just means that request should be handled differently. In your example (and I ask this as I remained confused after also reading SO): Let's say that you need the client to provide the ID in the request body. In this case, how is using PUT when creating a new resource idempotent if the ID should be unique and you have a constraint on the DB level for example? What happens when the second call will be made with the same ID? If I execute the following sequence: Call 1: PUT resource + request.body {id: 1} => a new record is created so the state of the system changes Call 2: PUT resource + request.body {id: 1} => no record is created, maybe the existing one is updated IMO this is not idempotent nor should it be. Creating a new resource is not an idempotent operation. I also don't like that depending on the state of the DB two requests with the same attributes will have different impacts on the system. In my mind as a consumer of an API it is simpler: POST it creates a new record so I know what to expect, PUT updates an existing one.
  - dropofwill 5y ago
    Upserts are more powerful because the client can always generate a new uuid to get the POST behavior you desire, but the reverse is not the case: there is no straightforward way to safely retry timeouts, partial success, side effects, etc. You say the same request has a different impact on the system, but what that means is the system converges to the requested state. Maybe this is overkill if your retry strategy is to have a user in the loop, but I don't see how it's simpler for the client even without any retry behavior (if/else new/reuse uuid vs if/else put/post).
softfalcon 5y ago
Reading this makes me feel like a take all the magic GraphQL does for granted
egberts1 5y ago
For those in Security Theatre of HTTP APIs, I found this also useful: https://github.com/yosriady/api-development-tools https://github.com/yosriady/api-development-tools
tommiegannert 5y ago
> 6. Accept API key authentication ... using a custom HTTP header (such as Api-Key). Wouldn't a bearer token [1] make more sense? Defined for use by OAuth2, but I don't see why it couldn't be the general mechanism for... bearer tokens. > 11. Return created resources upon POST Especially important if your database+caching layers use eventual consistency. [1] https://datatracker.ietf.org/doc/html/rfc6750 https://datatracker.ietf.org/doc/html/rfc6750
- jamietanna 5y ago
  Authentication and Authorization are two subtly different things. In this case, you may want an API key (Authentication) to be required to ensure things like rate limiting is enforced, but then want proof that the call is operating on a user, or is a machine-to-machine interaction which OAuth2 Bearer tokens work nicely for (Authorization)
AtNightWeCode 5y ago
I agree with most things. I really hate when APIs use different api-key headers depending on the role of the consumer. It is very annoying when you get dates that are not in the ISO format. There are reasons to not use UTC everywhere. One should make that decision. The reason why many APIs use POST instead of DELETE is that POST is said to be more secure. Many APIs that I use do not have neither of PATCH, PUT or DELETE. An order for instance will have an order status resource that one just keeps adding status entities to. In general, well-designed systems minimize the need for changing data.
remoquete 5y ago
Extra points: document the API / have an API technical writer in the team. Part of a good API design is documenting it. If you use an API specification format, such as OpenAPI, design and documentation overlap nicely.
- BossingAround 5y ago
  Ah yes, the classic: POST /document/:id?params Creates a document with parameters
  - remoquete 5y ago
    More like API guides, a complete reference of the parameters, etc. :-)
zgiber 5y ago
Please don’t specify ISO8601 as the date format. That standard embodies dozens of formats. Use one specific format instead like RFC3339.
bmn__ 5y ago
The author has a poor understanding of HTTP and adjacent standards. I find it is so insufficient that he should not dispense advice yet, he must learn much more, especially about the essential yet completely unmentioned parts of a Web system: media types, hyperlinks, Link relations, cacheability. Critique: 2. ISO 8601 is a shit show of a standard. Last time I surveyed, there was not a single compliant implementation in the world. Recommend "Date and Time on the Internet: Timestamps" <http://rfc-editor.org/rfc/rfc3339 http://rfc-editor.org/rfc/rfc3339> instead. This standard is public/not encumbered by stupid copy fees and is restricted to a much more reasonable profile that can actually be implemented fully and in an interoperable fashion. 4. Use `OPTIONS ` instead. <http://rfc-editor.org/rfc/rfc7231#section-4.3.7 http://rfc-editor.org/rfc/rfc7231#section-4.3.7> 5. This goes against the design of the Web. Do not version URIs. If the representation changes and is not backward compatible, change the media type instead, e.g. by adding a profile <http://rfc-editor.org/rfc/rfc6906 http://rfc-editor.org/rfc/rfc6906>. You can serve multiple representations on the same URI and vary on the request headers (e.g. Accept-). 6. HTTP already contains a customisable mechanism. Use the standard header <http://rfc-editor.org/rfc/rfc7235#section-4.2 http://rfc-editor.org/rfc/rfc7235#section-4.2>, not the custom header `Api-Key` which is not interoperable. 7. "Don't use too many [HTTP status codes]": why? This opinion is not backed up with an explanation. The correct advice is: use as many status codes as arise from the requirements. `422 Unprocessable Entity` is useful in nearly every Web system. 8. What does "reasonable" mean? This lacks an explanation. 10. Use application/problem+json <http://rfc-editor.org/rfc/rfc7807 http://rfc-editor.org/rfc/rfc7807> instead. 11. "It's a good idea to return the created resource after creating it with a POST request": why? This opinion is not backed up with an explanation. If you follow this advice, the user agent cannot programmatically distinguish between a representation reporting on the requested action's status, and the representation of the newly created resource itself. Use the Content-Location header <http://rfc-editor.org/rfc/rfc7231#section-3.1.4.2 http://rfc-editor.org/rfc/rfc7231#section-3.1.4.2> to make that possible, use the Prefer header <http://rfc-editor.org/rfc/rfc7240 http://rfc-editor.org/rfc/rfc7240> to give the user agent some control over which representation to respond with. 12. "Prefer PATCH over PUT": disagree, ideally you offer both since there is a trade-off involved here. The downsides of PATCH are not mentioned: the method is not (required to be) idempotent meaning it becomes moderately tricky to keep track of state, and the client is required to implement some diff operation according to the semantics of the accepted media type in the Accept-Patch header which can be difficult to get right. 13. Missed opportunity to advertise the OPTIONS method and related Accept-Post <https://datatracker.ietf.org/doc/html/draft-wilde-accept-post https://datatracker.ietf.org/doc/html/draft-wilde-accept-pos...> / Accept-Patch <http://rfc-editor.org/rfc/rfc5789#section-3.1 http://rfc-editor.org/rfc/rfc5789#section-3.1> headers. A representation of a specific media type can self describe with the "type" Link relation <http://rfc-editor.org/rfc/rfc6903.html#section-6 http://rfc-editor.org/rfc/rfc6903.html#section-6>. 14. Don't mix data with metadata. Use the Range header <http://rfc-editor.org/rfc/rfc7233 http://rfc-editor.org/rfc/rfc7233> and next/prev Link relations <https://webconcepts.info/concepts/link-relation/next https://webconcepts.info/concepts/link-relation/next> instead. 15. Instead of complicating both server and client, the better idea is to simply link to related resources. We are not in the 1990s any more. A well written Web server offers HTTP persistent connections, HTTP pipelining, all of which make round-trips cheap or in the case of HTTP/2 server push, even unnecessary. Benchmark this. 1. + 9. betrays a weird obsession with naming. The advice is not wrong, but it shifts attention away from the genuinely useful areas that benefit much more from a careful design: the media types, the hyperlinks between and other hypermedia mechanisms for traversing resources (forms, URI templates). If an inexperienced programmer follows the advice from the article, he will the idea to spend lots of time mapping out resources in the identifier space and methods and possible responses for documentation purposes. This is both a waste of time because the single entry point is enough and the rest of the resources can be reached by querying the server via OPTIONS and traversing hyperlinks etc., and dangerously brittle because of strong coupling between the server and the client.
loup-vaillant 5y ago
For a second there I thought this was about non-web APIs: https://caseymuratori.com/blog_0024 https://caseymuratori.com/blog_0024
- spacechild1 5y ago
  Side note: I find it quite annoying that web devs have hijacked the term “API“ as a synonyme for “web API“, to the point where many devs are not even aware of the original broader meaning. Nevertheless, as a (largely) non-web dev I found the article interesting and useful.
11235813213455 5y ago
Something I wonder is how to design search/list endpoints where the query can be long (a list of IDs, ex: /users?ids=123e4567-e89b-12d3-a456-426614174000,123e4567-e89b-12d3-a456-426614174001,123e4567-e89b-12d3-a456-426614174002,...), so long that it can exceed the url max length (2048), after 50 UUIDs, you can quickly exceed that length, so GET is not ideal, so which method, SEARCH with a body? POST with a body?
- flowerbreeze 5y ago
  I would go with POST with a body in that case, where I interpret is as a "new search item" and use GET to scan through results, if there are many results available. I don't think I've needed something like this more than once or twice though. From users perspective, asking information on specific 50 items at once is not something commonly done.
  - 11235813213455 5y ago
    it's used for an export feature, where we join data from the client-side, not ideal, but for now it's a good compromise in terms of complexity for the API and client
- daniel-thompson 5y ago
  There was a new RFC published a few months ago to address this use case. It defines a new HTTP method QUERY, which is defined to be safe & idempotent like GET and explicitly allows a request body like POST. See https://www.ietf.org/id/draft-ietf-httpbis-safe-method-w-body-02.html https://www.ietf.org/id/draft-ietf-httpbis-safe-method-w-bod...
  - 11235813213455 5y ago
    great, thanks for the info!
- athrowaway3z 5y ago
  The type signature: ` list of uuids -> list of results ` is usually bad design to begin with.
ChrisMarshallNY 5y ago
It’s a good posting. I do many of these things, but not all. I’ve been designing SDKs for a long time; APIs, for a somewhat shorter time. I don’t like “pure” RESTful APIs, because I feel as if they are “human-hostile.” I tend to follow the patterns set by companies like Google, where the stimulus is a fairly straightforward URI (with parameters, as opposed to a block of XML or JSON —if possible, as I know that we often still need to send big data as transaction data, which can be a pain, depending on the method), and the response is a block of structured data. That also makes it a lot easier for API clients to build requests, and allows the server to “genericize” the processing of requests. > 10. Use standardized error responses Is something I do, along with a text header payload, describing internal information about the error. It is my experience that this text payload is never transferred, and I am forced to send a text response via the body, if I want to know internal information. That can be a pain, as it often breaks the transaction, so I have to play with the server and client, frequently, using a cloned server instance. I should note that my forte is not backend development, so the chances are good that I am not familiar with all the tools at my disposal.
- thatwasunusual 5y ago
  > I tend to follow the patterns [...] Can you elaborate on this? Give examples?
  - ChrisMarshallNY 5y ago
    I don’t feel like doing that in a comment, but feel free to check out some of my work. The BAOBAB server[0] is one of my more recent examples. I’m using a modified version as the backend for the app I’m developing, now. It works great. I should note that the BASALT layer uses “plugins,” that can be extended to include “pure” REST APIs. I just haven’t found these practical, for my purposes. [0] https://riftvalleysoftware.com/work/open-source-projects/#baobab https://riftvalleysoftware.com/work/open-source-projects/#ba...
kuon 5y ago
ISO 8601 is bad, use RFC 3339. A lot of implementation are actually based on an ISO draft which changed in final release. But most dev do not have access to the spec. For example, timezone can only be specified as offset in the standard, while implementations accept name. Globally avoid ISO for software, it is non free crap. Also, do not use those standard for dates with timezone in the future. Use wall time/date and location. As timezone can change way more often than you think.
- idoubtit 5y ago
  My own experience is that (unix) timestamps are much less error-prone than textual representations like ISO 8601 and such. A field like `update_time_seconds` is clear and easy to convert into any representation. This is what the Google API Improvement Proposals recommends in most cases, though civil timestamps are also described. https://google.aip.dev/142 https://google.aip.dev/142 Of course, when preparing queries against the API, you may need a helper to build valid timestamps. But this is mostly relevant to the discovery phase which isn't automated. And textual dates also have drawbacks, for instance the need to encode it when used a an URL parameter.
  - kuon 5y ago
    I agree, for timestamp, I often use ms or sec offsets. But I do not dare say it because I pass for an old fool.
- grishka 5y ago
  Why use string-based timestamps at all? Use unixtime. It's much easier to parse and it literally can't be malformed.
  - arendtio 5y ago
    Sometimes timezones or the like change. So for future dates, those textual represations are probably better. Example: in November you create an appointment in 6 months at 15h, but then your government decides, to not use summer time next year. If you use timestamps your appointment will be wrong by one hour (humans tend to keep the 15h). In general, I am a huge fan of timestamps, but I think it is good to know where they have their limits.
perlwle 5y ago
https://www.vinaysahni.com/best-practices-for-a-pragmatic-restful-api https://www.vinaysahni.com/best-practices-for-a-pragmatic-re... Helped me get started with API design in my early career. Learning from other existing APIs helps too. such as stripe, github, shopify. Any others? Something that We do at my current job: * set a standard and stick with it. tweak it if needed. we even have naming standard on some of the json key. for example, use XXX_count for counting. when it doesn't make sense, use total_XXX. * Document your API, we use postman and code review API changes too.
myshkin5 5y ago
> 5xx for internal errors (these should be avoided at all costs) An anti pattern I’ve often seen has devs avoiding 5xx errors in bizarre ways. I would change the above to make to have monitoring in place to address 5xx errors. By all means, let your code throw a 500 if things go off the rails.
- rbluethl 5y ago
  Author here. I generally agree - IF something goes terribly wrong within the application, a 500 is definitely the way to go. The thing is - these just shouldn't happen that often. Because in that case we're having a bigger problem. :D For example I have never seen Stripe return a HTTP 500 from their API. Thing just works.
rbluethl 5y ago
Hey friends! Author (@rbluethl) here. :) I just randomly noticed that my post made it to the front page! Thank you all for your comments.

The PUT vs PATCH is debatable in different levels. One simple issue is how to resolve complex merges in a PATCH. For example if we patch a key that contains a list, what will be the expected result?

ajcp 5y ago
Even beyond PUT vs PATCH I found this statement to be rather naive: "From my experience, there barely exist any use cases in practice where a full update on a resource would make sense" Just in finance I can think of *dozens* of use cases around invoicing and purchasing *alone*. A lot of times thousands of resources may need just one field "corrected", but hundreds others "reset" to ensure the proper retriggering of routing, workflows, and allocations. That the resources need to exist as originals is incredibly important for these kinds of things.

pokoleo 5y ago

The error messages could be better yet. The example uses a different code per issue, for instance: "user/email_required". Most integrators will build their UI to highlight the input fields that contain an error. Making them parse the `code` field (or special-case each possible code) is pretty toilsome. // from blog post { "code": "user/email_required", "message": "The parameter [email] is required." } Make it parseable: // improved { "message": "An email is required.", "error": "missing_parameter", "parameter": "user.email" } In addition, I: * rewrote `message` to be an acceptable error message displayed to (non-technical) end-users * moved message to be the first field: some developer tools will truncate the JSON response body when presenting it in the stack trace. --- As an added bonus, structured data allows you to analyze `error` frequencies and improve frontend validation or write better error messages: https://twitter.com/VicVijayakumar/status/1495092161821429761 https://twitter.com/VicVijayakumar/status/149509216182142976...

smoyer 5y ago
It also doesn't hurt to repeat the HTTP status code in the JSON body - when you receive a response, the status code and entity body are coupled but even if the server logs the status code, they're often decoupled in the logging system - having both in one log entry is way easier!
moltar 5y ago
Agreed. But rather than reinvent, let’s just use JSON API standard? https://jsonapi.org/format/ https://jsonapi.org/format/ (Scroll to the very bottom)
- pokoleo 5y ago
  The article is about REST API design, not JSON APIs! It’s a whole different ballpark.
pan69 5y ago
It might be to formal for your use-case, but there is a standard defined for error responses in RFC 7807: https://datatracker.ietf.org/doc/html/rfc7807 https://datatracker.ietf.org/doc/html/rfc7807
- aeontech 5y ago
  Wow, I had never seen an API with errors at this level of detail… I feel lucky when they at least use sane status codes instead of always giving back 200 and a maybe-json-maybe-plaintext-maybe-empty body… I’d love to hear from anyone who has encountered APIs in the wild that actually implement this standard!
  - JimDabell 5y ago
    I use Problem Details in most APIs I build. It’s as simple to generate a Problem Detail as it is to generate ad-hoc errors, but you can re-use code.
  - abdusco 5y ago

knighthack 5y ago

A good checklist of things to mind when designing APIs. Sometimes you forget the good things.

rbluethl 5y ago
Thanks for the kind words! Appreciate you reading it.

pan69 5y ago

Some nice tips in here. However, tip 15, I strongly disagree with: > 15. Allow expanding resources I would suggest the opposite. A REST API should not return nested resources at all. Instead, and to stay with the example provided on the website, to obtain the "orders", the /users/:id/orders endpoint should be called. It might be tempting to return nested resources, because clients would only have to make a single call.Technically this is true but once the domain of your API starts to grow, you will find that the interface will become increasingly muddled. The suggestion provided (use query parameters) is basically a work-around. If you want to offer this to your clients, front your REST API with a GraphQL API instead. It is literally the problem that GraphQL solves. Keep your REST API clean and dumb and focused around CRUD.

metadat 5y ago
Dear PAM69, this is great advice if you want to end up with a slow-to-load, low-performing web application that your customers complain about and hate using. But at least it'll adhere to a specific notion of architectural "purity", right? /s Anytime clients need to make 15 async calls before the UI can be displayed, you're headed up the creek. Generally speaking, this is an anti-pattern. There are exceptions, but they're not the rule. It's better to weigh the tradeoffs in any given situation and make a decision about bundling sub-resource references based on what you're optimizing for. A few quick examples: * Dev speed: Unbundled * Quick-loading UI: Bundled * Sub-resources are computationally-expensive to query? Unbundled This has been my experience consistently throughout 20 years of web-tier development.
- BulgarianIdiot 5y ago
  Well they said use GraphQL, which is a solid way to avoid REST's clumsiness once and for all.
  - nerdponx 5y ago
    Sort of, but is it any better if the GraphQL layer still has to make 15 requests in order to serve a single useful response?

drdrey 5y ago

Why would one prefer ISO 8601 dates over POSIX timestamps?

paulryanrogers 5y ago
Human readability most likely
mulmboy 5y ago
They can include a local timezone. Sometimes there's a big difference between 12am in UTC+0 and 3am in UTC+3 despite representing the same instant in time.
- inopinatus 5y ago
  True enough, but I would still recommend that API responses normalize to UTC (Z suffix) in the general case and document as much, and if actually returning a timestamp with a specific timezone, document the intended meaning.
  - guhidalg 5y ago
    +1, if your application cares about time then you should just tell your users all dates will be normalized to UTC and they are responsible for displaying them in a preferred time zone.
    - codebje 5y ago
      Time zone rules change relatively frequently - future dates may be better stored in the appropriate time zone, optionally along with the UTC offset at the time of recording, so you don't report incorrect information when the rules do change on you. Past dates should always be in UTC, for the same reason - timezone rule changes are sometimes even retroactive.
hn_throwaway_99 5y ago
1. Human readability 2. The ISO date format is "big endian" by default, which makes it trivial to chunk and compare just specific date parts. Want all events that occurred on the same (UTC) date? Just do dateField.substring(0, 10). Everything in the same year? dateField.substring(0, 4).

cyb_ 5y ago

Google published a similar recommendations: https://google.aip.dev/ https://google.aip.dev/

akavel 5y ago
AIPs are very deep and comprehensive, I would say "similar" is a diplomatic understatement here :)

hardwaresofton 5y ago

A few things I see rarely discussed that I often do: - Always use a response envelope - HTTP status is not the same as your application status. I always include a "status" field in the response envelope (OP recommends standardizing errors but I think standardize all of it) - Always have an unique error code for every error your API can return (they should also be URL safe ("some-thing-went-wrong"), basically an enum of these should exist somewhere, the more specific the better. - Offer OpenAPI whenever you can. - There is no such thing as a publicly exposed "private" API. If it can be hit (and is not protected), it eventually will be. - Do blackbox testing of your API, E2E tests are the most important kind of test you could have. - (controversial) build in special test/debug endpoints -- these help with blackbox testing.

metadat 5y ago
Are you able to elaborate on what is a response envelope? Edit: Nevermind, see [0]. It's the simple and intuitive concept of encasing the payload in an consistently structured object which includes metadata, e.g. {"Error": null, Data: ...} [0] https://stackoverflow.com/questions/9989135/when-in-my-rest-api-should-i-use-an-envelope-if-i-use-it-in-one-place-should-i https://stackoverflow.com/questions/9989135/when-in-my-rest-... Makes intuitive sense, as it's then easier to develop a nice, generic REST client.
- hardwaresofton 5y ago
  Here's an example straight from code I've rewritten at least 5 times because I'm allergic to saving myself time: export enum ErrorCode { InvalidEntity = 1, NotSupported = 2, UnexpectedServerError = 3, InvalidRequest = 4, Validation = 5, // ... } export enum ResponseStatus { Success = "success", Error = "error", } export class ResponseEnvelope<T> { public readonly status: ResponseStatus = ResponseStatus.Success; public readonly data: T | Pruned<T> | null = null; public readonly error?: { code: ErrorCode, message: string, details: object | string, status: ResponseStatus, }; constructor(opts?: Partial<ResponseEnvelope<T>>) { if (opts) { if (opts.status) { this.status = opts.status; } if (opts.data) { this.data = opts.data; } if (opts.error) { this.error = opts.error; } } if (!this.data) { return; } // Prune if the thing is prunable this.data = prune(this.data); } } Hopefully this isn't too hard to grok, it's Typescript. BTW, If you're cringing at the number scheme for the ErrorCode enum, don't worry I am too. This is why I prefer to use strings rather than numbers, and it basically just ends up being stuff like "invalid-entity", etc.

deleted 5y ago

[deleted]

BulgarianIdiot 5y ago

Mostly common-sense things, but I can't wait for the community to stop trying to use PUT, PATCH, DELETE and the like. There's a reason that in 2022 web forms only support GET and POST (and implicitly, HEAD).

jahewson 5y ago
What is the reason?
- runarberg 5y ago
  I don’t know what OP thinks is the reason, but the actual reason is CORS. Web Devs (such as me) have asked for e.g. DELETE as an acceptable formmethod (e.g. https://github.com/whatwg/html/issues/3577 https://github.com/whatwg/html/issues/3577) however WHATWG always pushes back citing security concerns such as CORS. I suspect this is not what OP had in mind since it is trivial to send a DELETE request with a simple JavaScript: <button onclick="fetch('/api/resource/42', { method: 'DELETE' })"> Delete </button>
  - egberts1 5y ago
    Right. If CORS can be weakened in any simple way with that HTRP-DELETE method, then your database could simply disappeared via HTTP-DELETE method. Besides, webmasters’ HTTP DELETE method is a different domain scoping issue than the web developers’ HTML/JavaScript FORM deleteThis row-entry approach. I marvel at designers trying to flatten the scoping/nesting of abstractions without factoring apart the disparate error and protocol handling.
jimbob45 5y ago
Are you saying dump DELETE because you should instead logically delete it with an IsDeleted column passed in via POST?
- djbusby 5y ago
  Or POST to '/resource/id/delete'? That's my least favourite pattern

polishdude20 5y ago

We don't use PATCH but use a PUT for partial objects. We have validator code at every endpoint and we validate both creates and updates. When a PUT comes in, the validator knows what can and can't be changed. Depending on your role, the validator lets you change certain things can be updated as well. A PATCH would need these too and now you have more code to deal with. Also, it requires the developer to now worry that they have all the fields for a complete object or not.

BulgarianIdiot 5y ago
Based on that description, you may be using PUT in conflict with its semantics (namely, idempotent way to replace an entire resource). This is one reason why I don't bother with these methods and stick to GET and POST.
- Supermancho 5y ago
  > I don't bother with these methods and stick to GET and POST. Most people don't bother with them. If you need caching or want to be able to manipulate params in the browser/link to resources, use GET. This idea that you need additional verbs for web services is a classic case of in-theory vs in-practice. Introducing non-trivial complication for very tiny benefit is a strange tradeoff.
  - deleted 5y ago
    [deleted]
- bcrosby95 5y ago
  PUT can be useful if you have a client that tries to handle errors. For example, our mobile application will automatically retry a PUT.

wokwokwok 5y ago

I’m gonna say it: Many rest apis are lazy and developer friendly, not consumer friendly. If you have related resources, let’s say, product and product options as two distinct endpoints: - /api/product - /api/options Then, and I want to be clear here, it is impossible for a client to perform an atomic operation on multiple distinct objects types. Let’s say the client needs to add a product with a single option or fail. You can create a product. You can create an option. You can add an option to a product. …but, at some point in time, a product will exist with no options on it. This kind of “pure” rest api is simply convenient to the developer because they push the problem of object consistency to the client. …but that’s not a client concern. If a product needs to be associated with an option on creation, then your api should offer that as an endpoint. It doesn’t meet the “standard” for a rest api? Too bad. Write APIs that do what the customer / consumer needs not lazy APIs that just make your life easier. I’ve worked with a lot of APIs and you know what I do not give the tiniest moment of care about? Consistency. I do. Not. Care. If POST /api/foo creates an object, or if the endpoint is /api/foo/create. Messy? Inconsistent? I don’t care. Just put it in the documentation and I’ll find it and use it. …but if your api forces object relational consistency onto me as an api consumer, you have no idea how much pain and hassle you’ve caused me having to implement my own set of fake transactions over your stupid api. Please, write APIs for consumers, not according to “the rules” of rest APIs. …and provide documentation. :)

1123581321 5y ago
Agree that the tendency exists but don't see the tradeoff between transactional and messy in this example--you should be able to create a new product with options [...] in the same request, perhaps using /options to ascertain what's available before posting to /products, depending on the situation.
- shoo 5y ago
  maybe a more illustrative example of where this can be messy is where a customer has placed an order for a bundle of products A B and C, and there's no "order bundle" API that can do that operation atomically, instead the client code trying to place the order has to call a bunch of random stateful create-product APIs to attempt to construct each product in the bundle, and deal with the mess if suddenly some bundle items error while others succeed. bonus points if some of the create-product APIs are not idempotent. "congratulations, we've provisioned the bundle A, C, C, C that you ordered! sorry about B!"

kumarvvr 5y ago

Sometimes, I feel that we ought to have a simple protocol, on top of HTTP, to simply do remote procedure calls and throw out all this HTTP verbs crap. Every request is a http POST, with or without any body and the data transfer is in binary. So that objects can be passed back and forth between client and server. Sure, there is gRPC, but it requires another API specification (the proto files). There I said it. HTTP Verbs constrained REST APIS are the worst thing ever. I hate them. They introduce un-necessary complexity, un-necessary granularity and they almost always stray away from the "REST principles". To hell with "Hypermedia" stuff. I find it such a joy to program in server rendered pages. No cognitive overhead of thinking in "REST". But, of course, all this is only where the client and server are developed by the same person / company. For publishing data and creating API for third party use, we have no serious, better alternative to REST.

bruhboribhe 5y ago
We do, it's called JSON-RPC.
- kumarvvr 5y ago
  Whoa, completely missed that boat. Is it in active use? Wikipedia page says last spec update was in 2010. No other details online. I could not find any specific implementations.
  - bruhboribhe 5y ago
    Active use is really irrelevant if you only plan on using it inside a company, because you can implement a client and server within 30 mins to an hour, no external tools needed. The spec is clear and readable. It's excellent. We use it with a TypeScript codebase and just share the interfaces for the services in a monorepo. The spec is so simple it doesn't really need an update. If you want a more advanced implementation with type inference for Typescript you can use this: https://github.com/shekohex/jsonrpc-ts https://github.com/shekohex/jsonrpc-ts I'd still recommend implementing your own, my own implementation is based off the one above. The only caveat compared to GRPC is you lose out on field validation due to it not being protobuf/and the obvious json decode overhead Edit: if you want to know some current users of it, I believe the Ethereum protocol uses it heavily

dimitrios1 5y ago

For #2, better yet, prefer RFC 3339.

rbluethl 5y ago
Author here. I read that a couple of times already. RFC 3339 is what you should use - I should probably change that (it's what I actually meant). :) Thanks for pointing that out.

ramraj07 5y ago

To resurface the debate, is including related endpoint urls pre-filled in a “misc” field a good idea?

solatic 5y ago

a) Use standardized error codes, not standardized error messages. Clients are responsible for internationalization, which includes presenting error messages in the user's language. If you document a set of error codes as an enum, the client can present a user-friendly error message in the user's language based on the error code. If there are dynamic parts of the error message, i.e. "404: There is no user with ID 123456 in the system", then the user ID should be extracted into the error response body, so that it can be provided correctly to the user in the user's language. b) Pagination is the devil. The state of the server can change while the user is paginating, leading to fragile clients. Don't paginate your API. If you think you have a need to paginate, have one API call return a list of IDs, and have a separate API call return a list of resources for a given list of IDs, where the second API call accepts some maximum number of IDs as a query parameter. This ensures consistency from one API call to the next.

CrimsonRain 5y ago
b -> What happens if the "list of IDs" is 10k or 100k or 1M+?
- solatic 5y ago
  There's typically business-logic ways to require a range to be specified, and then you specify a maximum range. For example, if the resource in question is audit events, you require the user to specify a time range, up to a maximum of a day, or seven days, or whatever performance / budget constraints allow for.
  - akvadrako 5y ago
    That's just like paging by day or whatever.
  - CrimsonRain 5y ago
    So if I specify a range and something changes between in that range while I'm interacting with those, sounds like same problem is back? Just same issue; different package :) If I do a search, a paginated search result can have an ID so I can paginate between the data without a new data messing up my pagination. But for normal entities, simple pagination is (mostly) more than enough. The solution you are describing is overkill and almost no benefit at all.

ceeker 5y ago

Can someone share how they handle versioning in their API when it comes to data model changes? For example `POST /users` now takes a required field `avatar_url` but it was not part of `v1`. Since this field is validated in the DB, merely having `v1` `v2` distinction at the API layer is not sufficient. So I was thinking we will have to either 1) disable DB validations and rely on app validations or 2) run two separate systems (e.g., one DB per version) and let people 'upgrade' to the new version (once you upgrade you cannot go back). Even though people refer to Stripe's API versioning blog, I don't recall any mention of actual data model changes and how it is actually managed

ComputerGuru 5y ago
I upvoted because I'm curious to hear what others are doing. We typically make sure to only make such breaking changes where either the now-required value or a sane filler value could be used. If it's the same API for the same purpose, it's usually not a stretch to assume the values for a new field are derived from some combination of an old field or else are primitive components of an old field such that they can be deduced and stubbed in your transition layer (or calculated/looked-up/whatever one-by-one as part of a bulk migration script during the transition). If your v2 is so drastic of a breaking upgrade that it bears no relationship to v1, I imagine your SOL and probably should have thought out your v1 or your v1-to-v2 story better, if only for the sake of the poor devs using your API (and you probably need separate tables at that point). For other fields like your example of `avatar_url` I would use a placeholder avatar for all legacy users (the grey anonymous snowman profile comes to mind).
- ceeker 5y ago
  Thanks. This is a fair point. I made up the example only to illustrate the idea. Since Stripe is considered some sort of benchmark here I was curious to see how they tackle all the learnings they will have over time...I feel it is very hard to think through all the future cases especially when you are just about starting out with your product. For example, in financial services and insurance, regs change and what data we need to collect change and sometimes their dependency will change. I am curious what's companies that have grown substantially had to do to their APIs.

CipherThrowaway 5y ago

> 14. Use pagination Generally agree with everything else but strongly disagree with pagination envelopes and offset pagination. Allow keyset pagination using ordering and limiting query parameters.

tlarkworthy 5y ago

A good resource, some others I consult with https://cloud.google.com/apis/design https://cloud.google.com/apis/design https://opensource.zalando.com/restful-api-guidelines/ https://opensource.zalando.com/restful-api-guidelines/

trinovantes 5y ago

Do you guys use plural or singular terms in your API endpoints or both? /books /books/:id /book /book/:id It gets harder to keep consistent when there are a lot of nouns that have the same singular/plural form like "clothing"

imnitishng 5y ago
Both /books /book/:id
systemvoltage 5y ago
/clothes/:clothing_id The reason for using plural is because without the `:clothing_id`, `/clothes` endpoint would return a collection of clothing.
akvadrako 5y ago
I always prefer singular just because it's easier to spell if you know the type. It's stupid to have /geese return a Goose. If the APIs were in Esperanto it would be different.
barefeg 5y ago
We use the form that represents a collection of objects. For example books and clothing both refer to a collection of things so I take them as valid forms

sandreas 5y ago

While I agree with the most aspects of this very good blog post, there is a minor detail that I would like to note: While pagination is important, there is another possibility of pure size - it is using cursors, like mentioned in the JSONAPI specification[1] (containing many of the hints in the topics' post) and in this blog post[1] [1] https://jsonapi.org/format/#fetching-pagination https://jsonapi.org/format/#fetching-pagination [2] https://dev.to/jackmarchant/offset-and-cursor-pagination-explained-b89 https://dev.to/jackmarchant/offset-and-cursor-pagination-exp...

rbluethl 5y ago
Author here. Thanks for the nice words - I appreciate it. I've read that a couple of times in the comments already. Definitely something worth considering, Stripe does this as well. Thanks for your comment.

gigatexal 5y ago

lol this could have been a single line blog post: “Do whatever stripe does with their APIs.” I kid. There’s some good stuff in here.

rbluethl 5y ago
Author here. Haha! You are right. I admire Stripe's API and there's absolutely a lot of things in it that are my "benchmark" when it comes to good APIs. However, they return UNIX timestamps, for example, which I don't like that much (as stated in the post). They also use POST for updates, because, if I recall correctly, PUT/PATCH wasn't a thing yet when Stripe was built initially. Thanks for the comment.

janaagaard 5y ago

My recommendation: Use PUT everywhere instead of POST. PUT has to be idempotent, so if the request fails, the client can simply post it again. This solves issue like worrying about creating a second copy of an item if a POST timed out. Using PUT to create elements means that the client has to supply the ID, but this is easily solved by using GUIDs as IDs. Most languages have a package for create a unique GUIDs.

systemvoltage 5y ago
What are the use cases where idempotency is not needed or even harmful?
gls2ro 5y ago
I don't think we should strive to remove non-idempotent cases. If something is not idempotent does not mean it is bad. It just means that request should be handled differently. In your example (and I ask this as I remained confused after also reading SO): Let's say that you need the client to provide the ID in the request body. In this case, how is using PUT when creating a new resource idempotent if the ID should be unique and you have a constraint on the DB level for example? What happens when the second call will be made with the same ID? If I execute the following sequence: Call 1: PUT resource + request.body {id: 1} => a new record is created so the state of the system changes Call 2: PUT resource + request.body {id: 1} => no record is created, maybe the existing one is updated IMO this is not idempotent nor should it be. Creating a new resource is not an idempotent operation. I also don't like that depending on the state of the DB two requests with the same attributes will have different impacts on the system. In my mind as a consumer of an API it is simpler: POST it creates a new record so I know what to expect, PUT updates an existing one.
- dropofwill 5y ago
  Upserts are more powerful because the client can always generate a new uuid to get the POST behavior you desire, but the reverse is not the case: there is no straightforward way to safely retry timeouts, partial success, side effects, etc. You say the same request has a different impact on the system, but what that means is the system converges to the requested state. Maybe this is overkill if your retry strategy is to have a user in the loop, but I don't see how it's simpler for the client even without any retry behavior (if/else new/reuse uuid vs if/else put/post).

softfalcon 5y ago

Reading this makes me feel like a take all the magic GraphQL does for granted

egberts1 5y ago

For those in Security Theatre of HTTP APIs, I found this also useful: https://github.com/yosriady/api-development-tools https://github.com/yosriady/api-development-tools

tommiegannert 5y ago

> 6. Accept API key authentication ... using a custom HTTP header (such as Api-Key). Wouldn't a bearer token [1] make more sense? Defined for use by OAuth2, but I don't see why it couldn't be the general mechanism for... bearer tokens. > 11. Return created resources upon POST Especially important if your database+caching layers use eventual consistency. [1] https://datatracker.ietf.org/doc/html/rfc6750 https://datatracker.ietf.org/doc/html/rfc6750

jamietanna 5y ago
Authentication and Authorization are two subtly different things. In this case, you may want an API key (Authentication) to be required to ensure things like rate limiting is enforced, but then want proof that the call is operating on a user, or is a machine-to-machine interaction which OAuth2 Bearer tokens work nicely for (Authorization)

AtNightWeCode 5y ago

I agree with most things. I really hate when APIs use different api-key headers depending on the role of the consumer. It is very annoying when you get dates that are not in the ISO format. There are reasons to not use UTC everywhere. One should make that decision. The reason why many APIs use POST instead of DELETE is that POST is said to be more secure. Many APIs that I use do not have neither of PATCH, PUT or DELETE. An order for instance will have an order status resource that one just keeps adding status entities to. In general, well-designed systems minimize the need for changing data.

remoquete 5y ago

Extra points: document the API / have an API technical writer in the team. Part of a good API design is documenting it. If you use an API specification format, such as OpenAPI, design and documentation overlap nicely.

BossingAround 5y ago
Ah yes, the classic: POST /document/:id?params Creates a document with parameters
- remoquete 5y ago
  More like API guides, a complete reference of the parameters, etc. :-)

zgiber 5y ago

Please don’t specify ISO8601 as the date format. That standard embodies dozens of formats. Use one specific format instead like RFC3339.

bmn__ 5y ago

The author has a poor understanding of HTTP and adjacent standards. I find it is so insufficient that he should not dispense advice yet, he must learn much more, especially about the essential yet completely unmentioned parts of a Web system: media types, hyperlinks, Link relations, cacheability. Critique: 2. ISO 8601 is a shit show of a standard. Last time I surveyed, there was not a single compliant implementation in the world. Recommend "Date and Time on the Internet: Timestamps" <http://rfc-editor.org/rfc/rfc3339 http://rfc-editor.org/rfc/rfc3339> instead. This standard is public/not encumbered by stupid copy fees and is restricted to a much more reasonable profile that can actually be implemented fully and in an interoperable fashion. 4. Use `OPTIONS ` instead. <http://rfc-editor.org/rfc/rfc7231#section-4.3.7 http://rfc-editor.org/rfc/rfc7231#section-4.3.7> 5. This goes against the design of the Web. Do not version URIs. If the representation changes and is not backward compatible, change the media type instead, e.g. by adding a profile <http://rfc-editor.org/rfc/rfc6906 http://rfc-editor.org/rfc/rfc6906>. You can serve multiple representations on the same URI and vary on the request headers (e.g. Accept-). 6. HTTP already contains a customisable mechanism. Use the standard header <http://rfc-editor.org/rfc/rfc7235#section-4.2 http://rfc-editor.org/rfc/rfc7235#section-4.2>, not the custom header `Api-Key` which is not interoperable. 7. "Don't use too many [HTTP status codes]": why? This opinion is not backed up with an explanation. The correct advice is: use as many status codes as arise from the requirements. `422 Unprocessable Entity` is useful in nearly every Web system. 8. What does "reasonable" mean? This lacks an explanation. 10. Use application/problem+json <http://rfc-editor.org/rfc/rfc7807 http://rfc-editor.org/rfc/rfc7807> instead. 11. "It's a good idea to return the created resource after creating it with a POST request": why? This opinion is not backed up with an explanation. If you follow this advice, the user agent cannot programmatically distinguish between a representation reporting on the requested action's status, and the representation of the newly created resource itself. Use the Content-Location header <http://rfc-editor.org/rfc/rfc7231#section-3.1.4.2 http://rfc-editor.org/rfc/rfc7231#section-3.1.4.2> to make that possible, use the Prefer header <http://rfc-editor.org/rfc/rfc7240 http://rfc-editor.org/rfc/rfc7240> to give the user agent some control over which representation to respond with. 12. "Prefer PATCH over PUT": disagree, ideally you offer both since there is a trade-off involved here. The downsides of PATCH are not mentioned: the method is not (required to be) idempotent meaning it becomes moderately tricky to keep track of state, and the client is required to implement some diff operation according to the semantics of the accepted media type in the Accept-Patch header which can be difficult to get right. 13. Missed opportunity to advertise the OPTIONS method and related Accept-Post <https://datatracker.ietf.org/doc/html/draft-wilde-accept-post https://datatracker.ietf.org/doc/html/draft-wilde-accept-pos...> / Accept-Patch <http://rfc-editor.org/rfc/rfc5789#section-3.1 http://rfc-editor.org/rfc/rfc5789#section-3.1> headers. A representation of a specific media type can self describe with the "type" Link relation <http://rfc-editor.org/rfc/rfc6903.html#section-6 http://rfc-editor.org/rfc/rfc6903.html#section-6>. 14. Don't mix data with metadata. Use the Range header <http://rfc-editor.org/rfc/rfc7233 http://rfc-editor.org/rfc/rfc7233> and next/prev Link relations <https://webconcepts.info/concepts/link-relation/next https://webconcepts.info/concepts/link-relation/next> instead. 15. Instead of complicating both server and client, the better idea is to simply link to related resources. We are not in the 1990s any more. A well written Web server offers HTTP persistent connections, HTTP pipelining, all of which make round-trips cheap or in the case of HTTP/2 server push, even unnecessary. Benchmark this. 1. + 9. betrays a weird obsession with naming. The advice is not wrong, but it shifts attention away from the genuinely useful areas that benefit much more from a careful design: the media types, the hyperlinks between and other hypermedia mechanisms for traversing resources (forms, URI templates). If an inexperienced programmer follows the advice from the article, he will the idea to spend lots of time mapping out resources in the identifier space and methods and possible responses for documentation purposes. This is both a waste of time because the single entry point is enough and the rest of the resources can be reached by querying the server via OPTIONS and traversing hyperlinks etc., and dangerously brittle because of strong coupling between the server and the client.

loup-vaillant 5y ago

For a second there I thought this was about non-web APIs: https://caseymuratori.com/blog_0024 https://caseymuratori.com/blog_0024

spacechild1 5y ago
Side note: I find it quite annoying that web devs have hijacked the term “API“ as a synonyme for “web API“, to the point where many devs are not even aware of the original broader meaning. Nevertheless, as a (largely) non-web dev I found the article interesting and useful.

11235813213455 5y ago

Something I wonder is how to design search/list endpoints where the query can be long (a list of IDs, ex: /users?ids=123e4567-e89b-12d3-a456-426614174000,123e4567-e89b-12d3-a456-426614174001,123e4567-e89b-12d3-a456-426614174002,...), so long that it can exceed the url max length (2048), after 50 UUIDs, you can quickly exceed that length, so GET is not ideal, so which method, SEARCH with a body? POST with a body?

flowerbreeze 5y ago
I would go with POST with a body in that case, where I interpret is as a "new search item" and use GET to scan through results, if there are many results available. I don't think I've needed something like this more than once or twice though. From users perspective, asking information on specific 50 items at once is not something commonly done.
- 11235813213455 5y ago
  it's used for an export feature, where we join data from the client-side, not ideal, but for now it's a good compromise in terms of complexity for the API and client
daniel-thompson 5y ago
There was a new RFC published a few months ago to address this use case. It defines a new HTTP method QUERY, which is defined to be safe & idempotent like GET and explicitly allows a request body like POST. See https://www.ietf.org/id/draft-ietf-httpbis-safe-method-w-body-02.html https://www.ietf.org/id/draft-ietf-httpbis-safe-method-w-bod...
- 11235813213455 5y ago
  great, thanks for the info!
athrowaway3z 5y ago
The type signature: ` list of uuids -> list of results ` is usually bad design to begin with.

ChrisMarshallNY 5y ago

It’s a good posting. I do many of these things, but not all. I’ve been designing SDKs for a long time; APIs, for a somewhat shorter time. I don’t like “pure” RESTful APIs, because I feel as if they are “human-hostile.” I tend to follow the patterns set by companies like Google, where the stimulus is a fairly straightforward URI (with parameters, as opposed to a block of XML or JSON —if possible, as I know that we often still need to send big data as transaction data, which can be a pain, depending on the method), and the response is a block of structured data. That also makes it a lot easier for API clients to build requests, and allows the server to “genericize” the processing of requests. > 10. Use standardized error responses Is something I do, along with a text header payload, describing internal information about the error. It is my experience that this text payload is never transferred, and I am forced to send a text response via the body, if I want to know internal information. That can be a pain, as it often breaks the transaction, so I have to play with the server and client, frequently, using a cloned server instance. I should note that my forte is not backend development, so the chances are good that I am not familiar with all the tools at my disposal.

thatwasunusual 5y ago
> I tend to follow the patterns [...] Can you elaborate on this? Give examples?
- ChrisMarshallNY 5y ago
  I don’t feel like doing that in a comment, but feel free to check out some of my work. The BAOBAB server[0] is one of my more recent examples. I’m using a modified version as the backend for the app I’m developing, now. It works great. I should note that the BASALT layer uses “plugins,” that can be extended to include “pure” REST APIs. I just haven’t found these practical, for my purposes. [0] https://riftvalleysoftware.com/work/open-source-projects/#baobab https://riftvalleysoftware.com/work/open-source-projects/#ba...

kuon 5y ago

ISO 8601 is bad, use RFC 3339. A lot of implementation are actually based on an ISO draft which changed in final release. But most dev do not have access to the spec. For example, timezone can only be specified as offset in the standard, while implementations accept name. Globally avoid ISO for software, it is non free crap. Also, do not use those standard for dates with timezone in the future. Use wall time/date and location. As timezone can change way more often than you think.

idoubtit 5y ago
My own experience is that (unix) timestamps are much less error-prone than textual representations like ISO 8601 and such. A field like `update_time_seconds` is clear and easy to convert into any representation. This is what the Google API Improvement Proposals recommends in most cases, though civil timestamps are also described. https://google.aip.dev/142 https://google.aip.dev/142 Of course, when preparing queries against the API, you may need a helper to build valid timestamps. But this is mostly relevant to the discovery phase which isn't automated. And textual dates also have drawbacks, for instance the need to encode it when used a an URL parameter.
- kuon 5y ago
  I agree, for timestamp, I often use ms or sec offsets. But I do not dare say it because I pass for an old fool.
grishka 5y ago
Why use string-based timestamps at all? Use unixtime. It's much easier to parse and it literally can't be malformed.
- arendtio 5y ago
  Sometimes timezones or the like change. So for future dates, those textual represations are probably better. Example: in November you create an appointment in 6 months at 15h, but then your government decides, to not use summer time next year. If you use timestamps your appointment will be wrong by one hour (humans tend to keep the 15h). In general, I am a huge fan of timestamps, but I think it is good to know where they have their limits.

perlwle 5y ago

https://www.vinaysahni.com/best-practices-for-a-pragmatic-restful-api https://www.vinaysahni.com/best-practices-for-a-pragmatic-re... Helped me get started with API design in my early career. Learning from other existing APIs helps too. such as stripe, github, shopify. Any others? Something that We do at my current job: * set a standard and stick with it. tweak it if needed. we even have naming standard on some of the json key. for example, use XXX_count for counting. when it doesn't make sense, use total_XXX. * Document your API, we use postman and code review API changes too.

myshkin5 5y ago

> 5xx for internal errors (these should be avoided at all costs) An anti pattern I’ve often seen has devs avoiding 5xx errors in bizarre ways. I would change the above to make to have monitoring in place to address 5xx errors. By all means, let your code throw a 500 if things go off the rails.

rbluethl 5y ago
Author here. I generally agree - IF something goes terribly wrong within the application, a 500 is definitely the way to go. The thing is - these just shouldn't happen that often. Because in that case we're having a bigger problem. :D For example I have never seen Stripe return a HTTP 500 from their API. Thing just works.

rbluethl 5y ago

Hey friends! Author (@rbluethl) here. :) I just randomly noticed that my post made it to the front page! Thank you all for your comments.