34 ms·
API Design Guide
- RubenSandwich 10y agoSeems pretty good. Specifically this part of the guide is pretty well written: https://cloud.google.com/apis/design/resources https://cloud.google.com/apis/design/resources. One thing that is surprising to me however is that their is no mention of using HTTP Status Codes in responses.
- moca 10y agoThanks for the comment. The error handling chapter will be published in a few weeks. For now, you can reference https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto https://github.com/googleapis/googleapis/blob/master/google/.... Disclaimer: I am one of the co-authors.
- programd 10y agoCanonical error codes used by Google, mapped to HTTP error codes: https://github.com/googleapis/googleapis/blob/master/google/rpc/code.proto https://github.com/googleapis/googleapis/blob/master/google/...
- lilbobbytables 10y agoWell, I can tell you from using their API's that Google doesn't necessarily use them. I don't recall that specific scenarios, but over the past year I learned this with the maps API. As long as you're hitting a real endpoint, then it will return 200 - even when there are errors and it should clearly return a matching http status code. I suppose it so the same API could be implemented in any given protocol, however, I don't think that is useful in many cases.
- euyyn 10y agoYou might have been hitting the old JSON-RPC endpoint of that API, which has to return 200 to follow the standard. Or a gRPC version if there's one already. Otherwise HTTP status codes for errors are used, and standardized internally.
- moca 10y agoMany Google APIs were created before this guide. New APIs published at https://github.com/googleapis https://github.com/googleapis follow this guide. Having the same API available via both REST and gRPC is very valuable, as gRPC often provides 10x performance.
- programd 10y agoUsing HTTP status codes in your responses is a trap. It conflates the API transport with the actual semantics of the API. The goal of HTTP error responses is to say that something went wrong in the transport layer. The goal of API error responses is to say that something went wrong in your service. For example, your HTTP REST server may be perfectly fine, but your back end DB may be misbehaving. Having separate API level error responses, for example an explicit field called "error" in your JSON response, I consider to be best practice. Frequently your client needs to know the difference, for example to determine what kind of error to return to the user or what retry strategy to use. In typical HTTP REST services this transport/API error split makes it really easy to create client code which only needs to check two conditions - if the HTTP response code is 200 or not, and if the error value is set or not. You also don't have to shoehorn your error handling into the very limited set of errors provided by the HTTP protocol. The other real-world advantage of this is that when you outgrow HTTP as the transport protocol for performance reasons this makes porting the API really easy to, e.g. protobuf RPC, or even raw TCP. The error is already defined as part of the API and you don't need to rewrite all your client code to deal with mapping multiple HTTP response codes to your new transport. It's good future proofing I've seen pay off in a at least a couple of real-world cases. Bottom line - your server should always return HTTP status 200 and a separate API error response. There's also a reasonable debate to have about whether non-error responses should also include an explicit "error" field with some default OK value. There may be good reasons to leave it out, e.g. if you want to save bandwidth, but I consider that a fairly insignificant point. For consistency my APIs always return a default OK error field on non-error responses - your mileage may vary.
- aaronblohowiak 10y agoThat is great in theory but it throws out all of the deeply rooted tooling that reports and aggregates http status codes across the service architecture. It is nice to be able to tail the apache log and see the status. In your situation I would need to have an additional system that uses application details for similar purposes.. you can do it as you describe but there is a cost! It is also against convention, which increases the custom tribal knowledge that people working in that system have to acquire. By the time you "outgrow" http I assume you'd have custom metrics and logging/tracing, but I would not suggest it as a good place to start.
- deleted 10y ago[deleted]
- arohner 10y agoStep 1) document the endpoints enough that outside developers can write their own clients. It took quite a bit of work for me to get a native Clojure client working to connect to the google cloud SDK. That was after wrestling with jar-hell around gRPC and calling the Java client from clojure, which is decidedly not pretty.
- chatmasta 10y agoIn these situations the best thing you can do is man-in-the-middle the HTTP requests from an existing client. That said, I've only ever had to do this when reversing private mobile API's. I can't believe a REST API from Google would be missing documentation of the raw HTTP endpoints?! That should be the first documentation of an API, before that of any specific client implementations.
- kbenson 10y agoOr get a pre-existing client for a non-compiled language. Chances are there's a Python/PHP/Ruby/Perl implementation, and those shouldn't be hard to pick apart.
- euyyn 10y agoFor future reference, here's the HTTP/REST documentation for the Cloud APIs: https://cloud.google.com/apis/docs/overview https://cloud.google.com/apis/docs/overview E.g. for the GKE API: https://cloud.google.com/container-engine/reference/rest/ https://cloud.google.com/container-engine/reference/rest/ (For people that rather use an existing library, this lets you pick one of 7 languages and start from there: https://cloud.google.com/docs/ https://cloud.google.com/docs/ )
- arohner 10y agoYes, but that doesn't deal with authentication. My problem was getting the JWT stuff working. The solution ended up being: https://gist.github.com/arohner/8d94ee5704b1c0c1b206186525d9f7a7 https://gist.github.com/arohner/8d94ee5704b1c0c1b206186525d9...
- KabuseCha 10y agoFantastic Read! But I am still looking for some books on good API-Design, anybody has any recommendations?
- rainhacker 10y agoIf you don't mind Java, Effective Java by Josh Bloch has good API design material. Josh designed collections API in Java
- jzsprague 10y agovery good read, but not really about web/http/rest API's.
- ericcumbee 10y agoNot sure what level you are looking for but Build APIs you won't hate was a great resource to me. https://www.amazon.com/Build-APIs-You-Wont-Hate/dp/0692232699/ref=sr_1_1?ie=UTF8&qid=1487789904&sr=8-1&keywords=apis+you+won%27t+hate https://www.amazon.com/Build-APIs-You-Wont-Hate/dp/069223269...
- akman 10y agoCheck out Apigee's ebook on good API-Design: https://pages.apigee.com/rs/apigee/images/api-design-ebook-2012-03.pdf https://pages.apigee.com/rs/apigee/images/api-design-ebook-2... disclaimer: both are my employers
- andreygrehov 10y agoI would like to add Microsoft's API Guidelines [1] here, which is also a well written document and can be helpful to anyone designing an API. [1]: https://github.com/Microsoft/api-guidelines/blob/master/Guidelines.md https://github.com/Microsoft/api-guidelines/blob/master/Guid...
- camus2 10y agoIt's interesting that both of these guidelines kind of reject HATEOAS by mandating explicit versioning. It seems that HATEOAS was never really a thing. It's just too complicated to implement in practice. In that sense, REST in practice has always been just RPC without a clear spec for procedure call like XML or JSON RPC.
- einrealist 10y agoI think, there is just too much bias involved. At least thats what I am experiencing. Not using HATEOAS ever, but complaining about it makes me angry. My preaching: If you can write a client by using a semantic document format (e.g. HTML, XML+XSD, Json-Ld), you end up with a more elegant and stable implementation. And as a provider of such an API, I spend more focus on the surfacing domain than the structure of my resources. It makes me sad, that even Google does not try.
- veesahni 10y agoI wrote about API design (with a similar rejection of HATEOAS): http://www.vinaysahni.com/best-practices-for-a-pragmatic-restful-api#hateoas http://www.vinaysahni.com/best-practices-for-a-pragmatic-res... In short: humans can follow links, even as a website goes through significant changes. Code can follow links, but can't make clear independent decisions when significant changes happen to the API. [Updated for clarity]
- johnjuuljensen 10y agoCode can follow links as well, as long as the semantics doesn't change. Aside from versioning through mimetypes, which I believe is a really bad idea, I find HATEOAS to be a beautiful concept, although not very useful in practice. It's a good place to start though. Trying to design for that can help you shape your API properly, just like SOLID or TDD can do for code.
- camus2 10y agoPlease drop fixed headers from web pages. If you want easy access to the top of the page use anchor links instead. On a laptop headers often take a big chunk of available screen. It just pisses me off every time I see a page with a fixed header. All your reader aren't using imacs...
- chatmasta 10y agoThis is a marketing website. I'm sure they A/B tested the fixed header and it probably converts better than otherwise.
- camus2 10y agoThis isn't a marketing website this is the documentation website for google cloud. I'm logged in right now on google cloud and it's still displaying that header.
- chatmasta 10y agoYeah? Who's paying google the big bills? The people who are looking to "CONTACT SALES," which is conveniently a link in the fixed header.
- dkersten 10y agoExtremely annoying on mobile. So much that it made me not want to read the page anymore
- zeveb 10y agoAnd it tends to break using Space or Page Down to advance. I really wonder if hipsters ever actually read web pages, or if they just load them, look at them and then go back to discussing the merits of their fair trade, artisanally-roasted espressos.
- etaty 10y agoI am curious if anyone went to GraphQL without regrets?
- e1g 10y agoWe've been using GraphQL for everything since late 2015. All recent code is GraphQL-first, and all old code is proxied by a GraphQL layer in front of it. Our application helps BigCos to understand if they pay people fairly and to run smart pay reviews. It's a relatively small codebase, ~100k LOC, but it's essential complexity is in managing and connecting dispersed data about employees and markets. GraphQL allows us to represent the natural links within this data, then the app frontends can present whatever business information is helpful in that page/sidebar/widget/card without separate endpoints. With REST, we had the same problems with every feature: over/under-fetching, can't express relationships well, and can't evolve the schema easily. When we tried to work around these issues (e.g. "v2?fields=a,b,c"), we ended up with a poorly implemented subsection of GraphQL that's not benefiting from Facebook's experience. To compare to the world of databases, I view REST as a Key-Value protocol and GraphQL as an SQL with joins and functions. If all you need is to lookup a document, don't overcomplicate it. But if you need to express relations, you don't want to do that in userland. The only advantage of REST is using a widely known standard with rich tooling and well-published "best practices" (that just try to work around REST limitations).
- daliwali 10y agoREST describes relationships very well, via hyperlinks. You navigated to this page via a hyperlink. If your API doesn't do that, it's not REST, this is what people usually mean when they point out that an API isn't complying to the REST style.
- justinsaccount 10y agoREST describes relationships just fine. Now return a list of 100 documents that each have a list of related comments. Your client just needs to request GET /documents GET /documents/1/comments GET /documents/2/comments GET /documents/3/comments GET /documents/4/comments GET /documents/5/comments .. GET /documents/99/comments GET /documents/100/comments Easy, right?
- amingilani 10y agoMy biggest pain when designing a rest API is a standard authentication method that won't drive me crazy. So far i've always used 3rd party modules to implement different kinds of authentication but I never quite understood it in depth. Apart from HTTP Basic Auth, but please don't use that.
- deleted 10y ago[deleted]
- camus2 10y ago> Apart from HTTP Basic Auth, but please don't use that. What's wrong with basic auth with HTTPS? You can delegate authentication with OAUTH and then use OAUTH for authorization but authentication still has to be done somewhere.
- avenoir 10y agoI think Twillio API still uses or used basic auth via HTTPS.
- tomschlick 10y agoSo does Stripe. There is nothing wrong with basic auth for api tokens so long as you're using HTTPS.
- zeveb 10y ago> What's wrong with basic auth with HTTPS? The only thing wrong that I can see is that it's 2017 and the browsers still don't have a good (indeed, AFAIK, any) UI for logging out.
- nandhp 10y agoOr for staying logged in across sessions (or not). But it should be fine for an API.
- johnjuuljensen 10y ago
- Dirlewanger 10y agoProtocol Buffers...GraphQL...JSON-API...so many damn choices for API implementation! Next we need someone's essay of a blog post comparing/contrasting them all. Also, the Protocol Buffers link in the 3rd paragraph is 404.
- euyyn 10y agoThanks for catching that! Fixing it (it should have pointed to "proto3", not "proto3.md").
- scarface74 10y agoIf you're using a good framework like C# Web Api, you don't have to choose. Just implement them all by adding them to the pipeline and the framework will automatically serialize/deserialize based on the accept header.
- mbesto 10y agoInteresting, do you have more details about this?
- scarface74 10y agoYou register your custom serializer at application startup... http://www.strathweb.com/2014/11/formatters-asp-net-mvc-6/ http://www.strathweb.com/2014/11/formatters-asp-net-mvc-6/ Your controller action looks like this: public List<Employee> Get() { ..... return employeeList; } Based on the serializers you have registered and the request Accept header, WebApi will serialize the list into JSON, XML, BSon (all built in) or a custom serializer that you add like Protocal Buffers http://www.infoworld.com/article/2982579/application-architecture/working-with-protocol-buffers-in-web-api.html http://www.infoworld.com/article/2982579/application-archite... Basically you can have all four registered and the client decides what it accepts.
- jeppebemad 10y agoI don't see the actual guidelines, only Contents, Introduction and Conventions. On iOS Chrome /Safari. Also the fixed buttons overflows.
- hathawsh 10y agoThe page's responsive design is buggy. On narrow screens, the entire left column disappears. All the important content is only accessible from that left column.
- nandhp 10y agoI don't think it disappears, it just moves into the menu button, which seems fairly typical for navigation sidebars on narrow screens.
- daliwali 10y agoWhat they describe is not REST. Nowhere in this document mentions hyperlinks, a strict requirement of the REST architectural style. The best analogy would be a simple web page, which usually contains hyperlinks that a client can follow to discover new information. Unfortunately, web developers' understanding of REST ends with HTML, and they re-invent the wheel, badly, every time they create an ad hoc JSON-over-HTTP service. There is a standardized solution for machine-to-machine REST: JSON-LD [1], with best practices[2] to follow, and even some formalized specs[3][4]. To Google's credit, they are now parsing JSON-LD in search results, which is much nicer to read and write than the various HTML-based micro-data formats. On a related note, REST has nothing to do with pretty URLs, naming conventions, or even HTTP verbs. That is to say, it is independent of the HTTP protocol, but maps quite naturally to it. [1]: http://json-ld.org/ http://json-ld.org/ [2]: http://json-ld.org/spec/latest/json-ld-api-best-practices/ http://json-ld.org/spec/latest/json-ld-api-best-practices/ [3]: http://micro-api.org/ http://micro-api.org/ [4]: http://www.markus-lanthaler.com/hydra/ http://www.markus-lanthaler.com/hydra/
- BigChiefSmokem 10y agoThis is an API guideline, not a strict REST one. Also it has become very clear to me everyone has a different interpretation of what REST can or should be. We all are quick to forget that the actual acronym stands for "representational state transfer" which is an abstract concept and therefore can be implemented in many, many different ways.
- deleted 10y ago[deleted]
- daliwali 10y ago>it has become very clear to me everyone has a different interpretation of what REST can or should be This is very true, the term has been abused so much that it has no relation to its original meaning. We can loosely define REST by one of its key qualities: hypermedia, that is a lot easier to say than HATEOAS. By focusing on this distinction, it rules out 99% of APIs in the wild. On top of that, hypermedia media types and linked data are important for an API to be self-documenting, so there is a rather high bar for developers to implement.
- rodionos 10y agoHTTP Method DELETE. Payload: empty. I know DELETE is not supposed to have any payload, but using PATCH is awkward if you have to delete multiple resources based on a query or a filter. You need to specify a 'delete' action as part of PATCH request which means the payload model has to be different. Just awkward.
- euyyn 10y agoYou face the same problem if you want to just list multiple resources based on a query or a filter. The solution isn't to send a GET with an HTTP body, which is counterintuitive to most people, and many proxies just drop. It's to make your filter part of the URL (as a query parameter). Same with DELETE.
- veesahni 10y agoA bulk deletion isn't defined as part of their "Standard Methods" .. For stuff that doesn't fit the standard methods, they have this page on custom methods: https://cloud.google.com/apis/design/custom_methods https://cloud.google.com/apis/design/custom_methods That said, I'd implement a bulk deletion in one of three ways: :::: ONE DELETE /thingies/id1,id2,id3 When all is ok, the response is a 200. However, if one of the deletions fail, how you handle the response is more complicated. :::: TWO POST /thingies/bulk_delete and have your ID's listed in the body. Still same problem with handling the response. :::: THREE POST /bulk Instead of implementing a bulk deletion at all, think about how you can implement "bulk requests" as a higher level feature of the API. So you can transport a bunch of delete's as part of a single HTTP request, and get back a bunch of response codes packaged into a single response.
- rodionos 10y agoPOST /thingies/bulk_delete Exactly. That's the approach we've taken. It has worked better for us than sending a delete action via PATCH.
- somedumbguy22 10y agoI wonder if someone from the Apigee team wrote these, as Google recently acquired Apigee[1], and the guidelines are mostly inline with what Apigee recommends.[2] [1]https://techcrunch.com/2016/09/08/google-will-acquire-apigee-for-625-million/ https://techcrunch.com/2016/09/08/google-will-acquire-apigee... [2]https://apigee.com/about/resources/ebooks/web-api-design https://apigee.com/about/resources/ebooks/web-api-design
- moca 10y agoThis guide has been in use since 2014, including recently launched Cloud Spanner API. Disclaimer: co-author of the design guide.
- oaktowner 10y agoAnd one more thing: Apigee's (awesome) e-book is designed to help customers write APIs. The Google Style guide is a guide we use (and have been using for years) when designing our own APIs. (I work on the API team at Google).
- pbreit 10y agoWhat is current consensus on client libraries? Braintree for example requires that you use their client libraries where as Stripe makes them optional. With Google's gRPC thing I can definitely understand using libraries for performance. Otherwise, isn't making simple REST calls without custom libraries sufficient for most uses? Or if you want a library, something generic like Unirest [1]? 1. http://unirest.io/ http://unirest.io/
- ihsw 10y agoClient libraries can be helpful and sometimes reduce a lot of plumbing/boilerplate that you would end up writing on your own (eg: authentication, paging). Most environments -- Ruby, Python, PHP -- have minimal HTTP clients in standard libraries however they all have capable third-party libraries. Unirest as you mention, but also Requests for Python or Guzzle for PHP. Product-specific clients generally build on these third-party libraries or go with the standard libraries instead. The product-specific clients generally offer more comprehensive error handling, for example if an API relies on arcane error codes that wouldn't immediately be obvious to an end-user.
- tyingq 10y agoMight be helpful to see Braintree's rationale for why they don't publish docs for plain external REST: https://www.braintreepayments.com/blog/when-rest-isnt-good-enough/ https://www.braintreepayments.com/blog/when-rest-isnt-good-e... I don't necessarily agree with all the points, but I can see why they made the decision they did.
- pbreit 10y agoI've read it several times and found it uncompelling.
- thesandlord 10y agoI've used the simpler GCP APIs (like the Machine Learning ones) with direct REST calls. I've also been forced to use the REST API for things like Google Sheets because the client library documentation was so confusing. For more complicated services, using a client library makes sense. Why reinvent the wheel? With gRPC/Swagger/OpenAPI/etc you can also generate your own client stubs if you need to. IMO, if you require a client library, there better be a really good reason... (I work at Google Cloud, and often work with the API/libraries team. Opinions are my own)
- nevi-me 10y agoVery interesting read! I like that GOOG is pushing gRPC more on their own services. I've been a gRPC user since Sep/Oct last year, and it's made developing for Android, Node.js, JVM, Python more pleasant from a networking perspective. The ease of just moving logic from Node.js to a Java gRPC server, and then redirecting the HTTP2 proxy to the right place, has been awesome. I've started teaching some people in the team how to use gRPC, and we're def going to be using it where permissible on client projects.
- theptip 10y agoAn interesting design question arrises around nested resources. Google in this doc buys into deep nested structures, e.g. `//calendar.googleapis.com/users/john smith/events/123` (from [1]). I think this pattern is unambiguously sensible when the child objects are strictly scoped under the parent. But it's less clear how to represent resources that are shared between multiple parents; for example, what if event 123 can be referenced under another user's API resource as well? If we permit `//calendar.googleapis.com/users/bob/events/123`, now we have multiple URLs referring to the same object, and things can get quite tricky in the implementation. Django Rest Framework strongly discourages (and makes it quite hard to implement) nested resources, FWIW. I've found that a policy of only permitting one level of nesting seems to be a good balance for shared objects, e.g. `//calendar.googleapis.com/users/john smith/events/` returns: ``` [ { url: "/events/123"}, ... ] ``` Interested to know how others have solved this problem. [1]: https://cloud.google.com/apis/design/resource_names https://cloud.google.com/apis/design/resource_names
- combatentropy 10y agoHere's one way to tackle it, if behind your REST API is an SQL database: /schema/table/key So: //www.example.com/calendar/events/123 To address many records, like all that belong to Bob, use the query string instead of purely the path: //www.example.com/calendar/events/?user=bob
- theptip 10y agoThis is in my experience the `standard` design; it does give you a lot of freedom to change what filters you allow, and to stack them. Nobody's going to get fired for this design, and there's a lot of prior art around it to draw examples from. It also has the benefit of keeping the API surface small and clean. But it makes it a bit weird to do HATEOAS; you _could_ do `GET Bob => {events: "calendar/events/?user=bob"}` -- but then you're hyperlinking to a search and not a resource. It also tells less of a narrative in the structure; `user=bob` is just another filter that you can use to apply to the events set. But we get a chance to describe the shape of the data a bit more if we choose to declare an intermediate resource (/users/) and attach some links to it (=>/users/bob/events). Now, if there are ten ways that you need to slice your `events` set, and ?user=bob is but one of them, then scoping a sub-resource /users/events/ isn't that useful/descriptive. As an aside, I think this is where HATEOAS is nice; it makes it very easy to navigate an API as a developer, see what actions are possible at every node, and hopefully learn the intent of the author of the API without having to chew through a set of API documentation. Django Rest Framework's API browser is a great example here.
- ex3ndr 10y agoThey didn't mention one very important thing - querying only required data and making connections between resources. For example, you need to download some git commits with user profiles. User is a different resource than git repo. How we can request such data in one single request? Then you will need also to load referenced issues (if present) that is implemented as different resource. GraphQL solve this problems in a very nice and flexible way.
- kevincox 10y agoIf the data is different there is very little need to ask in a single request. Just send two parallel requests. Of course in some cases getting a list of objects is cheaper then multiple requests but that is only if the objects are "related" and stored together. However I do agree that GraphQL is great for a lot of use cases.
- kazagistar 10y agoNeither side supports the case of [promise chaining](https://capnproto.org/rpc.html https://capnproto.org/rpc.html), where one or more resources can be used to look up further resources in a single round trip. Each style has tradeoffs.
- andyfleming 10y agoDo any of these API guides have good guidance around batch endpoints like handling a PATCH on multiple resources as a single request?
- webmaven 10y agoNot sure how common this pattern is, but one way I've seen this handled is to do a query to get a list of results, the document listing the results (or more commonly, the first page of results) has a link to a temporary resource representing the set of all results, which you can send a DELETE to remove all the members of the set. By analogy, you could send a PATCH to modify all the members instead.
- nhumrich 10y agoThis guide talks about batch get which is www.example.com/foos/123/bars:batchGet You could do something similar for other batches
- tofflos 10y agoI've used two strategies: 1. If it's a transaction (all-or-nothing) I POST a new transaction resource which references all the target resources. Remember that you can create as many resources as you want. There is no need to have a 1:1 mapping between your resources and the database. 2. If it's a batch statement (some-can-fail-some-can-lose) I simply stick to issuing multiple requests. This frees me and my clients from having to write complicated code that deal with partial success - and it can still be very fast especially with request pipelining. If issuing multiple statements is too slow then I would attempt to increase the speed of the stack before adding the complexity of having to deal with partial success. So in conclusion... No really good ideas on how to write batch statements. I don't really think REST APIs with their one HTTP response code maps very well for that use case. But I hope one of the two strategies above will be useful to you.
- novaleaf 10y agoon a related note, anyone know a good saas for api documentation? preferably one that could take jsdoc imports or other code based generated docs...
- jbattle 10y agoI'm not sure exactly what you are looking for but try this tool from the swagger people: http://editor.swagger.io/#/ http://editor.swagger.io/#/ It gets a little laggy for large swagger docs but its' quick and easy for smaller API docs.