7 ms·

Some People Understand REST and HTTP

191 points by llambda 15y ago

technoweenie 15y ago
Any notion of HATEOAS in the GitHub API is purely experimentation. With the exception of the pagination links, the rest of them could change format or be removed at any time (until something is properly documented at http://developer.github.com/ http://developer.github.com/). I don't think the Link header is descriptive enough, so most of them will probably go away.
- deleted 15y ago
  [deleted]
shimonamit 15y ago
Another interesting post by Steve. What are your thoughts on twilio's date prefix in the URI versus using "v1" for example? Does discovery appease the purists qualms about versioning? But if using HATEOS enables discovery (which twilio is doing), why put a date there in the first place? I am thinking they're future-proofing their top-level resource discovery, but maybe I'm missing something.
- technoweenie 15y ago
  Even with all that in place, it doesn't mean every client is actually going to use it. http://www.w3.org/Provider/Style/URI.html http://www.w3.org/Provider/Style/URI.html
  - shimonamit 15y ago
    Yes, but that is yet another good reason not to version.
- necubi 15y ago
  [Disclaimer: I work for Twilio.] It's actually not future-proofing, in that today we have different version of the API: 2008-08-01 and 2010-04-01. Some sort of versioning is necessary so that we can improve our API without breaking current clients or forcing all of our users to update their code. And while the goal of a completely discoverable API is laudable, I've never seen it work well in practice. We have a real service with real users, and we need something that works well and is simple to consume. And I don't see how date-based versioning is any better than version numbers from a HATEOS perspective, since it's not really any more discoverable. There is however an argument to be made that version numbers are simpler, clearer and easier to remember.
  - apaprocki 15y ago
    Isn't it simpler to infer backwards compatibility from proper major.minor version number bumps? A major version number bump indicates API(/ABI) breakage, plain and simple. A minor version bump is simply a backwards-compatible change to the existing major. Version numbers for marketing purposes are not really that useful, but if they are strictly used to indicate compatibility, they are very helpful. No one cares if the API is at, say, version 1023.301 as long as they know that 1023.200 is compatible with the latest version and 1022.499 isn't.
    - johns 15y ago
      (I'm also from Twilio) We historically have not done version bumps for minor improvements to the API that do not break existing applications. 2010-04-01 has been updated a couple times since it was first released with new features (like subaccounts, applications and short codes) but all of those were additive there was no reason to change the version.
icebraining 15y ago
The good GitHub uses custom MIME types for all of their responses. They're using the vendor extensions that I talked about in my post, too. For example: application/vnd.github-issue.text+json Super cool. Hmm, I wouldn't call that "good." It's definitively better than sending 'application/json' or 'application/xml', which tell us nothing about the structure of the data, and it's probably inevitable in their context, but "good" would be to use an actual standard mimetype instead. The problem with using mimetypes tied to the service is that it undermines the concept of Uniform Interface, by forcing developers to write clients specifically for that service. Imagine if instead of standardizing on (X)HTML, CSS, JS and a couple of image formats, each website used their own format.
- dasil003 15y ago
  Yes but HTML/CSS are general purpose formats. What can be standardized between the wide variety of web APIs out there?
  - dreamdu5t 15y ago
    All of these popular services can be expressed in standard ones like HTML/JSON.
    - masklinn 15y ago
      Except as far as an automated tool is concerned, knowing that something is "HTML" or "JSON" is useless: it will not be able to consume it. So it's not a useful information for an API. Instead, specifying the precise subset of the general-purpose format (essentially the language, where the general-purpose format is a meta-language) helps just about everybody. And that's what github does: the `+json` postfix indicates that they're returning a JSON dialect, the `vnd.github-issue.text` explains what that dialect precisely is.
  - icebraining 15y ago
    I'm not saying that there should be "one format to rule them all" or that in many cases, having a custom format isn't inevitable, but there are a lot of standard formats that can be reused. And you can use a mix of standard and custom formats for different resources. In this case, I think the best solution would be to develop a standard "issue format" that would be used by all bug trackers, instead of each having its own. In another example, the "Users" part of the API could very well reuse one of the existing formats. This also circles back to a (controversial) opinion I have, which is that JSON isn't the best solution for most of these services. With all of its flaws, XML has features like namespaces which enable a better reuse of standard formats - in the "Issues" example, where there is a "user" key, an XML document could integrate a foaf:Person element which is standardized.
deleted 15y ago
[deleted]
dreamdu5t 15y ago
Vendor mimetypes are bad. They defeat the purpose of REST. I should be able to request from Github the content-type(s) I am willing to accept, and Github serves them to me. It's ironic that a blog post about people understanding REST highlights their misunderstanding of it. What Github does isn't bad, but you shouldn't praise it for being RESTful.
- ddagradi 15y ago
  This is not about requesting the content type though, it's about requesting the format of the content. If I make a request to https://api.github.com/users/dominic/gists https://api.github.com/users/dominic/gists, I'm getting back JSON, since that's all Github's API supports. Github's custom mime types allow you to request the content of the response in a specific format. In their case, either in Github Flavored Markdown, text, html, or all of the above. They allow you to do this with an Accept header, but the same could be accomplished with a GET parameter tacked onto the request (but that's not RESTful at all!). Using HTTP headers appropriately feels like exactly the right way to handle this situation.
- masklinn 15y ago
  > Vendor mimetypes are bad. They defeat the purpose of REST. No they don't. Most standard mimetypes are far too broad and completely useless as an actual content type. "application/json" for instance does not tell you anything about the content you're getting, apart from the meta-format in which that content is expressed. Useless.
  - KevBurnsJr 15y ago
    It tells you it's not HTML. That's useful if you hope to parse the message body.
    - reinhardt 15y ago
      Ok, so you can "parse" it, which is just a way of saying you can transform a bunch of bytes to a nested structure consisting of a few basic types (dicts, lists, strings, floats, etc). Now what? What can you (or actually your program, not you as a human) do with it without some sort of schema, explicit or implicit?
snprbob86 15y ago
So there are some parts of REST that make perfect sense to me. In particular, the preference for nouns w/ CRUD over verbs has many great side effects: easier caching, better logging, easier discoverability, etc. And I also get the Content-Type and Content-Language stuff, especially in terms of avoiding ".json" or ".xml" so that you can compare resource identity via a simple string compare. But I simply do not see any value in HATEOAS outside of largely read-only datasets and generic dataset explorer type applications. Maybe it makes sense for someone like Freebase, but it's completely useless for pretty much every other API out there. You simply cannot build a useful API client application without deep knowledge of the problem domain and the interface part of API. You're going to have API documentation and you're going to have to read it. Now, I understand the desire to avoid IDs and manual URL construction. That's a valuable goal. And I'll admit that I never thought of using 201 and the Location header on create; clever. But just knowing the list of relative URLs from a resource is useless. It's not like a Link rel="newcomment" header is going to show up and magically you'll have comment form. Besides, you need to know which "rel" to lookup, so you might as well just append "/comments" and avoid the indirection. And this all breaks down yet again when you get to offline support. If you've got a web app which is going to deal with not-yet-saved objects, you're back to being unable to compare URLs, or constructing them. Lastly, while I like working with clean URLs and GET/POST over RPC calls. I dislike the ad-hoc specifications necessary to build real applications. We've got a "RESTful" API for our app, but we keep running into situations where different views need subtly different data. For example, decorating a resource with relationship to the current user (eg. isAdmin) or joining data when returning a list of related objects (eg. members vs memberships). The query param spaghetti is growing unwieldy, subtle authorized data leak problems are an inevitability, client-side models get confusing and easily create bugs if passed around. The only solutions to these problems are excessive discipline. Discipline is something that compilers are great at providing, which is why you see things like ProtoBufs and Thift. There's no arguing over HATEOAS or RESTfulness or GET/POST or Content-Type or any of that. The message definition files act as a baseline API documentation, which are enforced programmatically. The designers of these tools had things to do and didn't have time to deal with this nonsense. Stop the pontificating and get back to work.
- masklinn 15y ago
KevBurnsJr 15y ago
REST is a style, not a pattern. The application of this manner of classification to application architectures on the web was the broader goal of Fielding's dissertation, titled "Architectural Styles and the Design of Network-based Software Architectures". See chapter 1 http://www.ics.uci.edu/~fielding/pubs/dissertation/software_arch.htm#sec_1_5 http://www.ics.uci.edu/~fielding/pubs/dissertation/software_...
extension 15y ago
Can clients of your API talk to other people's APIs using the same interface? If not, then the client is obviously coupled to your server and you definitely do not have a REST application. You have a thick client application. REST is the antithesis of this -- the client is generic and doesn't need to change along with the server. Putting links in your proprietary data format does not make it hypermedia, it just makes your API easy to reverse engineer. Maybe it also allows you to change your URLs, but it doesn't allow you to change the structure of your data. Hypermedia does. Hypermedia comes in generic media formats that clients know what to do with. If your API is called "The [company] API" then it is almost certainly not RESTful. If your API is called "[generic type of data] interchange protocol" then it might be RESTful. But we don't usually call that kind of thing an API, we call it a protocol or a format. Really, I don't see how an API, as they are commonly understood, can possibly be RESTful. The main big important point of REST is to not have APIs.
- lautis 15y ago
  REST known by everyone and what Roy Fielding described are quite different beasts. There should be a different name for "plain old HTTP APIs" using JSON or XML. http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hyperte...
rjd 15y ago
Recently I decided to use the ASP.Net MVC framework for a project and tried to combine the API into the web site project for ease. It became quick to me all the mentions of REST within the framework where fictitious. What it actually is an object API exposed via HTTP, its not REST at all. Thinking I mis-understood what REST was I started doing some research only to discover than no I was correct in my understanding (from white papers) and secondly almost ever single developer article I read (blogs) was wrong. There is a fundamental misunderstanding of what REST is out there. Its kind of disheartening to see how many people just don't get it, and are perpetuating falsehoods :/ ASP.net's main problem is technical. (and I assume this is the same for many languages) You can't have functions with the same parameters. You can't have a separate endpoint methods for POST, GET, PUT, DELETE etc... So if you try to shunt data objects around using it you can't just put them back where you got, of do smart discovery, to things i like from REST. So the whole framework falls to pieces, its designed to mimic REST, but not be REST. For .net devs reading this I'd avoid the MVC framework for REST. But if you have to for what ever reason you will have to build your own "verb" dispatchers, and drop the use of "action" functions in your controller classes, only have one public method, the standard Index() one, expose nothing else. I may float my samples online at some point but I'm to busy at the moment so I hope this is enough to help.
- rodh257 15y ago
  You can't have functions with the same parameters, but ASP.NET MVC gives you a way around this using attributes. See this SO question. http://goo.gl/wJ4jE http://goo.gl/wJ4jE Does that solve what you were talking about?
  - sk5t 15y ago
    Seconded - grandparent post is way off with regard to ASP.NET MVC. See all the concrete classes of ActionMethodSelectorAttribute.
  - rjd 15y ago
    Man you're a life saver, I spent an entire afternoon researching this, and finally gave up after dozens of blogs said it was impossible. Infact an article on stack overflow explained the whole situation as impossible so I gave up researching and wrote my own dispatcher.
sukuriant 15y ago
I don't hate to be the guy to point this out, but I absolutely hated the layout of that website. I am on a 1440x900 screen. I do not want a website to be the god of my computer while I'm reading. This website absolutely did not support scaling on screen. It did not resize when I moved it to half of my screen, and it did not provide me with a scroll-bar at the bottom so that I could adjust my viewing window of the screen to see all of the text per line.
antonyme 15y ago
Well, I for one don't understand REST. In particular, how to implement it in current HTML with current browsers. There's all this nice, theoretical stuff about URI design, HTTP verbs and stuff. So I went to actually implement this the other day, only to find that you can't actually do it properly without hacks, because HTML and browsers only support the GET and POST methods for FORMs. WTF? Can you actually implement a set of CRUD pages for an entity without resorting to hacks like hidden _method fields? Surely REST is intended to be used for more than AJAXy APIs?
- icebraining 15y ago
  You're confusing REST (an architectural style) with HTTP (which defines the verbs/methods). REST isn't only HTTP, and HTTP clients are not only general browsers. But it's true, you can't use the full range of HTTP verbs using normal HTML, and that's a shame. Apparently HTML5 was supposed to support them, but it was removed.

191 points by llambda 15y ago

Any notion of HATEOAS in the GitHub API is purely experimentation. With the exception of the pagination links, the rest of them could change format or be removed at any time (until something is properly documented at http://developer.github.com/ http://developer.github.com/). I don't think the Link header is descriptive enough, so most of them will probably go away.

deleted 15y ago
[deleted]

shimonamit 15y ago

Another interesting post by Steve. What are your thoughts on twilio's date prefix in the URI versus using "v1" for example? Does discovery appease the purists qualms about versioning? But if using HATEOS enables discovery (which twilio is doing), why put a date there in the first place? I am thinking they're future-proofing their top-level resource discovery, but maybe I'm missing something.

technoweenie 15y ago
Even with all that in place, it doesn't mean every client is actually going to use it. http://www.w3.org/Provider/Style/URI.html http://www.w3.org/Provider/Style/URI.html
- shimonamit 15y ago
  Yes, but that is yet another good reason not to version.
necubi 15y ago
[Disclaimer: I work for Twilio.] It's actually not future-proofing, in that today we have different version of the API: 2008-08-01 and 2010-04-01. Some sort of versioning is necessary so that we can improve our API without breaking current clients or forcing all of our users to update their code. And while the goal of a completely discoverable API is laudable, I've never seen it work well in practice. We have a real service with real users, and we need something that works well and is simple to consume. And I don't see how date-based versioning is any better than version numbers from a HATEOS perspective, since it's not really any more discoverable. There is however an argument to be made that version numbers are simpler, clearer and easier to remember.
- apaprocki 15y ago
  Isn't it simpler to infer backwards compatibility from proper major.minor version number bumps? A major version number bump indicates API(/ABI) breakage, plain and simple. A minor version bump is simply a backwards-compatible change to the existing major. Version numbers for marketing purposes are not really that useful, but if they are strictly used to indicate compatibility, they are very helpful. No one cares if the API is at, say, version 1023.301 as long as they know that 1023.200 is compatible with the latest version and 1022.499 isn't.
  - johns 15y ago
    (I'm also from Twilio) We historically have not done version bumps for minor improvements to the API that do not break existing applications. 2010-04-01 has been updated a couple times since it was first released with new features (like subaccounts, applications and short codes) but all of those were additive there was no reason to change the version.

icebraining 15y ago

The good GitHub uses custom MIME types for all of their responses. They're using the vendor extensions that I talked about in my post, too. For example: application/vnd.github-issue.text+json Super cool. Hmm, I wouldn't call that "good." It's definitively better than sending 'application/json' or 'application/xml', which tell us nothing about the structure of the data, and it's probably inevitable in their context, but "good" would be to use an actual standard mimetype instead. The problem with using mimetypes tied to the service is that it undermines the concept of Uniform Interface, by forcing developers to write clients specifically for that service. Imagine if instead of standardizing on (X)HTML, CSS, JS and a couple of image formats, each website used their own format.

dasil003 15y ago
Yes but HTML/CSS are general purpose formats. What can be standardized between the wide variety of web APIs out there?
- dreamdu5t 15y ago
  All of these popular services can be expressed in standard ones like HTML/JSON.
  - masklinn 15y ago
    Except as far as an automated tool is concerned, knowing that something is "HTML" or "JSON" is useless: it will not be able to consume it. So it's not a useful information for an API. Instead, specifying the precise subset of the general-purpose format (essentially the language, where the general-purpose format is a meta-language) helps just about everybody. And that's what github does: the `+json` postfix indicates that they're returning a JSON dialect, the `vnd.github-issue.text` explains what that dialect precisely is.
- icebraining 15y ago
  I'm not saying that there should be "one format to rule them all" or that in many cases, having a custom format isn't inevitable, but there are a lot of standard formats that can be reused. And you can use a mix of standard and custom formats for different resources. In this case, I think the best solution would be to develop a standard "issue format" that would be used by all bug trackers, instead of each having its own. In another example, the "Users" part of the API could very well reuse one of the existing formats. This also circles back to a (controversial) opinion I have, which is that JSON isn't the best solution for most of these services. With all of its flaws, XML has features like namespaces which enable a better reuse of standard formats - in the "Issues" example, where there is a "user" key, an XML document could integrate a foaf:Person element which is standardized.

deleted 15y ago

[deleted]

dreamdu5t 15y ago

Vendor mimetypes are bad. They defeat the purpose of REST. I should be able to request from Github the content-type(s) I am willing to accept, and Github serves them to me. It's ironic that a blog post about people understanding REST highlights their misunderstanding of it. What Github does isn't bad, but you shouldn't praise it for being RESTful.

ddagradi 15y ago
This is not about requesting the content type though, it's about requesting the format of the content. If I make a request to https://api.github.com/users/dominic/gists https://api.github.com/users/dominic/gists, I'm getting back JSON, since that's all Github's API supports. Github's custom mime types allow you to request the content of the response in a specific format. In their case, either in Github Flavored Markdown, text, html, or all of the above. They allow you to do this with an Accept header, but the same could be accomplished with a GET parameter tacked onto the request (but that's not RESTful at all!). Using HTTP headers appropriately feels like exactly the right way to handle this situation.
masklinn 15y ago
> Vendor mimetypes are bad. They defeat the purpose of REST. No they don't. Most standard mimetypes are far too broad and completely useless as an actual content type. "application/json" for instance does not tell you anything about the content you're getting, apart from the meta-format in which that content is expressed. Useless.
- KevBurnsJr 15y ago
  It tells you it's not HTML. That's useful if you hope to parse the message body.
  - reinhardt 15y ago
    Ok, so you can "parse" it, which is just a way of saying you can transform a bunch of bytes to a nested structure consisting of a few basic types (dicts, lists, strings, floats, etc). Now what? What can you (or actually your program, not you as a human) do with it without some sort of schema, explicit or implicit?

snprbob86 15y ago

So there are some parts of REST that make perfect sense to me. In particular, the preference for nouns w/ CRUD over verbs has many great side effects: easier caching, better logging, easier discoverability, etc. And I also get the Content-Type and Content-Language stuff, especially in terms of avoiding ".json" or ".xml" so that you can compare resource identity via a simple string compare. But I simply do not see any value in HATEOAS outside of largely read-only datasets and generic dataset explorer type applications. Maybe it makes sense for someone like Freebase, but it's completely useless for pretty much every other API out there. You simply cannot build a useful API client application without deep knowledge of the problem domain and the interface part of API. You're going to have API documentation and you're going to have to read it. Now, I understand the desire to avoid IDs and manual URL construction. That's a valuable goal. And I'll admit that I never thought of using 201 and the Location header on create; clever. But just knowing the list of relative URLs from a resource is useless. It's not like a Link rel="newcomment" header is going to show up and magically you'll have comment form. Besides, you need to know which "rel" to lookup, so you might as well just append "/comments" and avoid the indirection. And this all breaks down yet again when you get to offline support. If you've got a web app which is going to deal with not-yet-saved objects, you're back to being unable to compare URLs, or constructing them. Lastly, while I like working with clean URLs and GET/POST over RPC calls. I dislike the ad-hoc specifications necessary to build real applications. We've got a "RESTful" API for our app, but we keep running into situations where different views need subtly different data. For example, decorating a resource with relationship to the current user (eg. isAdmin) or joining data when returning a list of related objects (eg. members vs memberships). The query param spaghetti is growing unwieldy, subtle authorized data leak problems are an inevitability, client-side models get confusing and easily create bugs if passed around. The only solutions to these problems are excessive discipline. Discipline is something that compilers are great at providing, which is why you see things like ProtoBufs and Thift. There's no arguing over HATEOAS or RESTfulness or GET/POST or Content-Type or any of that. The message definition files act as a baseline API documentation, which are enforced programmatically. The designers of these tools had things to do and didn't have time to deal with this nonsense. Stop the pontificating and get back to work.

masklinn 15y ago

KevBurnsJr 15y ago

REST is a style, not a pattern. The application of this manner of classification to application architectures on the web was the broader goal of Fielding's dissertation, titled "Architectural Styles and the Design of Network-based Software Architectures". See chapter 1 http://www.ics.uci.edu/~fielding/pubs/dissertation/software_arch.htm#sec_1_5 http://www.ics.uci.edu/~fielding/pubs/dissertation/software_...

extension 15y ago

Can clients of your API talk to other people's APIs using the same interface? If not, then the client is obviously coupled to your server and you definitely do not have a REST application. You have a thick client application. REST is the antithesis of this -- the client is generic and doesn't need to change along with the server. Putting links in your proprietary data format does not make it hypermedia, it just makes your API easy to reverse engineer. Maybe it also allows you to change your URLs, but it doesn't allow you to change the structure of your data. Hypermedia does. Hypermedia comes in generic media formats that clients know what to do with. If your API is called "The [company] API" then it is almost certainly not RESTful. If your API is called "[generic type of data] interchange protocol" then it might be RESTful. But we don't usually call that kind of thing an API, we call it a protocol or a format. Really, I don't see how an API, as they are commonly understood, can possibly be RESTful. The main big important point of REST is to not have APIs.

lautis 15y ago
REST known by everyone and what Roy Fielding described are quite different beasts. There should be a different name for "plain old HTTP APIs" using JSON or XML. http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hyperte...

rjd 15y ago

Recently I decided to use the ASP.Net MVC framework for a project and tried to combine the API into the web site project for ease. It became quick to me all the mentions of REST within the framework where fictitious. What it actually is an object API exposed via HTTP, its not REST at all. Thinking I mis-understood what REST was I started doing some research only to discover than no I was correct in my understanding (from white papers) and secondly almost ever single developer article I read (blogs) was wrong. There is a fundamental misunderstanding of what REST is out there. Its kind of disheartening to see how many people just don't get it, and are perpetuating falsehoods :/ ASP.net's main problem is technical. (and I assume this is the same for many languages) You can't have functions with the same parameters. You can't have a separate endpoint methods for POST, GET, PUT, DELETE etc... So if you try to shunt data objects around using it you can't just put them back where you got, of do smart discovery, to things i like from REST. So the whole framework falls to pieces, its designed to mimic REST, but not be REST. For .net devs reading this I'd avoid the MVC framework for REST. But if you have to for what ever reason you will have to build your own "verb" dispatchers, and drop the use of "action" functions in your controller classes, only have one public method, the standard Index() one, expose nothing else. I may float my samples online at some point but I'm to busy at the moment so I hope this is enough to help.

rodh257 15y ago
You can't have functions with the same parameters, but ASP.NET MVC gives you a way around this using attributes. See this SO question. http://goo.gl/wJ4jE http://goo.gl/wJ4jE Does that solve what you were talking about?
- sk5t 15y ago
  Seconded - grandparent post is way off with regard to ASP.NET MVC. See all the concrete classes of ActionMethodSelectorAttribute.
- rjd 15y ago
  Man you're a life saver, I spent an entire afternoon researching this, and finally gave up after dozens of blogs said it was impossible. Infact an article on stack overflow explained the whole situation as impossible so I gave up researching and wrote my own dispatcher.

sukuriant 15y ago

I don't hate to be the guy to point this out, but I absolutely hated the layout of that website. I am on a 1440x900 screen. I do not want a website to be the god of my computer while I'm reading. This website absolutely did not support scaling on screen. It did not resize when I moved it to half of my screen, and it did not provide me with a scroll-bar at the bottom so that I could adjust my viewing window of the screen to see all of the text per line.

antonyme 15y ago

Well, I for one don't understand REST. In particular, how to implement it in current HTML with current browsers. There's all this nice, theoretical stuff about URI design, HTTP verbs and stuff. So I went to actually implement this the other day, only to find that you can't actually do it properly without hacks, because HTML and browsers only support the GET and POST methods for FORMs. WTF? Can you actually implement a set of CRUD pages for an entity without resorting to hacks like hidden _method fields? Surely REST is intended to be used for more than AJAXy APIs?

icebraining 15y ago
You're confusing REST (an architectural style) with HTTP (which defines the verbs/methods). REST isn't only HTTP, and HTTP clients are not only general browsers. But it's true, you can't use the full range of HTTP verbs using normal HTML, and that's a shame. Apparently HTML5 was supposed to support them, but it was removed.