32 ms·

We have broken HTTP

373 points by nkurz 12y ago

tekacs 12y ago
> Anyone working in this space 20 years ago couldn’t possibly have known of their problems so every problem deserves a new solution. It's worth noting that sometimes the reason for developing a new solution is to be free of the baggage that comes with all the failed attempts on the old infrastructure... e.g. JSON-based solutions over the many attempts to use XML (XML is a capable base, but many find it unwieldy to say the least)
- csirac2 12y ago
  Well, I'm sure the creators of JSON weren't thinking of XML use-cases at the time. When it comes to solution-agnostic, self-describing, self-validating semantic markup, I'm not sure JSON-LD buys you much over RDF serialized to XML (for example).
  - erikpukinskis 12y ago
    I can speak for myself and say XML died for me when it was decided that the standard interface to it would be stream parsing. I found that stuff incomprehensible. I get the perf benefits, but it was always premature optimization for me. You throw a JSON string at a simple parser and get an object back. That's why it won a place in my heart. Nothing prevented xml from being that except that's just not what the lib authors did. There was so much talk about XML being the One True Data Structure I think it ended up being a jack of all trades, master of none.
    - lmm 12y ago
      The problem with XML was that people tried to make it semantic. You had XML Schema (awful), and because of schema you had to have namespacing (also awful). Take that away - plain unnamespaced tags, with no semantics beyond "this is a character sequence", common-sense business-specific tag names, and no schema - and you get something that's just as nice to work with as JSON is. And there are plenty of libraries (e.g. Jackson) that will serialize back and forth to objects. But too many people drank the semantic kool-aid and tried to autogenerate classes from schemata or other such silliness.
deleted 12y ago
[deleted]
ctz 12y ago
Google cache: http://webcache.googleusercontent.com/search?q=cache:0PUMGvUWC70J:gadgetopia.com/&hl=en&gl=uk&strip=1 http://webcache.googleusercontent.com/search?q=cache:0PUMGvU... (Server is not responding at the time of writing, from two locations.)
- 3princip 12y ago
  Well, he does claim that we have broken HTTP. :) I think the situation is improving and serious developers are using a broader range of HTTP features. Proper status code and verb usage has become very common with the proliferation of RESTful APIs.
savanaly 12y ago
Web developer for less than a year here, so just offering my perspective on what "new web devs" know these days. I know there are status codes beyond 200 and 404, and work with maybe a dozen in regular rotation. I know there are probably hundreds more, but until they are in more regular use I think it would just inhibit clarity to use them (just like how using less precise language paradoxically makes your prose clearer sometimes). This is sort of a chicken and the egg problem, but the paradigm could definitely shift if the marginal usefulness is high enough. URN's I had not heard of, but the article didn't make it clear to me what problem they would solve. Of course I know about HTTP verbs PUT and DELETE and HEAD-- is there really any web dev who doesn't? And I am pretty rigorous about using the appropriate one (unlike with status codes). I don't have an opinion on how apps are isolated from the web and don't have interlinks between them. Don't really use mobile apps myself except utility ones like Spotify and Netflix.
- eropple 12y ago
  > URN's I had not heard of, but the article didn't make it clear to me what problem they would solve. The idea of a globally unique, permanent identifier that would resolve to a URL to indicate the current location of a resource is fairly commonplace, isn't it? We've tried to bolt all sorts of equivalents onto the existing URL infrastructure without them.
- regularfry 12y ago
  Do you know where to find the canonical list of HTTP status codes? If the answer's "yes", you're probably OK.
  - spb 12y ago
    Yes... Wikipedia.
    - pluma 12y ago
      To the downvoters: sure, Wikipedia is not the canonical source (that would be IANA's list[0]). But Wikipedia mostly gets it right and also lists some of the "non-standard but can be encountered in the wild" ones. I actually prefer the Wikipedia page[1] exactly because it includes more than just the canonical subset. For example, 451 is not yet official, but pretty much as close as you can get to a standard without having an actual standard (well, there's a draft). [0]: http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml http://www.iana.org/assignments/http-status-codes/http-statu... [1]: http://en.wikipedia.org/wiki/List_of_HTTP_status_codes http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
dalke 12y ago
Was HTTP ever non-broken? I remember trying hard to get URNs to work, and proposed them for one project. They required a URN resolver, which is entire new set of infrastructure. I tried using path segment parameters (I believe the author refers to syntax like 'http://a/b/c/g;x=1/y' http://a/b/c/g;x=1/y' rather than '/key1:value1/key2:value2/'.) At the time (mid-1990s) the tools I used didn't support it, and I never came up with a need for them. Still haven't. The current spec (ftp://ftp.rfc-editor.org/in-notes/rfc3986.txt) says that the text after a ';' is a "non-zero-length segment without any colon" and that the contents are "considered opaque by the generic syntax." "[I]n most cases the syntax of a parameter is specific to the implementation of the URI's dereferencing algorithm." Regarding "404 (traditionally “Not Found”) means it was never here. 410 (traditionally “Gone”) means it was once here but is now gone"; the spec says: > [404] The server has not found anything matching the Request-URI. No indication is given of whether the condition is temporary or permanent. The 410 (Gone) status code SHOULD be used if the server knows, through some internally configurable mechanism, that an old resource is permanently unavailable and has no forwarding address. This status code is commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable. It doesn't say anything about 404 meaning "was never here", and in fact includes the possibility that perhaps the server doesn't want to give more information.
- Jabbles 12y ago
  And indeed the spec for 403 (Forbidden) includes 404 as a valid return code if you do not wish to reveal the fact that there's anything there: An origin server that wishes to "hide" the current existence of a forbidden target resource MAY instead respond with a status code of 404 (Not Found).
  - chris_wot 12y ago
    Which is precisely the default behaviour of IIS.
ekimekim 12y ago
I've heard a couple of times that requests coming from JS "need to get a 200" in order to see the response data at all. Now, I've never worked with JS myself, but to me this sounds more like a badly-designed library problem, but it keeps becoming an issue every time I push for an API without broken error codes.
- adregan 12y ago
  This is not the case with the great library superagent[0], which I can confirm with a little test case I wrote. I have a server that returns a 401 when the Authorization header is missing, but I also like to return a JSON body with a little more information as to what went wrong (ie. the header is flat out missing or the credentials aren't being sent in the correct way, with a link to the documentation), and superagent allows you to output the `response.body` in the `end()` method callback. I think of superagent as the requests[1] of js. In fact, I usually require it under the name requests as they feel so similar. 0: https://github.com/visionmedia/superagent https://github.com/visionmedia/superagent 1: http://docs.python-requests.org/en/latest/ http://docs.python-requests.org/en/latest/ EDIT: link formatting
- deleted 12y ago
  [deleted]
- rmrfrmrf 12y ago
  Just looked this up and you're right: in Firefox 3.6, there's a bug where the XMLHttpRequest response body won't be populated by Firefox if the status code is 400. Seems like a little-known bug that has been vaguely referenced elsewhere. http://bugs.jquery.com/ticket/7868 http://bugs.jquery.com/ticket/7868 http://stackoverflow.com/questions/5884286/jquery-ajax-with-non-200-responses http://stackoverflow.com/questions/5884286/jquery-ajax-with-...
  - umurkontaci 12y ago
    That's a regression from 4 years ago. You can get the response body in any status code in JavaScript.
- 12y ago
nimnam1 12y ago
This is just symptomatic of a saturated market. Between Google and apple there are over 2.5 million apps and probably over a billions website or something like that, all of which are using HTTP(S). On top of that, the market for developers is exceptional, because of this people are flocking to it by the minute. They are all excited to build a bridge without taking a class in engineering.
ChuckMcM 12y ago
I had to chuckle at this, its true of course that http as a standard is greatly abused and it is one of the big reasons that writing a web crawler is not just a matter of calling GET again and again. In providing APIs for third parties that are rate limited, our APIs return a '509' error to tell you that you've exceeded your query rate. Which if you just sleep for a bit and then send a request you'll get an answer, but no, as a 'soft' error too many developers just retry it immediately which is useless. Some folks want only 3 errors, 'ok', 'dead', or 'try again' and can't deal with any more subtlety than that.
rhapsodyv 12y ago
In my experience as interviewer, the major part of web developers doesn't know nothing besides URL in the address bar is a GET and form submission "must" be POST... Headers? What is this?! It's sad...
- danellis 12y ago
  Who's supposed to be teaching them, though? If they don't know this stuff exists, how are they going to know to look for it? I learned it because I was always a complete RFC nerd, but I know that many developers are not like that.
  - eyko 12y ago
    Any web developer that is writing not just front-end code but also backend services or an APIs should have come across HTTP status codes and know the reason for their existence. I came across this quite early in my programming career when writing a very basic http service (it's not that long ago, I've been programming for about 10 years) and the first thing you learn are that your HTTP response will usually have headers, a body, and a status code. In my experience, though, when working as a front end developer I've had to deal with developers (mostly in the .Net camp) who will gladly return errors with 200 OK and an xml/json object with the typical "StatusCode: xxx", where XXX is some made up error code that you'll need to reference in their application. I believe this is because many backend developers are in web development by accident, and don't really care much about the web as a platform.
- eyko 12y ago
  I hope you're wrong because I find it extremely discouraging that a major part of web developers would know nothing (or at least so little) about HTTP methods. For now, I'll take your post as anecdotal evidence until a proper survey has been done...
Yadi 12y ago
Amen to this: "My point is that a lot of web developers today are completely ignorant of the protocol that is the basis for their job."
- benihana 12y ago
  What a ridiculous statement. Forgetting about the false hyperbole of "completely ignorant," HTTP is no more the basis of my job than the mixing of asphalt is the basis for a trucker's job. The basis of my job is making electronic tools for people to help them accomplish tasks. I would do that with or without HTTP (which I'm knowledgeable in thanks).
  - danellis 12y ago
    It's more like a trucker that only knows a couple of basic road signs.
byuu 12y ago
The current state of the web is the result of too many cooks in the kitchen. Nobody is ever willing to stand up and say no to countless new additions that often simply replicate old behavior, or don't add anything compelling or likely to ever be used by others. We even have outright gags in the official specs, like the 418 response. It's already too late to make HTTP something sane. What we need are developers who understand and appreciate simplicity and minimalism, to make something awesome with a chance of actually catching on. (Please don't xkcd/927 me.) And they'll have to be fierce in protecting it against third-party extensions (which may not even be possible with something this popular.) That's unlikely to happen, so I guess we'll just keep playing this perverted game of Jenga ad infinitum.
- nattaylor 12y ago
  Ha, I had to look up 418. 418 I'm a teapot (RFC 2324) This code was defined in 1998 as one of the traditional IETF April Fools' jokes, in RFC 2324, Hyper Text Coffee Pot Control Protocol, and is not expected to be implemented by actual HTTP servers. http://tools.ietf.org/html/rfc2324 http://tools.ietf.org/html/rfc2324
  - DanWaterworth 12y ago
    You might also appreciate: http://en.wikipedia.org/wiki/IPoAC http://en.wikipedia.org/wiki/IPoAC
  - ddoolin 12y ago
    Sorta related, The HTTP2 Error Code of ENHANCE_YOUR_CALM strikes me as a joke because of the name... http://http2.github.io/http2-spec/index.html#ErrorCodes http://http2.github.io/http2-spec/index.html#ErrorCodes A quick Google shows that it's previously just been used by Twitter to demonstrate rate limiting.
shortstuffsushi 12y ago
I largely understand what he's saying, in terms of misuse of the protocol. I think you're going to find those sorts of things being the case for any toolset, though. Given enough time, and enough developers, you'll eventually run into people using it an entirely different way than you initially intended, regardless of the specifications or documentation written around it. One thing I'm not sure I agree with, though, is the mobile application "linking" he mentions. When have non-web applications ever linked? What of all of the old "desktop applications" that everyone used to use? Those don't typically link, why is there a different expectation for mobile applications? Oh, and most mobile apps do support linking (in some sense, though it's not really a primary feature). Perhaps I'm misunderstanding what he means with this.
- umurkontaci 12y ago
  I think he is referring to apps that only show content, like a news-site reader or something along the lines of that. However, most of the apps I've seen have a Share feature which one can use to retrieve the link.
- maephet 12y ago
  I work at Branch (referenced in the NYT article) which tries to make it way easier for mobile devs to leverage better deep links to stuff in their apps. Making the analogy to desktop apps is fair, but I think the better question would be why not? The URI scheme registration mechanism in Android and iOS allow the dev to create a new OS level namespace for his or her app and all 'pages' inside. For example, http:// http:// is the scheme for browser app based addresses just as venmo:// is the scheme to address all payments or user profiles in the Venmo app. Why not leverage this technology to make app content more open and easier to access? Now, if we could just get rid of the app store enforced install process and have apps automatically retrieved and cached, things would be a lot smoother...
  - TeMPOraL 12y ago
    > Now, if we could just get rid of the app store enforced install process and have apps automatically retrieved and cached, things would be a lot smoother... I totally see what you did there ;). http://xkcd.com/1367/ http://xkcd.com/1367/ I feel the lack of linking in apps is done partially because of laziness, and partially on purpose. Most apps that are neither games nor utilities seem to be made to control the viewing experience and/or help the authors trick the users into parting with their money. Increasing interoperability seems to be counterproductive if the only reason for your app to exist is to earn a quick buck from the less savvy users.
wtbob 12y ago
This intrigued me: > Don’t even get me started about URL parameters. No, not querystrings – there was originally a specification where you could do something like “/key1:value1/key2:value2/” to pass data into a request. I couldn't find a specification, but I did find [1] which led to [2], both of which indicate that there's an idea of parameters within the URL hierarchy itself. It seems to me that it could be useful for API versioning at least. As to the article itself, I completely agree. The problem is, though, that there's just not enough incentive for folks to actually use the Web standards suite the way (I want to believe) it was meant to be used. How much would cost to make a fully-semantic, JavaScript-extended (but not -requiring), machine-usable website which is also adaptive and beautiful and appealing to humans? And how much benefit would one realise? Rather than do things the Right Way, folks just hack through and do it A Way, and get on with life. I wish that it weren't so, I truly do, but it is. At this point, railing against it feels like railing against the tides. [1] http://doriantaylor.com/policy/http-url-path-parameter-syntax http://doriantaylor.com/policy/http-url-path-parameter-synta... [2] http://tools.ietf.org/html/rfc3986#section-3.3 http://tools.ietf.org/html/rfc3986#section-3.3
- waxjar 12y ago
  API versioning should be done with a HTTP Header.
djur 12y ago
Most of the good stuff here (using the full range of HTTP verbs and status codes, idempotent GET, stable URLs) is already common knowledge among web developers and built into any modern framework. URNs are interesting but difficult to work with in practice; 402 Payment Required has been "reserved for future use" forever. Then there's the suggestion to use custom HTTP methods, which is ill-advised [1]. [1]: http://programmers.stackexchange.com/a/193826 http://programmers.stackexchange.com/a/193826
cozuya 12y ago
> Do you know why your FORM tag has an attribute called “method”? Because you’re calling a method on a web server, like a method on an object in OOP. Did you know there are other methods besides GET and POST? There’s HEAD and OPTIONS and PUT and DELETE. Except HTML(5) forms do not support any methods other than GET and POST.
- visionscaper 12y ago
  I always found this extremely odd. Does anyone have a satisfying answer as to why this is the case?
  - jandrese 12y ago
    Because nobody uses them and supporting them can cause more problems than they are worth. The author was ranting about unused features, but not all features are good ideas. Sometimes things that look good on paper don't work so well in practice.
    - cozuya 12y ago
      Its not a good idea for HTML to support REST?
      - olavk 12y ago
        HTML is designed for REST. Which is why a form does not support arbitrary HTTP methods, but only the methods which semantically makes sense for a form.
      - lmm 12y ago
        REST-the-actual-paper is a monstrosity that, thankfully, no-one actually uses. "REST" as a name for the practice of using simple JSON and using existing HTTP features (methods, status codes) rather than using HTTP as a dumb transport layer is... an ok idea, but overhyped. The supported HTTP vocabulary is very limited, and adding new verbs like PATCH is a multi-year process that's never going to be well supported by every existing piece of software. So sooner or later you're going to have to layer a more complicated (business-specific) semantics over HTTP, and have e.g. a POST that actually has semantic meaning beyond being a POST. Given that, if you just used POST for everything, and had a simple encoding of method kinds and error responses over that... would that be so bad? Those who forget SOAP are doomed to recreate it, badly. I worry that things like JSON-schema are headed in that direction.
Animats 12y ago
We suck at error reporting to the user. One of the legacy curses of UNIX is trying to encode all error info in "errno". Then came HTTP, which most languages try to treat as something like a file access. When there's some complex problem at the remote end, the error presented to the user has often been hammered down to some errno value that's vaguely appropriate. This came up recently on HN in connection with GoGo WiFi on airliners, using satellite links. GoGo doesn't have the satellite bandwidth to let users view streaming video, so they block YouTube and some other sites. The problem is telling the user what's going on. They used an awful hack involving a self-signed fake SSL certificate for YouTube to try to get a coherent message to the user. According to the IP standard, they should send back an ICMP Destination Unreachable message with a code 13, "Communication administratively prohibited". Most IP/TCP/HTTP/browser stacks won't get that info all the way up to the end user, especially the "Communication administratively prohibited" part. Even if the error code made it to the user, many customers would not interpret "Communication administratively prohibited" as "No, you can't watch Youtube because you're on an airplane and we don't have the bandwidth for that".
- jdmichal 12y ago
  Why not just reroute those addresses via proxy to a static page explaining?
  - eropple 12y ago
    They're under SSL. All a reroute would do is pop up Chrome's bad cert page.
  - simoncion 12y ago
    Hell, it would have been far better to use traffic shaping to transfer at (like) 1bps than to do anything else.
- simoncion 12y ago
  > According to the IP standard, they should send back an ICMP Destination Unreachable message with a code 13, "Communication administratively prohibited". Agreed. However, I've discovered that even folks who really should know better don't know what to do with anything other that port-unreachable: In the past, I tried setting my ip[6]tables REJECT targets to send back admin-prohibit. For... reasons[0], I was REJECTing IPv6 traffic to Google's networks. I discovered that YouTube on a Nexus device running Android Jellybean would not fail over to IPv4 but would, instead, sit and spin for multiple minutes before reporting a fatal connection failure. [0] At the time I had a IPv6 tunnel that (for some reason) transferred packets destined for Google very, very slowly.
gaius 12y ago
As a protocol, it works perfectly well for transporting hypertext, what's the problem?
- M2Ys4U 12y ago
  Implementations.
andrewstuart2 12y ago
For the most part, I say "1000 times yes!" I love the general idea he expresses that developers MUST take the time to understand the giants on whose shoulders they stand. To that end, I have one noteworthy point (more of a clarification) I think is worth making: > And, technically, you’re supposed to make sure GET requests are idempotent, meaning they can be repeated with no changes to a resource. So you should be able to hit Refresh all day on a GET request without causing any data change. I think it's important to note here that idempotence does not mean that the resource will never change. It means that the resource will not change as a result of a get request (barring some meta requests, for example `GET /my-last-http-request`). A resource can definitely change. I think the author would agree with me but found his language there a bit ambiguous. Allow me to quote the good reverened himself, Dr. Roy Fielding: > "More precisely, a resource R is a temporally varying membership function MR(t), which for time t maps to a set of entities, or values, which are equivalent. The values in the set may be resource representations and/or resource identifiers." (emphasis mine) [1] Immediately prior this quote he references "the weather for Los Angeles today" as an example of a resource complete with its own resource identifier (perhaps GET /weather/la/today, most-to-least specific). The contents at the end of that URI will most certainly change, but they shouldn't ever change based on the request. [1] Section 5.2.1.1 https://www.ics.uci.edu/~fielding/pubs/dissertation/fielding_dissertation.pdf https://www.ics.uci.edu/~fielding/pubs/dissertation/fielding...
- teilo 12y ago
  Yes, thank you. Every framework I have ever used maintains this distinction between GET and POST. It's common knowledge.
- spb 12y ago
  To hit the most important point of what you mean by "change based on the request", this doesn't mean "changing your request changing the resource you get" (which wouldn't make any sense). It means that the act of GETting the weather right now will never change the weather. (If you want to change the weather via HTTP, you'll need to do it with a method like "POST /weather/hill-valley" and a body like '{"rain": "stop"}'.) Think of it like a getter function that gets syntactically hidden by the language: you wouldn't want "if (user.accesses < limit || user.accesses > threshold )" to have two different values for "user.accesses" based on the number of times you referred to it. In a less ludicrous example, it also means "GET /tokens/reset/dsf46JtCb385PDs2" shouldn't log you in and invalidate the token, something that most password-reset pages fail to comply with.
pixie_ 12y ago
Also see the discussion here - http://www.reddit.com/r/programming/comments/2rzlay/we_suck_at_http/ http://www.reddit.com/r/programming/comments/2rzlay/we_suck_...
craigds 12y ago
Sounds like the OP's problem is not so much the protocol itself, as the long tail of developers using it who have no idea what it can actually do. There are developers who do use most/all of HTTP. I'm one. I think I've used most of the obscure features he mentions, including URL parameters and all those status codes (excepting 402 Payment Required, which I've never had a need for, but I do know it exists). Most of the features he mentions aren't very useful for browser interaction, but really shine for APIs. If you're trying to write an HTTP API, and you haven't at least got to grips with all the status codes and methods, you should and take the time to understand them all and when they should be used. It's a real joy to use an API that uses methods and statuses properly. If in doubt, read the Github API docs - their API is (mostly) very good in this regard.
wrs 12y ago
> ...developers tend to think they can solve every problem, and they’re pretty sure that nothing good happened before they arrived on the scene. ... they feel no need to study the history of their technology to gain some context about it. So, so true. In fact, this happens depressingly often when "technology" is defined as narrowly as "the app a developer is working on" and "history" is as short as "a month ago". This has always happened, but it feels like it's gotten worse since "agile" became widespread. As if the principle of YAGNI ("You Aren't Going to Need It") has been turned into a poisonous assumption that nobody before you could have put any thought into the future. Rather than trying to understand the natural extension mechanisms that might already be in place, people just jump in and write more special-case code.
WoodenChair 12y ago
My problem with this article is not the general sentiment that developers should know the history of the technologies they work with, which I think is absolutely right, but instead how the author seems to put HTTP on a pedestal. It just happens to be the protocol TBL & company came up with in the early 90s for this thing that took off beyond anyone's wildest imagination. It's not some perfect work of art that should be revered. Also native apps don't make me sad. They're faster, more featureful, and have more consistent platform specific interfaces than their Web brethren. There are good reasons they exist and consumers prefer them.
- Guthur 12y ago
  I am guess that the apps referred to are the ones which have existed quite happily as a web page before moving to smart phones where for more likely business rather that user experience reasons they became apps. The ones I can think of are Twitter, Facebook, and LinkedIn. I am certainly not a proponent of web as a platform for everything but there are certainly some things it is rather good at, and making content discoverable in a consistent manner is one of those. If you hid the content in an app it's rarely exposed.
  - TheOtherHobbes 12y ago
    This debate has been going since the days of AOL and CompuServe. As here, it usually misses the differences between public content, private content, and corporately farmed content. Twitter etc are corporately farmed content, and they all provide web and app access. The access method is just a front end. Personal content in apps isn't usually all that interesting. It's hard to think why anyone would care about my Angry Birds scores, or what level I'm on in Candy Crush. (If I played Candy Crush, which I don't.) If you want to make an app that shares content on the web, it's not difficult. I don't see a problem with the fact that it's extra effort, and if you're going to do it you need to justify that effort in some way.
- yawboakye 12y ago
  Agreed. Apps didn't break HTTP. They consume APIs (which can also be used by web applications). Thus, designing the API is what really matters.
james-skemp 12y ago
Wow. Dublin Core. Now that is a name I've not heard since around a decade ago. It is amazing to look back to my own development history and remember that I too used to care about proper status code, using metadata elements, and the like. But after adopting so many, and seeing so many either fail to catch on, or infighting tear them apart, or the big names pick them up and then put them aside ... it's not surprising that current developers don't think about them or don't care.
junto 12y ago
One of my personal pet peeves is the misuse of the 401 Not Authorised status code being issued to logged in users. No, try a 403 instead please.
drcomputer 12y ago
I love implementing stuff like standardizations. Seriously, I love reading documentation that has lists of codes meant to concisely summarize specific frequently occurring events. I am the biggest nerd when it comes to pouring through documentation that was designed specifically with the intent to unify a common need that started off fractured. In general though, it's just hard to know exactly when and where to implement what standardization. It's not just HTTP. The entire world of programming and computer science would be much more elegant with intelligent standardization, but it's literally a hard problem. The source of information is the hard problem. It is subject to change. How can you standardize something that is consistently evolving, evolving to the point that it is expected that the functionality of the system is going to exceed all known expectations of it's operation and behavior? As developers, we are consistently striving to exceed our own boundaries, to do things better, more optimally, more safely, more legibly, more precisely, more correctly. Our expectations of ourselves is what screws us when we think about 'the big picture', because we apply the same model of reasoning to 'the big picture'. Yes, we do not know what we do not know. However, we additionally do not know how to determine the difference between 'that which exists that we do not know' and 'that which does not exist that we do not know'. That is the core problem that lays underneath the lack of implementing standards, and it the reason people standardize things in the first place. Standardization prevents us from re-inventing the wheel. Standardization makes things safe. Standardization makes complex systems predictable even when information about adjacent systems are unknown. Standardization is a vehicle for implementing trust. People get confused because there are too many words for the same thing - too many ways to describe the same things, too many languages to do the same thing, too many explanations meant to teach the same thing. Either we have to learn the hard way (by observing massive failure, or by experiencing it), or we have to hope we are implementing things the most correct way, and trust we will get there.
fndrplayer13 12y ago
I feel like the ignorance of developers here is overstated. Most APIs that I know of make good use of the HTTP status codes. This includes APIs from big players like Google, all the way down to small startup APIs. Sure I've seen bad ones, but I don't know that I would agree that its the majority? Maybe I've just had good luck or lived a blissfully unaware existence as a developer. Hard to say.
nkozyra 12y ago
I'd say the reason why following spec on status codes has fallen out of favor is because browsers don't do anything interesting or informative with them for most users. On APIs, I'd say it's a sin to not return proper HTTP headers, but when the end user on a web site sees a File Not Found that's 200 Status OK versus File Not Found that's 404, it doesn't really matter. And while it matters for things like the Googlebot, over the years developers stopped caring, because delivering an attractive and descriptive status page was more important than delivering an error status and not rendering a page at all. I'm not saying it's right, but it was largely a ornamental step in the early days of the Web, so I understand why it disappeared. If browsers proactively provided data to users about HTTP status codes, I think they'd be adhered to much more.
- seandhi 12y ago
  Returning a 404 status code and displaying an attractive and descriptive status page are not mutually exclusive. If you go to http://www.google.com/unknown http://www.google.com/unknown, you will get a 404, but you will also see a custom page.
  - nkozyra 12y ago
    Absolutely, I don't disagree. I'm saying that for most end users, there's no distinction. Status codes are for machines, but if early browsers did something noteworthy with those codes, developers wouldn't have been so quick to dismiss them in a way that is now largely habit.
olssy 12y ago
HTTP is too flexible not to theoretically break but in practice it can be used and abused and still deliver.
peterkelly 12y ago
Ironically, the URL of the post is http://gadgetopia.com/post/9236?retitle http://gadgetopia.com/post/9236?retitle
- einrealist 12y ago
  And?
  - peterkelly 12y ago
    It doesn't exactly follow the recommendations in the second link from the post: http://gadgetopia.com/post/6346 http://gadgetopia.com/post/6346
    - einrealist 12y ago
      Yes and that is why the linked post and this post itself are bad. Both authors do not understand the principles of an URI in the context of the web. To a client, the URI should be opaque. If a client has to start to make meaning of the contents of an URI, the server loses the ability to change its implementation (or just move resources around). Edit: Ah, its the same author for both blog posts. That explains it.
    - woof 12y ago
      From the same page: (And yes, I know Gadgetopia’s URLs are somewhat lame in this regard. There are reasons why I did it, but changing it now - See more at: http://gadgetopia.com/post/6346#.dpuf http://gadgetopia.com/post/6346#.dpuf)
overgard 12y ago
Maybe the simpler explanation is that HTTP/WWW was always broken. It's never been a great application platform. It's popular and universal and simple (sortof), so that's awesome, but in terms of great tech it's kind of been junk all along. It's not like it ever was universally conformed to. It's not like there was this golden era. What would be the golden era? The web before AJAX? Geocities and Hotmail? Frames broke URLS more than almost anything. PHPNuke forums? Web apps sucked back in the early days. Makes me think of a fun Alan Kay quote: > The Internet was done so well that most people think of it as a natural resource like the Pacific Ocean, rather than something that was man-made. When was the last time a technology with a scale like that was so error-free? The Web, in comparison, is a joke. The Web was done by amateurs. -- Alan Kay.
- woah 12y ago
  Yea, the specification of status codes is cumbersome and annoying to deal with, like you're supposed to go find a list of what they are supposed to be on wikipedia and then match it up to what's going in your application as best you can. Also the PUT, POST etc stuff. It's just a pain to try to match up your application's behavior to a crusty old spec that was never that good. It's really refreshing to code with websockets and just write an application naturally. HTTP is good for GETing static web pages and that's about it.
  - TylerE 12y ago
    Way to oversimplify. HTTP works ok for some super dumb use cases. But pray tell, what's the proper way to update multiple objects at once? Simple verbs work for simple tasks - only. There is more to life than REST.
    - jdmichal 12y ago
      Updating multiple resources of the same type is easy: POST to the plural form of the resource. If you need to update multiple resources of different types atomically, then you probably need a new resource abstraction that covers both of the resources anyway. Then just POST to that.
halayli 12y ago
It sounds like this guy just learnt about HTTP details.
bipin-nag 12y ago
I feel that the article continued in the wrong vein. The original NY times article from which the author takes inspiration talks about how mobile apps are segregated from the web. It says that Google, Facebook and other companies have apps whose content is not part of the web. If they wanted they could have done it. They decided not to because they don't want their content freely accessible. They are more than capable of doing that, Google even spearheading development of HTTP2. The technical challenge of deep linking remains an issue for many others which prevents developers to orgainze app content for web and people from accessing it directly. The article was more about getting the basics of HTTP right.
xsace 12y ago
What a pointless rant. Things evolve, even norms and specs. There's a reason why URNs are not used in the code, there are not enough by themselves, if you need use URIs. There's a reason people don't use every status code in every webapps. It's a waste of time, better spent elsewhere. Most webapps uses more then GET. Etc. Anyway, HTML is not broken, it's used everywhere by billions of users every seconds.
- TheSwordsman 12y ago
  Agreed. This guy needs to take a break from the Internet for a few years.
  - einrealist 12y ago
    Indeed. It is the internet where everybody can write about something, even if he/she is (mostly) wrong. And posting it on HN or Reddit does not make it a better blog entry.
- gtramont 12y ago
  Perhaps if people understood better the technology that allows their business to operate, they wouldn't waste time versioning APIs and having to re-write clients (web, mobile...) every time the server changes something!
  - xsace 12y ago
    You can't expect everyone to master everything. It's like ranting about how the whole classroom haven't got A+. It's pointless ultimately. The real question is how can we improve a protocol so it's better used by most people. Hence why it's evolving. Nothing should be set in stone forever and then expect everyone to comply to it or die.
stevenspins 12y ago
HTTP comprehension and usage are critical. People wonder why websites don't perform and api are difficult to program for. I have seen that developers have this weird need to create your own method and means of communicating errors. Even ws-* developers don't use soap errors. That is what we are talking about mostly in this article non-standard messages/state. Treating your application and data as resources (REST is the style of the web) and leveraging the power of HTTP so powerful. You are not a crabby old man. If you do count me in, 33 years old. I could make a career consulting and only fixing http and resource abstraction/usage problems. IMO though educating our fellow developers on HTTP is easy compared to the REST HATEOAS constraint.
- bluetomcat 12y ago
  > I have seen that developers have this weird need to create your own method and means of communicating errors. Developers ignore most of the status codes because they introduce unnecessary complexity on both ends (client and server) and do not always map well with the application logic. With JSON payloads, HTTP is used as a "light vehicle" to carry the application logic. Since the payload is going to be specially interpreted anyway, I don't see a benefit in cramming legacy HTTP cruft in the headers and in status codes. The really annoying thing is the web server spitting out HTML on error conditions instead of the expected JSON.
iamleppert 12y ago
You know what really bothers me? It's not the abuse of HTTP status codes. It's the closed nature of apps. How many of us got our start in web development by viewing the source of the page? By looking at how others did things? Hell, I still do that now. Apps are the Microsoft of the Internet -- closed, proprietary and secretive. Open source and github has helped this some, but meh.
mark_l_watson 12y ago
I liked the author's comment about using canonical URLs (with a link to another of his articles on this subject). This is something that I have been intending on doing for a while - I just pushed changes supporting canonical URLs to my main 3 web portals.

373 points by nkurz 12y ago

> Anyone working in this space 20 years ago couldn’t possibly have known of their problems so every problem deserves a new solution. It's worth noting that sometimes the reason for developing a new solution is to be free of the baggage that comes with all the failed attempts on the old infrastructure... e.g. JSON-based solutions over the many attempts to use XML (XML is a capable base, but many find it unwieldy to say the least)

csirac2 12y ago
Well, I'm sure the creators of JSON weren't thinking of XML use-cases at the time. When it comes to solution-agnostic, self-describing, self-validating semantic markup, I'm not sure JSON-LD buys you much over RDF serialized to XML (for example).
- erikpukinskis 12y ago
  I can speak for myself and say XML died for me when it was decided that the standard interface to it would be stream parsing. I found that stuff incomprehensible. I get the perf benefits, but it was always premature optimization for me. You throw a JSON string at a simple parser and get an object back. That's why it won a place in my heart. Nothing prevented xml from being that except that's just not what the lib authors did. There was so much talk about XML being the One True Data Structure I think it ended up being a jack of all trades, master of none.
  - lmm 12y ago
    The problem with XML was that people tried to make it semantic. You had XML Schema (awful), and because of schema you had to have namespacing (also awful). Take that away - plain unnamespaced tags, with no semantics beyond "this is a character sequence", common-sense business-specific tag names, and no schema - and you get something that's just as nice to work with as JSON is. And there are plenty of libraries (e.g. Jackson) that will serialize back and forth to objects. But too many people drank the semantic kool-aid and tried to autogenerate classes from schemata or other such silliness.

deleted 12y ago

[deleted]

ctz 12y ago

Google cache: http://webcache.googleusercontent.com/search?q=cache:0PUMGvUWC70J:gadgetopia.com/&hl=en&gl=uk&strip=1 http://webcache.googleusercontent.com/search?q=cache:0PUMGvU... (Server is not responding at the time of writing, from two locations.)

3princip 12y ago
Well, he does claim that we have broken HTTP. :) I think the situation is improving and serious developers are using a broader range of HTTP features. Proper status code and verb usage has become very common with the proliferation of RESTful APIs.

savanaly 12y ago

Web developer for less than a year here, so just offering my perspective on what "new web devs" know these days. I know there are status codes beyond 200 and 404, and work with maybe a dozen in regular rotation. I know there are probably hundreds more, but until they are in more regular use I think it would just inhibit clarity to use them (just like how using less precise language paradoxically makes your prose clearer sometimes). This is sort of a chicken and the egg problem, but the paradigm could definitely shift if the marginal usefulness is high enough. URN's I had not heard of, but the article didn't make it clear to me what problem they would solve. Of course I know about HTTP verbs PUT and DELETE and HEAD-- is there really any web dev who doesn't? And I am pretty rigorous about using the appropriate one (unlike with status codes). I don't have an opinion on how apps are isolated from the web and don't have interlinks between them. Don't really use mobile apps myself except utility ones like Spotify and Netflix.

eropple 12y ago
> URN's I had not heard of, but the article didn't make it clear to me what problem they would solve. The idea of a globally unique, permanent identifier that would resolve to a URL to indicate the current location of a resource is fairly commonplace, isn't it? We've tried to bolt all sorts of equivalents onto the existing URL infrastructure without them.
regularfry 12y ago
Do you know where to find the canonical list of HTTP status codes? If the answer's "yes", you're probably OK.
- spb 12y ago
  Yes... Wikipedia.
  - pluma 12y ago
    To the downvoters: sure, Wikipedia is not the canonical source (that would be IANA's list[0]). But Wikipedia mostly gets it right and also lists some of the "non-standard but can be encountered in the wild" ones. I actually prefer the Wikipedia page[1] exactly because it includes more than just the canonical subset. For example, 451 is not yet official, but pretty much as close as you can get to a standard without having an actual standard (well, there's a draft). [0]: http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml http://www.iana.org/assignments/http-status-codes/http-statu... [1]: http://en.wikipedia.org/wiki/List_of_HTTP_status_codes http://en.wikipedia.org/wiki/List_of_HTTP_status_codes

dalke 12y ago

Was HTTP ever non-broken? I remember trying hard to get URNs to work, and proposed them for one project. They required a URN resolver, which is entire new set of infrastructure. I tried using path segment parameters (I believe the author refers to syntax like 'http://a/b/c/g;x=1/y' http://a/b/c/g;x=1/y' rather than '/key1:value1/key2:value2/'.) At the time (mid-1990s) the tools I used didn't support it, and I never came up with a need for them. Still haven't. The current spec (ftp://ftp.rfc-editor.org/in-notes/rfc3986.txt) says that the text after a ';' is a "non-zero-length segment without any colon" and that the contents are "considered opaque by the generic syntax." "[I]n most cases the syntax of a parameter is specific to the implementation of the URI's dereferencing algorithm." Regarding "404 (traditionally “Not Found”) means it was never here. 410 (traditionally “Gone”) means it was once here but is now gone"; the spec says: > [404] The server has not found anything matching the Request-URI. No indication is given of whether the condition is temporary or permanent. The 410 (Gone) status code SHOULD be used if the server knows, through some internally configurable mechanism, that an old resource is permanently unavailable and has no forwarding address. This status code is commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable. It doesn't say anything about 404 meaning "was never here", and in fact includes the possibility that perhaps the server doesn't want to give more information.

Jabbles 12y ago
And indeed the spec for 403 (Forbidden) includes 404 as a valid return code if you do not wish to reveal the fact that there's anything there: An origin server that wishes to "hide" the current existence of a forbidden target resource MAY instead respond with a status code of 404 (Not Found).
- chris_wot 12y ago
  Which is precisely the default behaviour of IIS.

ekimekim 12y ago

I've heard a couple of times that requests coming from JS "need to get a 200" in order to see the response data at all. Now, I've never worked with JS myself, but to me this sounds more like a badly-designed library problem, but it keeps becoming an issue every time I push for an API without broken error codes.

adregan 12y ago
This is not the case with the great library superagent[0], which I can confirm with a little test case I wrote. I have a server that returns a 401 when the Authorization header is missing, but I also like to return a JSON body with a little more information as to what went wrong (ie. the header is flat out missing or the credentials aren't being sent in the correct way, with a link to the documentation), and superagent allows you to output the `response.body` in the `end()` method callback. I think of superagent as the requests[1] of js. In fact, I usually require it under the name requests as they feel so similar. 0: https://github.com/visionmedia/superagent https://github.com/visionmedia/superagent 1: http://docs.python-requests.org/en/latest/ http://docs.python-requests.org/en/latest/ EDIT: link formatting
deleted 12y ago
[deleted]
rmrfrmrf 12y ago
Just looked this up and you're right: in Firefox 3.6, there's a bug where the XMLHttpRequest response body won't be populated by Firefox if the status code is 400. Seems like a little-known bug that has been vaguely referenced elsewhere. http://bugs.jquery.com/ticket/7868 http://bugs.jquery.com/ticket/7868 http://stackoverflow.com/questions/5884286/jquery-ajax-with-non-200-responses http://stackoverflow.com/questions/5884286/jquery-ajax-with-...
- umurkontaci 12y ago
  That's a regression from 4 years ago. You can get the response body in any status code in JavaScript.
12y ago

nimnam1 12y ago

This is just symptomatic of a saturated market. Between Google and apple there are over 2.5 million apps and probably over a billions website or something like that, all of which are using HTTP(S). On top of that, the market for developers is exceptional, because of this people are flocking to it by the minute. They are all excited to build a bridge without taking a class in engineering.

ChuckMcM 12y ago

I had to chuckle at this, its true of course that http as a standard is greatly abused and it is one of the big reasons that writing a web crawler is not just a matter of calling GET again and again. In providing APIs for third parties that are rate limited, our APIs return a '509' error to tell you that you've exceeded your query rate. Which if you just sleep for a bit and then send a request you'll get an answer, but no, as a 'soft' error too many developers just retry it immediately which is useless. Some folks want only 3 errors, 'ok', 'dead', or 'try again' and can't deal with any more subtlety than that.

rhapsodyv 12y ago

In my experience as interviewer, the major part of web developers doesn't know nothing besides URL in the address bar is a GET and form submission "must" be POST... Headers? What is this?! It's sad...

danellis 12y ago
Who's supposed to be teaching them, though? If they don't know this stuff exists, how are they going to know to look for it? I learned it because I was always a complete RFC nerd, but I know that many developers are not like that.
- eyko 12y ago
  Any web developer that is writing not just front-end code but also backend services or an APIs should have come across HTTP status codes and know the reason for their existence. I came across this quite early in my programming career when writing a very basic http service (it's not that long ago, I've been programming for about 10 years) and the first thing you learn are that your HTTP response will usually have headers, a body, and a status code. In my experience, though, when working as a front end developer I've had to deal with developers (mostly in the .Net camp) who will gladly return errors with 200 OK and an xml/json object with the typical "StatusCode: xxx", where XXX is some made up error code that you'll need to reference in their application. I believe this is because many backend developers are in web development by accident, and don't really care much about the web as a platform.
eyko 12y ago
I hope you're wrong because I find it extremely discouraging that a major part of web developers would know nothing (or at least so little) about HTTP methods. For now, I'll take your post as anecdotal evidence until a proper survey has been done...

Yadi 12y ago

Amen to this: "My point is that a lot of web developers today are completely ignorant of the protocol that is the basis for their job."

benihana 12y ago
What a ridiculous statement. Forgetting about the false hyperbole of "completely ignorant," HTTP is no more the basis of my job than the mixing of asphalt is the basis for a trucker's job. The basis of my job is making electronic tools for people to help them accomplish tasks. I would do that with or without HTTP (which I'm knowledgeable in thanks).
- danellis 12y ago
  It's more like a trucker that only knows a couple of basic road signs.

byuu 12y ago

The current state of the web is the result of too many cooks in the kitchen. Nobody is ever willing to stand up and say no to countless new additions that often simply replicate old behavior, or don't add anything compelling or likely to ever be used by others. We even have outright gags in the official specs, like the 418 response. It's already too late to make HTTP something sane. What we need are developers who understand and appreciate simplicity and minimalism, to make something awesome with a chance of actually catching on. (Please don't xkcd/927 me.) And they'll have to be fierce in protecting it against third-party extensions (which may not even be possible with something this popular.) That's unlikely to happen, so I guess we'll just keep playing this perverted game of Jenga ad infinitum.

nattaylor 12y ago
Ha, I had to look up 418. 418 I'm a teapot (RFC 2324) This code was defined in 1998 as one of the traditional IETF April Fools' jokes, in RFC 2324, Hyper Text Coffee Pot Control Protocol, and is not expected to be implemented by actual HTTP servers. http://tools.ietf.org/html/rfc2324 http://tools.ietf.org/html/rfc2324
- DanWaterworth 12y ago
  You might also appreciate: http://en.wikipedia.org/wiki/IPoAC http://en.wikipedia.org/wiki/IPoAC
- ddoolin 12y ago
  Sorta related, The HTTP2 Error Code of ENHANCE_YOUR_CALM strikes me as a joke because of the name... http://http2.github.io/http2-spec/index.html#ErrorCodes http://http2.github.io/http2-spec/index.html#ErrorCodes A quick Google shows that it's previously just been used by Twitter to demonstrate rate limiting.

shortstuffsushi 12y ago

I largely understand what he's saying, in terms of misuse of the protocol. I think you're going to find those sorts of things being the case for any toolset, though. Given enough time, and enough developers, you'll eventually run into people using it an entirely different way than you initially intended, regardless of the specifications or documentation written around it. One thing I'm not sure I agree with, though, is the mobile application "linking" he mentions. When have non-web applications ever linked? What of all of the old "desktop applications" that everyone used to use? Those don't typically link, why is there a different expectation for mobile applications? Oh, and most mobile apps do support linking (in some sense, though it's not really a primary feature). Perhaps I'm misunderstanding what he means with this.

umurkontaci 12y ago
I think he is referring to apps that only show content, like a news-site reader or something along the lines of that. However, most of the apps I've seen have a Share feature which one can use to retrieve the link.
maephet 12y ago
I work at Branch (referenced in the NYT article) which tries to make it way easier for mobile devs to leverage better deep links to stuff in their apps. Making the analogy to desktop apps is fair, but I think the better question would be why not? The URI scheme registration mechanism in Android and iOS allow the dev to create a new OS level namespace for his or her app and all 'pages' inside. For example, http:// http:// is the scheme for browser app based addresses just as venmo:// is the scheme to address all payments or user profiles in the Venmo app. Why not leverage this technology to make app content more open and easier to access? Now, if we could just get rid of the app store enforced install process and have apps automatically retrieved and cached, things would be a lot smoother...
- TeMPOraL 12y ago
  > Now, if we could just get rid of the app store enforced install process and have apps automatically retrieved and cached, things would be a lot smoother... I totally see what you did there ;). http://xkcd.com/1367/ http://xkcd.com/1367/ I feel the lack of linking in apps is done partially because of laziness, and partially on purpose. Most apps that are neither games nor utilities seem to be made to control the viewing experience and/or help the authors trick the users into parting with their money. Increasing interoperability seems to be counterproductive if the only reason for your app to exist is to earn a quick buck from the less savvy users.

wtbob 12y ago

This intrigued me: > Don’t even get me started about URL parameters. No, not querystrings – there was originally a specification where you could do something like “/key1:value1/key2:value2/” to pass data into a request. I couldn't find a specification, but I did find [1] which led to [2], both of which indicate that there's an idea of parameters within the URL hierarchy itself. It seems to me that it could be useful for API versioning at least. As to the article itself, I completely agree. The problem is, though, that there's just not enough incentive for folks to actually use the Web standards suite the way (I want to believe) it was meant to be used. How much would cost to make a fully-semantic, JavaScript-extended (but not -requiring), machine-usable website which is also adaptive and beautiful and appealing to humans? And how much benefit would one realise? Rather than do things the Right Way, folks just hack through and do it A Way, and get on with life. I wish that it weren't so, I truly do, but it is. At this point, railing against it feels like railing against the tides. [1] http://doriantaylor.com/policy/http-url-path-parameter-syntax http://doriantaylor.com/policy/http-url-path-parameter-synta... [2] http://tools.ietf.org/html/rfc3986#section-3.3 http://tools.ietf.org/html/rfc3986#section-3.3

waxjar 12y ago
API versioning should be done with a HTTP Header.

djur 12y ago

Most of the good stuff here (using the full range of HTTP verbs and status codes, idempotent GET, stable URLs) is already common knowledge among web developers and built into any modern framework. URNs are interesting but difficult to work with in practice; 402 Payment Required has been "reserved for future use" forever. Then there's the suggestion to use custom HTTP methods, which is ill-advised [1]. [1]: http://programmers.stackexchange.com/a/193826 http://programmers.stackexchange.com/a/193826

cozuya 12y ago

> Do you know why your FORM tag has an attribute called “method”? Because you’re calling a method on a web server, like a method on an object in OOP. Did you know there are other methods besides GET and POST? There’s HEAD and OPTIONS and PUT and DELETE. Except HTML(5) forms do not support any methods other than GET and POST.

visionscaper 12y ago
I always found this extremely odd. Does anyone have a satisfying answer as to why this is the case?
- jandrese 12y ago
  Because nobody uses them and supporting them can cause more problems than they are worth. The author was ranting about unused features, but not all features are good ideas. Sometimes things that look good on paper don't work so well in practice.
  - cozuya 12y ago
    Its not a good idea for HTML to support REST?
    - olavk 12y ago
      HTML is designed for REST. Which is why a form does not support arbitrary HTTP methods, but only the methods which semantically makes sense for a form.
    - lmm 12y ago
      REST-the-actual-paper is a monstrosity that, thankfully, no-one actually uses. "REST" as a name for the practice of using simple JSON and using existing HTTP features (methods, status codes) rather than using HTTP as a dumb transport layer is... an ok idea, but overhyped. The supported HTTP vocabulary is very limited, and adding new verbs like PATCH is a multi-year process that's never going to be well supported by every existing piece of software. So sooner or later you're going to have to layer a more complicated (business-specific) semantics over HTTP, and have e.g. a POST that actually has semantic meaning beyond being a POST. Given that, if you just used POST for everything, and had a simple encoding of method kinds and error responses over that... would that be so bad? Those who forget SOAP are doomed to recreate it, badly. I worry that things like JSON-schema are headed in that direction.

Animats 12y ago

We suck at error reporting to the user. One of the legacy curses of UNIX is trying to encode all error info in "errno". Then came HTTP, which most languages try to treat as something like a file access. When there's some complex problem at the remote end, the error presented to the user has often been hammered down to some errno value that's vaguely appropriate. This came up recently on HN in connection with GoGo WiFi on airliners, using satellite links. GoGo doesn't have the satellite bandwidth to let users view streaming video, so they block YouTube and some other sites. The problem is telling the user what's going on. They used an awful hack involving a self-signed fake SSL certificate for YouTube to try to get a coherent message to the user. According to the IP standard, they should send back an ICMP Destination Unreachable message with a code 13, "Communication administratively prohibited". Most IP/TCP/HTTP/browser stacks won't get that info all the way up to the end user, especially the "Communication administratively prohibited" part. Even if the error code made it to the user, many customers would not interpret "Communication administratively prohibited" as "No, you can't watch Youtube because you're on an airplane and we don't have the bandwidth for that".

jdmichal 12y ago
Why not just reroute those addresses via proxy to a static page explaining?
- eropple 12y ago
  They're under SSL. All a reroute would do is pop up Chrome's bad cert page.
- simoncion 12y ago
  Hell, it would have been far better to use traffic shaping to transfer at (like) 1bps than to do anything else.
simoncion 12y ago
> According to the IP standard, they should send back an ICMP Destination Unreachable message with a code 13, "Communication administratively prohibited". Agreed. However, I've discovered that even folks who really should know better don't know what to do with anything other that port-unreachable: In the past, I tried setting my ip[6]tables REJECT targets to send back admin-prohibit. For... reasons[0], I was REJECTing IPv6 traffic to Google's networks. I discovered that YouTube on a Nexus device running Android Jellybean would not fail over to IPv4 but would, instead, sit and spin for multiple minutes before reporting a fatal connection failure. [0] At the time I had a IPv6 tunnel that (for some reason) transferred packets destined for Google very, very slowly.

gaius 12y ago

As a protocol, it works perfectly well for transporting hypertext, what's the problem?

M2Ys4U 12y ago
Implementations.

andrewstuart2 12y ago

For the most part, I say "1000 times yes!" I love the general idea he expresses that developers MUST take the time to understand the giants on whose shoulders they stand. To that end, I have one noteworthy point (more of a clarification) I think is worth making: > And, technically, you’re supposed to make sure GET requests are idempotent, meaning they can be repeated with no changes to a resource. So you should be able to hit Refresh all day on a GET request without causing any data change. I think it's important to note here that idempotence does not mean that the resource will never change. It means that the resource will not change as a result of a get request (barring some meta requests, for example `GET /my-last-http-request`). A resource can definitely change. I think the author would agree with me but found his language there a bit ambiguous. Allow me to quote the good reverened himself, Dr. Roy Fielding: > "More precisely, a resource R is a temporally varying membership function MR(t), which for time t maps to a set of entities, or values, which are equivalent. The values in the set may be resource representations and/or resource identifiers." (emphasis mine) [1] Immediately prior this quote he references "the weather for Los Angeles today" as an example of a resource complete with its own resource identifier (perhaps GET /weather/la/today, most-to-least specific). The contents at the end of that URI will most certainly change, but they shouldn't ever change based on the request. [1] Section 5.2.1.1 https://www.ics.uci.edu/~fielding/pubs/dissertation/fielding_dissertation.pdf https://www.ics.uci.edu/~fielding/pubs/dissertation/fielding...

teilo 12y ago
Yes, thank you. Every framework I have ever used maintains this distinction between GET and POST. It's common knowledge.
spb 12y ago
To hit the most important point of what you mean by "change based on the request", this doesn't mean "changing your request changing the resource you get" (which wouldn't make any sense). It means that the act of GETting the weather right now will never change the weather. (If you want to change the weather via HTTP, you'll need to do it with a method like "POST /weather/hill-valley" and a body like '{"rain": "stop"}'.) Think of it like a getter function that gets syntactically hidden by the language: you wouldn't want "if (user.accesses < limit || user.accesses > threshold )" to have two different values for "user.accesses" based on the number of times you referred to it. In a less ludicrous example, it also means "GET /tokens/reset/dsf46JtCb385PDs2" shouldn't log you in and invalidate the token, something that most password-reset pages fail to comply with.

pixie_ 12y ago

Also see the discussion here - http://www.reddit.com/r/programming/comments/2rzlay/we_suck_at_http/ http://www.reddit.com/r/programming/comments/2rzlay/we_suck_...

craigds 12y ago

Sounds like the OP's problem is not so much the protocol itself, as the long tail of developers using it who have no idea what it can actually do. There are developers who do use most/all of HTTP. I'm one. I think I've used most of the obscure features he mentions, including URL parameters and all those status codes (excepting 402 Payment Required, which I've never had a need for, but I do know it exists). Most of the features he mentions aren't very useful for browser interaction, but really shine for APIs. If you're trying to write an HTTP API, and you haven't at least got to grips with all the status codes and methods, you should and take the time to understand them all and when they should be used. It's a real joy to use an API that uses methods and statuses properly. If in doubt, read the Github API docs - their API is (mostly) very good in this regard.

wrs 12y ago

> ...developers tend to think they can solve every problem, and they’re pretty sure that nothing good happened before they arrived on the scene. ... they feel no need to study the history of their technology to gain some context about it. So, so true. In fact, this happens depressingly often when "technology" is defined as narrowly as "the app a developer is working on" and "history" is as short as "a month ago". This has always happened, but it feels like it's gotten worse since "agile" became widespread. As if the principle of YAGNI ("You Aren't Going to Need It") has been turned into a poisonous assumption that nobody before you could have put any thought into the future. Rather than trying to understand the natural extension mechanisms that might already be in place, people just jump in and write more special-case code.

WoodenChair 12y ago

My problem with this article is not the general sentiment that developers should know the history of the technologies they work with, which I think is absolutely right, but instead how the author seems to put HTTP on a pedestal. It just happens to be the protocol TBL & company came up with in the early 90s for this thing that took off beyond anyone's wildest imagination. It's not some perfect work of art that should be revered. Also native apps don't make me sad. They're faster, more featureful, and have more consistent platform specific interfaces than their Web brethren. There are good reasons they exist and consumers prefer them.

Guthur 12y ago
I am guess that the apps referred to are the ones which have existed quite happily as a web page before moving to smart phones where for more likely business rather that user experience reasons they became apps. The ones I can think of are Twitter, Facebook, and LinkedIn. I am certainly not a proponent of web as a platform for everything but there are certainly some things it is rather good at, and making content discoverable in a consistent manner is one of those. If you hid the content in an app it's rarely exposed.
- TheOtherHobbes 12y ago
  This debate has been going since the days of AOL and CompuServe. As here, it usually misses the differences between public content, private content, and corporately farmed content. Twitter etc are corporately farmed content, and they all provide web and app access. The access method is just a front end. Personal content in apps isn't usually all that interesting. It's hard to think why anyone would care about my Angry Birds scores, or what level I'm on in Candy Crush. (If I played Candy Crush, which I don't.) If you want to make an app that shares content on the web, it's not difficult. I don't see a problem with the fact that it's extra effort, and if you're going to do it you need to justify that effort in some way.
yawboakye 12y ago
Agreed. Apps didn't break HTTP. They consume APIs (which can also be used by web applications). Thus, designing the API is what really matters.

james-skemp 12y ago

Wow. Dublin Core. Now that is a name I've not heard since around a decade ago. It is amazing to look back to my own development history and remember that I too used to care about proper status code, using metadata elements, and the like. But after adopting so many, and seeing so many either fail to catch on, or infighting tear them apart, or the big names pick them up and then put them aside ... it's not surprising that current developers don't think about them or don't care.

junto 12y ago

One of my personal pet peeves is the misuse of the 401 Not Authorised status code being issued to logged in users. No, try a 403 instead please.

drcomputer 12y ago

I love implementing stuff like standardizations. Seriously, I love reading documentation that has lists of codes meant to concisely summarize specific frequently occurring events. I am the biggest nerd when it comes to pouring through documentation that was designed specifically with the intent to unify a common need that started off fractured. In general though, it's just hard to know exactly when and where to implement what standardization. It's not just HTTP. The entire world of programming and computer science would be much more elegant with intelligent standardization, but it's literally a hard problem. The source of information is the hard problem. It is subject to change. How can you standardize something that is consistently evolving, evolving to the point that it is expected that the functionality of the system is going to exceed all known expectations of it's operation and behavior? As developers, we are consistently striving to exceed our own boundaries, to do things better, more optimally, more safely, more legibly, more precisely, more correctly. Our expectations of ourselves is what screws us when we think about 'the big picture', because we apply the same model of reasoning to 'the big picture'. Yes, we do not know what we do not know. However, we additionally do not know how to determine the difference between 'that which exists that we do not know' and 'that which does not exist that we do not know'. That is the core problem that lays underneath the lack of implementing standards, and it the reason people standardize things in the first place. Standardization prevents us from re-inventing the wheel. Standardization makes things safe. Standardization makes complex systems predictable even when information about adjacent systems are unknown. Standardization is a vehicle for implementing trust. People get confused because there are too many words for the same thing - too many ways to describe the same things, too many languages to do the same thing, too many explanations meant to teach the same thing. Either we have to learn the hard way (by observing massive failure, or by experiencing it), or we have to hope we are implementing things the most correct way, and trust we will get there.

fndrplayer13 12y ago

I feel like the ignorance of developers here is overstated. Most APIs that I know of make good use of the HTTP status codes. This includes APIs from big players like Google, all the way down to small startup APIs. Sure I've seen bad ones, but I don't know that I would agree that its the majority? Maybe I've just had good luck or lived a blissfully unaware existence as a developer. Hard to say.

nkozyra 12y ago

I'd say the reason why following spec on status codes has fallen out of favor is because browsers don't do anything interesting or informative with them for most users. On APIs, I'd say it's a sin to not return proper HTTP headers, but when the end user on a web site sees a File Not Found that's 200 Status OK versus File Not Found that's 404, it doesn't really matter. And while it matters for things like the Googlebot, over the years developers stopped caring, because delivering an attractive and descriptive status page was more important than delivering an error status and not rendering a page at all. I'm not saying it's right, but it was largely a ornamental step in the early days of the Web, so I understand why it disappeared. If browsers proactively provided data to users about HTTP status codes, I think they'd be adhered to much more.

seandhi 12y ago
Returning a 404 status code and displaying an attractive and descriptive status page are not mutually exclusive. If you go to http://www.google.com/unknown http://www.google.com/unknown, you will get a 404, but you will also see a custom page.
- nkozyra 12y ago
  Absolutely, I don't disagree. I'm saying that for most end users, there's no distinction. Status codes are for machines, but if early browsers did something noteworthy with those codes, developers wouldn't have been so quick to dismiss them in a way that is now largely habit.

olssy 12y ago

HTTP is too flexible not to theoretically break but in practice it can be used and abused and still deliver.

peterkelly 12y ago

Ironically, the URL of the post is http://gadgetopia.com/post/9236?retitle http://gadgetopia.com/post/9236?retitle

einrealist 12y ago
And?
- peterkelly 12y ago
  It doesn't exactly follow the recommendations in the second link from the post: http://gadgetopia.com/post/6346 http://gadgetopia.com/post/6346
  - einrealist 12y ago
    Yes and that is why the linked post and this post itself are bad. Both authors do not understand the principles of an URI in the context of the web. To a client, the URI should be opaque. If a client has to start to make meaning of the contents of an URI, the server loses the ability to change its implementation (or just move resources around). Edit: Ah, its the same author for both blog posts. That explains it.
  - woof 12y ago
    From the same page: (And yes, I know Gadgetopia’s URLs are somewhat lame in this regard. There are reasons why I did it, but changing it now - See more at: http://gadgetopia.com/post/6346#.dpuf http://gadgetopia.com/post/6346#.dpuf)

overgard 12y ago

Maybe the simpler explanation is that HTTP/WWW was always broken. It's never been a great application platform. It's popular and universal and simple (sortof), so that's awesome, but in terms of great tech it's kind of been junk all along. It's not like it ever was universally conformed to. It's not like there was this golden era. What would be the golden era? The web before AJAX? Geocities and Hotmail? Frames broke URLS more than almost anything. PHPNuke forums? Web apps sucked back in the early days. Makes me think of a fun Alan Kay quote: > The Internet was done so well that most people think of it as a natural resource like the Pacific Ocean, rather than something that was man-made. When was the last time a technology with a scale like that was so error-free? The Web, in comparison, is a joke. The Web was done by amateurs. -- Alan Kay.

woah 12y ago
Yea, the specification of status codes is cumbersome and annoying to deal with, like you're supposed to go find a list of what they are supposed to be on wikipedia and then match it up to what's going in your application as best you can. Also the PUT, POST etc stuff. It's just a pain to try to match up your application's behavior to a crusty old spec that was never that good. It's really refreshing to code with websockets and just write an application naturally. HTTP is good for GETing static web pages and that's about it.
- TylerE 12y ago
  Way to oversimplify. HTTP works ok for some super dumb use cases. But pray tell, what's the proper way to update multiple objects at once? Simple verbs work for simple tasks - only. There is more to life than REST.
  - jdmichal 12y ago
    Updating multiple resources of the same type is easy: POST to the plural form of the resource. If you need to update multiple resources of different types atomically, then you probably need a new resource abstraction that covers both of the resources anyway. Then just POST to that.

halayli 12y ago

It sounds like this guy just learnt about HTTP details.

bipin-nag 12y ago

I feel that the article continued in the wrong vein. The original NY times article from which the author takes inspiration talks about how mobile apps are segregated from the web. It says that Google, Facebook and other companies have apps whose content is not part of the web. If they wanted they could have done it. They decided not to because they don't want their content freely accessible. They are more than capable of doing that, Google even spearheading development of HTTP2. The technical challenge of deep linking remains an issue for many others which prevents developers to orgainze app content for web and people from accessing it directly. The article was more about getting the basics of HTTP right.

xsace 12y ago

What a pointless rant. Things evolve, even norms and specs. There's a reason why URNs are not used in the code, there are not enough by themselves, if you need use URIs. There's a reason people don't use every status code in every webapps. It's a waste of time, better spent elsewhere. Most webapps uses more then GET. Etc. Anyway, HTML is not broken, it's used everywhere by billions of users every seconds.

TheSwordsman 12y ago
Agreed. This guy needs to take a break from the Internet for a few years.
- einrealist 12y ago
  Indeed. It is the internet where everybody can write about something, even if he/she is (mostly) wrong. And posting it on HN or Reddit does not make it a better blog entry.
gtramont 12y ago
Perhaps if people understood better the technology that allows their business to operate, they wouldn't waste time versioning APIs and having to re-write clients (web, mobile...) every time the server changes something!
- xsace 12y ago
  You can't expect everyone to master everything. It's like ranting about how the whole classroom haven't got A+. It's pointless ultimately. The real question is how can we improve a protocol so it's better used by most people. Hence why it's evolving. Nothing should be set in stone forever and then expect everyone to comply to it or die.

stevenspins 12y ago

HTTP comprehension and usage are critical. People wonder why websites don't perform and api are difficult to program for. I have seen that developers have this weird need to create your own method and means of communicating errors. Even ws-* developers don't use soap errors. That is what we are talking about mostly in this article non-standard messages/state. Treating your application and data as resources (REST is the style of the web) and leveraging the power of HTTP so powerful. You are not a crabby old man. If you do count me in, 33 years old. I could make a career consulting and only fixing http and resource abstraction/usage problems. IMO though educating our fellow developers on HTTP is easy compared to the REST HATEOAS constraint.

bluetomcat 12y ago
> I have seen that developers have this weird need to create your own method and means of communicating errors. Developers ignore most of the status codes because they introduce unnecessary complexity on both ends (client and server) and do not always map well with the application logic. With JSON payloads, HTTP is used as a "light vehicle" to carry the application logic. Since the payload is going to be specially interpreted anyway, I don't see a benefit in cramming legacy HTTP cruft in the headers and in status codes. The really annoying thing is the web server spitting out HTML on error conditions instead of the expected JSON.

iamleppert 12y ago

You know what really bothers me? It's not the abuse of HTTP status codes. It's the closed nature of apps. How many of us got our start in web development by viewing the source of the page? By looking at how others did things? Hell, I still do that now. Apps are the Microsoft of the Internet -- closed, proprietary and secretive. Open source and github has helped this some, but meh.

mark_l_watson 12y ago

I liked the author's comment about using canonical URLs (with a link to another of his articles on this subject). This is something that I have been intending on doing for a while - I just pushed changes supporting canonical URLs to my main 3 web portals.