12 ms·

The 7-bit Internet

222 points by fogus 13y ago

herge 13y ago
Man, not having \n as a statement separator and having to pipe everything through xxd will definitely be a pain.
- danielweber 13y ago
  \n? or \r? or \r\n?
nine_k 13y ago
It's 2013, not 1973, so I'd replace 7-bit with UTF-8 which is its natural extension.
- lstamour 13y ago
  But UTF-8 is too complicated for many shells! ;)
- jimktrains2 13y ago
  In theory I agree. "In theory, theory and practice are the same. In practice they are not" :(
oconnore 13y ago
I don't understand why SPDY / HTTP 2.0 is being framed as "fitting the needs of a few major players". Google is not the only one who benefits from reduced latency, fast page loads, efficient use of SSL sessions, and server pushed resources. If anything, the people who benefit the most are those who _can't_ afford massive forward deployed CDN networks, large servers, and fat network pipes.
- zzzcpan 13y ago
  Google can afford to dedicate a bunch of engineers just to deal with all these complexities and inconveniences of binary protocols, but others can't. And this is far more important than some hypothetical tiny improvements in performance.
  - cmccabe 13y ago
    Yeah. It's too bad there aren't free web servers out there that everyone can use that implement these protocols. Oh wait, there are. By making things more efficient, HTTP 2.0 is really going to make startups spend more money. Oh wait, no, the opposite thing.
    - zzzcpan 13y ago
      These web servers aren't that useful for everyone, some of us have to write our own. EDIT: It's not the matter of implementation anyways, it's that something important only for Google is proposed as a standard for everyone, which is insane.
      - cmccabe 13y ago
        What startups do you know that are writing their own web servers? More efficient web servers will, if anything, be even better for startups than they are for Google, since they lower the barrier to entry.
- josteink 13y ago
  The mind-bending complexity of the proposed HTTP 2.0 does very little to gain the little guy, or a young generation exploring the web: in fact it shuts them out. If the internet should have taught us one thing so far, it is that it's the open technologies which are built to be easy to understand and explore for humans which have driven the net forward. HTTP 2.0 is massive step backwards in that respect, and the only thing we're getting in return is slightly shorted response times. You know what? We can deal. We're getting better and better bandwith. We're getting more and more computing power. A few milliseconds doesn't hurt anyone. Let's not create a shit protocol violating everything the internet was built on, just because we're all of a sudden feeling resource-constrained. Now, of all times. rolls eyes
  - bittired 13y ago
    HTTP 2.0 is going to happen whether we complain about it here or not. While we can certainly wait and see whether people will balk and make changes, the only true way to stop it is to replace it. If you can write a better set of protocols that meet the requirements, do it. If you can't then work with or support those that will.
    - gaadd33 13y ago
      Like IPv6? Its taken more than 10 years to approach 1% traffic share. Perhaps HTTP 2.0 will happen but I wouldn't bet money on it being everywhere anytime soon.
      - lstamour 13y ago
        I can't switch to IPv6 by downloading a browser. I could switch today to SPDY. In fact, it's got 55% market share including IE 11: http://caniuse.com/spdy http://caniuse.com/spdy
      - vidarh 13y ago
        Consider that until people start being told "no, you can't get more IP v4 addresses" or some services are unavailable via IPv4, the incentive to switch to IPv6 is quite low. We have IPv6 connectivity from our colo provider in one location, and a tunnel to our office, but two years in we're still only testing it for this reason - there's never enough time, and it's not yet urgent enough.
- weland 13y ago
  > If anything, the people who benefit the most are those who _can't_ afford massive forward deployed CDN networks, large servers, and fat network pipes. You win some, you lose some. The benefit of cutting down on some infrastructure should be weighed against the mind-boggling complexity of HTTP 2.0 and the added difficulty in debugging.
- alayne 13y ago
  For Google, a .0001% reduction in user overhead may translate to millions of dollars of resource savings. Google is willing to do crazy things like run their own modified Linux kernel that most companies wouldn't bother with. What is the end-user benefit in performance using HTTP 2.0? The only analysis of SPDY I saw indicated low single digit percentage benefit vs HTTPS. Whoop-dee-do. http://www.guypo.com/technical/not-as-spdy-as-you-thought/ http://www.guypo.com/technical/not-as-spdy-as-you-thought/
  - lstamour 13y ago
    It's funny, isn't it? We gave out more domains to speed things up, but now it slows us down (referring to your link). I would highlight that HTTP 2 > SPDY as far as standards go, and the "beta" protocol of SPDY is already at 55% market share including IE 11: http://caniuse.com/spdy http://caniuse.com/spdy For those who are interested in speed optimization for 2013 and beyond, plus why SPDY's faster-than-SSL descendants are crucial to the web's success, have a look at http://www.igvita.com/slides/2013/breaking-1s-mobile-barrier.pdf http://www.igvita.com/slides/2013/breaking-1s-mobile-barrier... -- I believe there's a video out there too somewhere. Edit: Video link is on first page of the slides. What's interesting too is that this talk focuses as much on what's achievable without SPDY as what might be with, down the road. We really need faster SSL negotiation for that first time connection cost.
- hcarvalhoalves 13y ago
  I see the proposals of binary protocols, push content, etc. as symptoms of a deeper issue. Different than the spirit of HTTP, those protocols have little to do with publishing content anymore. They are just more kludges, in the history of kludges, to patch browsers into application platforms. That mainly benefits the big web monopolists, who require the browser to be the ultimate application platform, where they can track to their hearts content, display unsolicited advertising, and basically extort business to advertise on their channels to remain relevant on the web. Not quite the idea of "information repository" that spawned the web in the first place.
  - lstamour 13y ago
    Right. Because the alternative, bloated pages out the wazoo makes so much more sense than having smarter Apache and Nginx servers who will do all this for you. Remember, most of the web will be connected by cell phones, we need better protocols for that.
    - hcarvalhoalves 13y ago
      Did you read the comment or just skimmed? I don't question the technical merits, but the underlying motivations, which I believe are fatally flawed.
      - lstamour 13y ago
        Read. That's why I mentioned Apache and Nginx, since the Google SPDY team has focused on making sure that there are plugins for each, plus CloudFront support, so that it can be applied "to the little guys". And in turn, I would ask you: Have you not used Web Sockets? How about HTML5 Web Components? New JS functionality? As "hackers", I would expect more excitement than this -- since you're going to enable a whole new generation of low latency cell phone apps, OSes based on web browsers and apps for them (e.g. Win8.1, ChromeOS and Webkit) and do so in a very standards-based way, rather than ad-hoc distributing SPDY through your proprietary browser. Short of security flaws like SSL's header compression, what could go wrong? :)
      - 13y ago
joosters 13y ago
I agree with most of the article, but would strongly argue against any use of the 'robustness principle'. There have been huge security vulnerabilities because systems disagree with each other about what is a valid request and response.
- skrebbel 13y ago
  And billions of money wasted because browser vendors, at some point long ago, decided to accept invalid HTML.
  - enqk 13y ago
    and lots of experiments, new websites and happy users appeared because HTML has been accepted leniently
    - pbsdp 13y ago
      And we've wasted vast mountains of money dealing with the resulting brokenness of the web. Those people would have figured out how to ship a web page; they'd just would have had so much trouble figuring out why one they shipped was't working, because the browser would have told them.
    - skrebbel 13y ago
      would those experimental users have given up right when the browser reported "Missing </p> tag"?
      - TeMPOraL 13y ago
        They would probably be more encouraged and happier to develop even more because they would understand HTML, which is a simple but powerful concept of block markup, and not some random pile of characters in angle brackets.
      - enqk 13y ago
        That end tags were an option and not mandatory was a feature of the SGML language that HTML was based on. I'm not sure specifically if this was allowed in the description for the P tag, however I suspect it is. So this was entirely by design and probably not all that difficult to implement. That it might not be appealing to some I can understand. I suspect the creators of SGML had very different concerns. I like that HTML has a very humane interface.
osth 13y ago
Amen, OP. herge: You could also use the "data" program from Hobbit's netcat.
zackbrown 13y ago
This sounds similar to an argument you could make for writing all of your code in assembly instead of C (or other higher-level languages.) Yes, there's a layer of obscurity between a program written in C and the resulting assembly: a tool (yes, a tool, spouting abstraction all over the place!) called a compiler. As a core tenet of engineering, we create tools that often leverage abstraction in order to let us tackle more complex problems without wrapping our heads around (or writing code to handle) the details. It's a matter of balance, and we can all choose our own camps along the abstraction <=> ease-of-understanding-every-detail continuum (I'd tend to plop myself toward the former.) Just as we can create a reliable C => assembly compiler, we can create reliable tools to let us leverage abstraction for the better and focus on the bigger picture rather than the smaller.
- scott_s 13y ago
  I find your analogy curious, because I think it's backwards. If we're going to use C and assembly, the author is arguing for C: easier for humans to read, but we'll have to translate (compile) it so that the machine can read it. Assembly is easier for machines to read (well, not really), but harder for humans to read. My "not really" bit is because the better analogy would be C to machine code. The author's point is that HTTP/1.1 is a text-based protocol. It's inefficient that we have to translate that text into meaningful bits on either end, but it makes it easier for humans to inspect. HTTP/2.0 allows for binary communication, removing the need to translate text, but making it harder for humans to read. It's removing layers of abstraction, not adding them. The introduction is quite clear on this point: The Hypertext Transfer Protocol (HTTP) is a wildly successful protocol. However, the HTTP/1.1 message format is optimized for implementation simplicity and accessibility, not application performance. As such it has several characteristics that have a negative overall effect on application performance. I'm not necessarily saying I agree with the author; I don't know enough about the tradeoffs in this domain to say if the loss of abstraction is worth it. But I do think it's worth understanding his arguments on its merits.
  - zackbrown 13y ago
    I think we're addressing different points in the article, especially in that 'readability' and 'complexity' are distinct. Also, I think the author does make valid points, which is why I qualified that we can all choose our preferred places on that continuum of abstraction. My point is that it is sometimes (or from my perspective, often) worth it to deal with more complex (as in higher-level) technologies via tools that allow us to de-abstract those technologies than to focus on using only technologies for which we can understand every detail of their implementation. Trying to understand assembly written by a human vs assembly spat out by a compiler (with its optimizations, etc.) is a very different beast, but thanks to the tools of higher-level programming languages and compilers we rarely need to fiddle around with machine-generated assembly. Similarly, with reliable tools, we would not need to fiddle around with raw HTTP 2.0 workings, and thus rarely would the concern of immediate human-readability be an issue.
Afforess 13y ago
I don't get the fuss. HTTP 2.0 is not a deprecation of HTTP 1.1/1.
jere 13y ago
>At first, this struck me as a very odd thing; text is inefficient, and machines, not humans, are meant to interpret protocols. A binary setup would save bytes—bytes!— and be all-around more manageable by software. >It wasn’t long, however, before I realized the true genius behind this decision. This reminded me very much of a talk by Jonathan Blow where he's talking about how he "nerd raged" while reading the Doom source code. He found a section that wasn't optimized, but in fact not optimizing it had so many other benefits (besides, the performance gain would have been negligible). Starts at 14 minutes: http://the-witness.net/news/2011/06/how-to-program-independent-games/ http://the-witness.net/news/2011/06/how-to-program-independe...
mbq 13y ago
Oh sure, let's keep buffer bugs, waste tons of bandwidth and storage, burn millions of CPU hours in parsing computer-generated text, completely ban encryption and screw it for all non-en_US-speakers so that it could be debugged via misuse of a protocol from 1983 that no one sane uses.
- jbert 13y ago
  I watched simple text-based internet protocols eat the efficient, ASN.1 OSI protocols for breakfast. From the OSI side of the fence. You can talk about efficiency until you're blue in the face, but human-readability is very useful. I used to be able to pretty much parse a hexdump of an X.400 P1 PDU by eye (certainly if I could reformat in a text editor), but even today I found it useful to eyeball a recalcitrant programs's HTTP request/response cycle by simply catching it under strace and grepping for "HTTP". It's a massive reduction of friction to have human-readable protocols. I used to make the argument that it didn't matter and I was wrong.
  - zxcdw 13y ago
    > * but human-readability is very useful.* For who, how often and when exactly? As far as I have understood, we are doing something wrong if we have to deal with debugging protocols which implement an abstraction, rather than just let the abstraction do it's job. This is like saying that "Well for programmers x86 assembly instruction mnemonics are useful compared to machine code bytes!", to which one could say that for an average programmer that makes no sense. I didn't quite care about stuff like this until I got introduced to information theory and really started thinking about what it means to send and receive information. The amount of totally unnecessary waste is astounding.
    - dsr_ 13y ago
      I'm starting to feel like I start all my HN posts with "Hi, I'm a sysadmin." Hi, I'm a sysadmin. Sometimes we have to debug things that are on the other side of the world. When we do that, we need to have a mental model of what we're doing. You're familiar with that as a software developer: you have a mental model of the capabilities of the language that you are using, and you are fitting that in with the mental model of the problem you are solving and the context of existing program code. To sysadmins, protocols are like programming languages. That's why we like text-based protocols: we can fit them in our heads and type them out, slowly, like ancient creaky teletypes that make lots of mistakes and pause in between commands. Meanwhile, we're running down checklists: can we connect? OK, there's no IP-based packet filtering. What does the banner say? Can it support STARTTLS? OK, disconnect and try again with telnet-tls. Hey, it doesn't negotiate... Totally unnecessary waste? Sure, assuming someone has already written a tool which does all the testing for you, and you know that tool exists, and you have access to it right now over your tiny smartphone's 3G connection at 0400 while you thought you were on vacation.
antitrust 13y ago
The original idea behind the web was a publishing platform that, like a classic car, would be easy to explore, maintain and develop. The new vision is the application. Whether it's on a phone, or running as a web site, these are not publishing so much as services. Perhaps it's time to fork protocols: one for publishing, one for interactive applications.
- weland 13y ago
  This has been on my mind, too, for a long time. Many applications don't fit the semantic framework of the WWW anyway; why not call it a day and fork it? This way the web app crowd will be able to do their somethingJS magic without worrying about what the WWW was meant to be, and the people who use WWW for publishing can get a sane platform back on track.
  - hollerith 13y ago
    That would be a big win if you can pull it off. I wonder what is the best way to do it. convincing all owners of web apps to migrate to a new infrastructure strikes me as extremely hard. convincing the millions of 'publishers' of web writings (including all bloggers) to migrate sounds really hard, too.
    - pbsdp 13y ago
      What do you think 'native' platforms are? They're the platforms built to support app development.
  - mbrock 13y ago
    This is all very interesting... I feel that the document-centric request-oriented nature of the web has been a powerful influence on the design of "web applications." But we still haven't really figured out how to do it. The web has been full of applications since the beginning, no? There was no golden age of pure document sharing. So much of the web throughout its history has been CGI, ASP, JSP, ad hoc setups to provide dynamic sites. Session cookies, inscrutable URLs, forms with two dozen hidden fields encoding cryptic parameters... (Here's a www-talk thread from 1993 discussing the new script support in NCSA httpd http://1997.webhistory.org/www.lists/www-talk.1993q4/0485.html http://1997.webhistory.org/www.lists/www-talk.1993q4/0485.ht...) But since the formulation of the REST architecture, it seems that more and more people are interested in what the WWW was meant to be. Maybe we're only beginning to understand the possible implications of the WWW design, and to purify that into a powerful conceptual framework. Does that essentially involve a text-based HTTP protocol? I don't think that's the most important part. The REST philosophy seems to say that the sane platform of the web is more crucially about clearly defined paths to typed resources providing a uniform set of actions and discoverable associations. This is not just for publishing -- what makes it so interesting and fruitful is the way it can be used to structure many kinds of applications in the form of published resources... "Calling it a day and forking it" seems in some way to mean giving up on this fruitful encounter between two paradigms. Then you get a binary socket protocol with no resource structure but with a lot of potential for shiny stuff, versus a document-centric protocol that's slower and more restricted... And then all the big players do the shiny thing, and the document protocol is left for enthusiastic hobbyists and legacy applications...
breadbox 13y ago
If you're tempted to argue against this by pointing out that very simple tools can make irrelevant the change to a non-textual protocol -- then you are, I suspect, missing the point. There's a real psychological difference between debugging something that is, at bottom, human-readable versus one that isn't. I'm willing to concede that it isn't really that big of a difference. For the sake of argument I'm even willing to concede that it's largely illusory -- because that doesn't matter. Who cares how irrational it may be? If it's that prevalent in the human psyche, you're just better off not fighting it. Like scripting languages, a text-based protocol doesn't just make it easier for you to get your hands dirty: it practically begs you to. The value of that sort of encouragement to the adoption of a global standard should not be underestimated.
- Afforess 13y ago
  HTTP 2.0 is not a deprecation of HTTP 1.1/1. You can have your cake and eat it too.
  - breadbox 13y ago
    Never thought that it was. This is just another reason why HTTP 2.0 adoption is going to be an uphill battle.
wmf 13y ago
This text vs. binary discussion is pretty high level. HTTP/2.0 has a mux layer (let's temporarily put aside whether that's a good idea) and some of the payloads have to be binary (e.g. JPEGs). There are basically three ways you can do this: A. base64 everything. Obviously this has high overhead. B. Escaping (aka byte stuffing). This is somewhat slow to escape and unescape, the overhead is variable (in rare cases 100%), and it's fragile to read or write by hand. C. Byte counting. This is the most efficient, but extremely inconvenient to write by hand. You could create an efficient "text" mux protocol (basically BEEP), but the result is so non-human-readable/writable IMO that I don't think people would be any happier. HTTP/2.0 is not complex because it's binary; it's binary because it's necessarily already complex enough that text doesn't save you anything.
- snuxoll 13y ago
  You have many of the same issues with HTTP/1.1 pipelining.
mjn 13y ago
My personal "wow, it's just this?" moment came when, in high school, I figured out how to IRC from a Unix box that didn't have an IRC client installed, just by telnetting to the IRC server. The only particularly annoying part was that you'd get disconnected if you were idling and didn't respond to a PING with a PONG quickly enough. It was also a fun way to learn programming. mIRC at some point added a sockets interface to mIRC script, and after that it was pretty easy to write toy clients for various text-based protocols, doing simple things like checking whether a URL was a 404. I can't say I did anything particularly useful with that, but it was a nice way to learn something about programming.
- thibauts 13y ago
  I remember just about the same, some 15 years back. Discovering networks and being able to mess with them, even though my programming skills were still very basic. This is a good part of what made me enjoy programming. Exploring this world, feeling it within grasp. I think TAOUP sums up a few important points about text protocols [1] and why going binary is not always wise. Obviously the main drive here is money and I'm sure we all have stories where short term money translates later to large unforseen and/or hidden costs. Bandwidth capacity is growing exponentially so do we really need this, even for the savings ? I'd rather see people invent clever new ways to do wonders with text streams than go back to opaque and obscure data structures that bring back to me rememberances of old days proprietary protocols. After all, if our fathers made the choice of text streams at a time when each byte cost much more than now, there may be good reasons. [1] http://www.catb.org/esr/writings/taoup/html/ch05s01.html http://www.catb.org/esr/writings/taoup/html/ch05s01.html
jperras 13y ago
For some reason, I feel as though the people in this thread against text-based protocols have never used telnet to inspect or debug an HTTP request/response. Or used telnet to help develop a client for something like memcached. The ease of discoverability of a protocol should not be underrated.
- josteink 13y ago
  Yeah. For some things seeing truly is believing. I guess if you've never had that moment where it all clicked for you, just by looking at traffic, the significance of human-readable protocols is easy to underestimate. But hey, Google says that with binary-compressed headers of sorts, they can fit all their tracking cookies in one TCP-packet and that means you wont feel inconvenienced by the tracking payload they are throwing at you everywhere you go on the internet. You know what? I want that huge bag of tracking cookies to hurt. If you violate my privacy, that should come at a cost. Google: You can take your SPDY and keep nice, old HTTP alone.
- rjsw 13y ago
  Writing new protocol modules for tcpdump isn't difficult.
  - mzs 13y ago
    After that you can see what? You also have to use something like sed, xxd, and nc at least to make a request unless you are really good with bitfields and a hex editor. You want to keep a connection open or use what comes back for he next request? Maybe you can quickly cobble together a crude client in expect and tcl, perl, or python.
  - iso-8859-1 13y ago
    A lot more difficult than using netcat though.
- flipperypokery 13y ago
  I feel as though the people in this thread against (well-designed) binary protocols have never tried implementing a binary protocol. Length-prefixed fields and direct encoding of binary data go a long way to simplifying implementing a protocol. The ease of implementation of a simple binary protocol should not be underrated.
kyrra 13y ago
While I agree that it sucks to lose text based protocols, we already have an underlying protocol that is completely binary in nature and requires tools to look at it (TCP). Same can be said for IP. We've survived a long time with those technologies and we have some awesome tools for working with them. While it's not as easy as ascii based protocols, it's more space efficient, which I'm fine with.
- gdwatson 13y ago
  But one of the beautiful features of TCP is that it has the properties necessary to treat it as a text stream -- you can build text-based protocols on top of it. Since troubleshooting an application-level protocol seldom requires you to delve into the TCP-level, it works very well in practice. The HTTP 2.0 draft not only makes the application-level protocol binary, but it eschews the simpler stream metaphor in favor of replicating a bunch of that TCP-level complexity in the higher layer. There are reasons for all of this, of course -- it's not being proposed on a lark. But it does have a very real complexity cost.
  - lstamour 13y ago
    So don't support HTTP 2.0 if it doesn't benefit you and your use case. It's that simple. In a world where HTML5 says browsers need to pretend to be Netscape Navigator when asked, you'll never lose HTTP 1.x no matter what happens. Never.
lstamour 13y ago
Am I the only one who thinks its ironic that the title references 7-bits? This shows that you're already binary encoded enough that your terminal has to parse it out. Why not just have a dumber terminal? Plus, it's SSL based. SSL means you're already used to not looking at things through telnet. Are you really saying we shouldn't use SSL because "it's not just 7-bit plaintext"? That's the point, after all...
- jimktrains2 13y ago
  > SSL means you're already used to not looking at things through telnet. Someone has never used `openssl s_client` before. Very few people actually delve into and edit the TLS protocol. People, including devs, often delve into the HTTP protocol and craft requests by hand, or with a simple text-processing script. To most people, TLS is transparent. This HTTP 2.0 protocol is not transparent.
  - lstamour 13y ago
    If you're using openssl s_client, why not use another such client? That's my point. The fact is, you type ascii because it's inconvenient to type binary, but you have software interpret bytes, so why not send what output you want, monitor what you want, using new tools. Haven't you ever used Fiddler, Charles Web Proxy, WireShark or anything else that makes binary protocols easier? Oh wait, yes, you have: openssl. My point exactly. You'll be able to unwrap the binary to ASCII representations as much as you like, no question, no harm, no foul. Or it won't get adopted.
    - jimktrains2 13y ago
      Because I'm not working at the TLS layer, I'm working at the HTTP layer. I'm sure if I had to often write TLS handshakes by hand, s_client wouldn't be my tool of choice, just as curl isn't always my tool of choice. Making the protocol I'm working in binary doesn't do me any favors.
      - lstamour 13y ago
weinzierl 13y ago
"I’ve been in this business for two decades, and the only way I know to hand-debug a SOAP transaction is with a hammer and a straitjacket2. Heck, I’ll take an old-school EDI transaction over SOAP any day." SOAP? You lucky. It's 2013 and I'm still knee deep in RMI-IIOP sometimes. Wireshark with GIOP dissector helps, but I’ll take SOAP over CORBA any day:-). Seriously: Your post summarized very well what I was thinking when reading the recent HTTP 2.0 posts. Full Ack.
D9u 13y ago
The site crashes my phone browser. Dolphin Mini. HTC Wildfire. Android 2.2.2 I guess I'll check it out later.
joshaidan 13y ago
I work in telecom, and this makes me think of two protocols I work a lot with, SS7 and SIP. SIP is like HTTP, in fact I believe it was modelled after HTTP and has a lot in common with it. You can troubleshoot SIP issues very easily using a packet inspector like ngrep (using Telnet might be a bit difficult as there are some timers that expire if you don't respond fast enough, but that's besides the point). Then there's a protocol like SS7 which is binary based. It's all structure binary bit fields. Even though it's binary based, and not very human readable, we have tools that decode the bits into a human readable format, which in turn makes it just as easy as SIP to troubleshoot. The question I ask myself is, which one would be easier to implement, text based or binary format? I guess I'd lean towards text based--in fact I have implemented a SIP client myself with success. But have yet to implement any kind of SS7 or binary based protocols. Perhaps experience has something to do with it, would having experience implementing binary protocols make it any bit easier? And to be fair, SIP is also much well more documented then SS7. As well it's easier to test SIP. I'm just starting to work with binary protocols, so it will be interesting how much progress I make. I realize there are also some libraries that help with working with binary/bit field data.
- toast0 13y ago
  binary protocols can be easier to develop, because generated and parsing the protocol stream is usually straight forward: if there's a variable length field, you will typically get the length first and then the field; no need to scan byte by byte for a terminator(s) or handle escaping of terminator(s). There's often less need to be liberal in what you accept too (but that may depend on how the protocol is versioned) But if you're developing for a protocol with garbage documentation (or no documentation), a text based protocol at least offers some amount of intrinsic documentation.
comex 13y ago
I don't think a simple binary protocol for HTTP would be bad at all. HTTP servers are still going to support text-based connections forever, so it only matters if you're snooping on existing traffic; but the ability to use Wireshark or tcpdump to grab the text out of packets directly is already disappearing because of encryption (SSL), which is such a good thing in general that it far outweighs that drawback. Therefore more specialized tools are required, and those may as well parse such a protocol - it wouldn't be that hard. The compression is a significant benefit, and can't be done well below the level of HTTP. What I don't understand is why we need both SPDY and QUIC. SPDY provides fast multiplexing, compression, and guaranteed SSL. QUIC provides fast multiplexing and guaranteed SSL at the transport(-ish) layer. QUIC does at least one thing that can't be done at the application layer - faster opens - and seems nice to have as a generic base for multiple protocols, so I call it a good thing (and I'd like to see it in the kernel). But the plan seems to be to run SPDY over QUIC - I can't actually find enough information on the Internet (and am too lazy to look through the source) to find out whether packets are going to be double encrypted for the time being, but even if/when that is avoided, the multiplexing and optional encryption seem to be wasteful complexity. I would prefer if HTTP/2.0 were a simple and easy-to-parse protocol providing compression only, expected to be used over QUIC.
- wmf 13y ago
  As I see it, SPDY/HTTP 2.0 is for the next ten years. QUIC is for the ten years after that.
makmanalp 13y ago
I don't get all the for-telnet arguments. Who uses telnet for this anyway? I'd bet that 90% of telnet-folk do: cake:~ mali$ telnet google.com 80 Trying 74.125.235.8... Connected to google.com. Escape character is '^]'. GET / HTTP/1.1 ....... HTTP/1.1 200 OK Date: Wed, 10 Jul 2013 22:25:02 GMT Expires: -1 Cache-Control: private, max-age=0 Content-Type: text/html; charset=ISO-8859-1 At which point they go "oh cool" and go do something else. For the rest of us who use curl, wget, requests, Chrome / Firefox developer tools every day (who are we kidding, it's everyone, you liars! :P), the binary transformation would be transparent. Hell, if you're going for pure cool-factor, how is pulling out your hex editor less cool? But in reality, you'd never do this. For a non-standardized and obscure protocol where tooling would likely be lacking, I can see why human readability is a good idea. But we're talking about the very protocol that makes up the fabric of the internet. Seriously, why? Give me one good reason.
- jimktrains2 13y ago
  Have you never had to send a specific request/response to a server? I've often found it easier to store the request in a text file and pipe it instead of using curl or another tool to craft it. When I was a webdev (over the past few years) I did this at least once a month. I'd daily look at the HTTP headers, though. Also, I'd often add custom headers for debugging purposes. Edit: It's also not just telnet, it's being able to use simple scripts to automate requests or responses for many reasons.
  - comex 13y ago
    Most likely all HTTP/2.0 servers will continue to support HTTP/1.1 forever, so you will still be able to do this.
    - jimktrains2 13y ago
      And clients will need to support it. I feel like the HTTP/1.0 to 1.1 switch allowed for a good deal of code reüse, but this won't.
zokier 13y ago
I think TLS/SSL is the elephant in the room here. A large part of the web is effectively already using binary protocol (HTTPS), and frankly I'd wish the part was even bigger. Think of HTTP 2.0 as a successor for HTTPS instead of HTTP if it makes you feel better.
- sergiosgc 13y ago
  Wrong. An encrypted text session is different from a purely for binary session (encrypted or not). The former can be mangled from text by a generic encryption proxy, the latter needs a protocol-specific analyser.
  - 9h1d9j809s 13y ago
    I don't get it. Why does it matter if your tool just decrypts SSL or you use a tool that decrypts SSL and additionally unpacks the binary representation of HTTP 2.0? The output is exactly the same - plain old HTTP.
    - denzquix 13y ago
      The point is that HTTP may not be the be-all end-all protocol that we will ever want to use on the open web, and encouraging any potential new protocols to take HTTP's lead and use an arbitrary binary encoding that requires protocol-specific tools to make sense of, is maybe not a decision we should take lightly.
deleted 13y ago
[deleted]
gohrt 13y ago
> You should follow me on Twitter. This meme needs to die. It's like ending every post with "Just kidding, I don't know what I'm talking about, I'm just cargo-culting blog attention-desperate person trying to build my Klout score to impress myself."
breck 13y ago
You may find this thing that we're working on relevant: https://github.com/nudgepad/space https://github.com/nudgepad/space It's a very understandable and very powerful language that I think will at some point be extended to replace HTTP, amongst other things.
jfarmer 13y ago
Binary vs. text is one dimension and it shouldn't be conflated with the overall alleged complexity of the HTTP/2.0 proposal. You can have robust and non-robust binary protocols. I don't buy the "argument from visibility", for example. HTTPS and DNS are both binary protocols and you need tools to parse them in order to get telnet-like visibility. Does that mean when it comes to those protocols "we are forced to rely on other tools to hide the underlying complexity and dumb things down to a point where they are manageable", as the author says?
cmccabe 13y ago
Marco claims that HTTP 2.0 is overly complex, but never gives any reasons why. He claims that text protocols are better than binary, but never really gives any reason for that either. Nostalgia for the days of telnet and gopher is not a reason. Then he goes off to talk about SOAP, which he hates, because "the only way [he] know[s] to hand-debug a SOAP transaction is with a hammer and a straitjacket." But SOAP is text based, which directly contradicts his argument from earlier. The other allegation here is that the new protocol "satisf[ies] what are essentially the needs of a small group of very influential players." Again, Marco just states this with no rationale at all. In fact, the opposite is true. By being more efficient, HTTP 2.0 will serve the needs of everyone. Should startups have to pay for 10 servers when they could have paid for 5 with HTTP 2.0? Apparently Marco thinks they should. It's easier for Google to pay for a few more servers than it is for Joe startup. This should be obvious to everyone, but apparently it's not. The whole binary versus text thing is a complete red herring. If you have ever used wireshark, you know that it makes binary fields easy to read. Binary protocols are easier to develop and standardize. Pretty much every good developer realizes this, and the responses here reflect that. The idea that making something LOOK like (but not actually be) "human text" makes anything better is an idea we should have buried along with COBOL and other mistakes from the past. But I guess not.
- jpatokal 13y ago
  SOAP is to REST as XML is to JSON. They're all technically 7-bit and notionally human-readable, but SOAP/XML are (well, can be) so verbose as to be near-impossible to parse by hand. Comparing the "Hello World"-style simple examples on Wikipedia is instructive: REST http://example.com/resources/item17 http://example.com/resources/item17 SOAP <?xml version="1.0"?> <soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"> http://www.w3.org/2003/05/soap-envelope"> <soap:Header> </soap:Header> <soap:Body> <m:GetStockPrice xmlns:m="http://www.example.org/stock"> http://www.example.org/stock"> <m:StockName>IBM</m:StockName> </m:GetStockPrice> </soap:Body> </soap:Envelope>
  - snogglethorpe 13y ago
    Amen. "Human readable" is not the same thing as human readable.
CrLf 13y ago
I love these discussions about text vs. binary protocols. I especially love the part where people defend binary protocols because they are more "efficient" and don't eat as much bandwith or processing resources. May I remind these nice people that most text-based protocols we rely on today were designed in a time when CPUs were orders of magnitude slower and modems ran at 2400 baud? Talk about processing and bandwidth resources... The resources consumed by having to interpret a text-based protocol pale in comparison with the resources consumed by all the levels of abstraction that modern frameworks/language runtimes that we use today. But wait, these save developer time, so they are a good thing. So does having protocols that can be troubleshooted by a human with minimal tools.

The 7-bit Internet

222 points by fogus 13y ago

herge 13y ago

Man, not having \n as a statement separator and having to pipe everything through xxd will definitely be a pain.

danielweber 13y ago
\n? or \r? or \r\n?

nine_k 13y ago

It's 2013, not 1973, so I'd replace 7-bit with UTF-8 which is its natural extension.

lstamour 13y ago
But UTF-8 is too complicated for many shells! ;)
jimktrains2 13y ago
In theory I agree. "In theory, theory and practice are the same. In practice they are not" :(

oconnore 13y ago

I don't understand why SPDY / HTTP 2.0 is being framed as "fitting the needs of a few major players". Google is not the only one who benefits from reduced latency, fast page loads, efficient use of SSL sessions, and server pushed resources. If anything, the people who benefit the most are those who _can't_ afford massive forward deployed CDN networks, large servers, and fat network pipes.

zzzcpan 13y ago
Google can afford to dedicate a bunch of engineers just to deal with all these complexities and inconveniences of binary protocols, but others can't. And this is far more important than some hypothetical tiny improvements in performance.
- cmccabe 13y ago
  Yeah. It's too bad there aren't free web servers out there that everyone can use that implement these protocols. Oh wait, there are. By making things more efficient, HTTP 2.0 is really going to make startups spend more money. Oh wait, no, the opposite thing.
  - zzzcpan 13y ago
    These web servers aren't that useful for everyone, some of us have to write our own. EDIT: It's not the matter of implementation anyways, it's that something important only for Google is proposed as a standard for everyone, which is insane.
    - cmccabe 13y ago
      What startups do you know that are writing their own web servers? More efficient web servers will, if anything, be even better for startups than they are for Google, since they lower the barrier to entry.
josteink 13y ago
The mind-bending complexity of the proposed HTTP 2.0 does very little to gain the little guy, or a young generation exploring the web: in fact it shuts them out. If the internet should have taught us one thing so far, it is that it's the open technologies which are built to be easy to understand and explore for humans which have driven the net forward. HTTP 2.0 is massive step backwards in that respect, and the only thing we're getting in return is slightly shorted response times. You know what? We can deal. We're getting better and better bandwith. We're getting more and more computing power. A few milliseconds doesn't hurt anyone. Let's not create a shit protocol violating everything the internet was built on, just because we're all of a sudden feeling resource-constrained. Now, of all times. rolls eyes
- bittired 13y ago
  HTTP 2.0 is going to happen whether we complain about it here or not. While we can certainly wait and see whether people will balk and make changes, the only true way to stop it is to replace it. If you can write a better set of protocols that meet the requirements, do it. If you can't then work with or support those that will.
  - gaadd33 13y ago
    Like IPv6? Its taken more than 10 years to approach 1% traffic share. Perhaps HTTP 2.0 will happen but I wouldn't bet money on it being everywhere anytime soon.
    - lstamour 13y ago
      I can't switch to IPv6 by downloading a browser. I could switch today to SPDY. In fact, it's got 55% market share including IE 11: http://caniuse.com/spdy http://caniuse.com/spdy
    - vidarh 13y ago
      Consider that until people start being told "no, you can't get more IP v4 addresses" or some services are unavailable via IPv4, the incentive to switch to IPv6 is quite low. We have IPv6 connectivity from our colo provider in one location, and a tunnel to our office, but two years in we're still only testing it for this reason - there's never enough time, and it's not yet urgent enough.
weland 13y ago
> If anything, the people who benefit the most are those who _can't_ afford massive forward deployed CDN networks, large servers, and fat network pipes. You win some, you lose some. The benefit of cutting down on some infrastructure should be weighed against the mind-boggling complexity of HTTP 2.0 and the added difficulty in debugging.
alayne 13y ago
For Google, a .0001% reduction in user overhead may translate to millions of dollars of resource savings. Google is willing to do crazy things like run their own modified Linux kernel that most companies wouldn't bother with. What is the end-user benefit in performance using HTTP 2.0? The only analysis of SPDY I saw indicated low single digit percentage benefit vs HTTPS. Whoop-dee-do. http://www.guypo.com/technical/not-as-spdy-as-you-thought/ http://www.guypo.com/technical/not-as-spdy-as-you-thought/
- lstamour 13y ago
  It's funny, isn't it? We gave out more domains to speed things up, but now it slows us down (referring to your link). I would highlight that HTTP 2 > SPDY as far as standards go, and the "beta" protocol of SPDY is already at 55% market share including IE 11: http://caniuse.com/spdy http://caniuse.com/spdy For those who are interested in speed optimization for 2013 and beyond, plus why SPDY's faster-than-SSL descendants are crucial to the web's success, have a look at http://www.igvita.com/slides/2013/breaking-1s-mobile-barrier.pdf http://www.igvita.com/slides/2013/breaking-1s-mobile-barrier... -- I believe there's a video out there too somewhere. Edit: Video link is on first page of the slides. What's interesting too is that this talk focuses as much on what's achievable without SPDY as what might be with, down the road. We really need faster SSL negotiation for that first time connection cost.
hcarvalhoalves 13y ago
I see the proposals of binary protocols, push content, etc. as symptoms of a deeper issue. Different than the spirit of HTTP, those protocols have little to do with publishing content anymore. They are just more kludges, in the history of kludges, to patch browsers into application platforms. That mainly benefits the big web monopolists, who require the browser to be the ultimate application platform, where they can track to their hearts content, display unsolicited advertising, and basically extort business to advertise on their channels to remain relevant on the web. Not quite the idea of "information repository" that spawned the web in the first place.
- lstamour 13y ago
  Right. Because the alternative, bloated pages out the wazoo makes so much more sense than having smarter Apache and Nginx servers who will do all this for you. Remember, most of the web will be connected by cell phones, we need better protocols for that.
  - hcarvalhoalves 13y ago
    Did you read the comment or just skimmed? I don't question the technical merits, but the underlying motivations, which I believe are fatally flawed.
    - lstamour 13y ago
      Read. That's why I mentioned Apache and Nginx, since the Google SPDY team has focused on making sure that there are plugins for each, plus CloudFront support, so that it can be applied "to the little guys". And in turn, I would ask you: Have you not used Web Sockets? How about HTML5 Web Components? New JS functionality? As "hackers", I would expect more excitement than this -- since you're going to enable a whole new generation of low latency cell phone apps, OSes based on web browsers and apps for them (e.g. Win8.1, ChromeOS and Webkit) and do so in a very standards-based way, rather than ad-hoc distributing SPDY through your proprietary browser. Short of security flaws like SSL's header compression, what could go wrong? :)
    - 13y ago

joosters 13y ago

I agree with most of the article, but would strongly argue against any use of the 'robustness principle'. There have been huge security vulnerabilities because systems disagree with each other about what is a valid request and response.

skrebbel 13y ago
And billions of money wasted because browser vendors, at some point long ago, decided to accept invalid HTML.
- enqk 13y ago
  and lots of experiments, new websites and happy users appeared because HTML has been accepted leniently
  - pbsdp 13y ago
    And we've wasted vast mountains of money dealing with the resulting brokenness of the web. Those people would have figured out how to ship a web page; they'd just would have had so much trouble figuring out why one they shipped was't working, because the browser would have told them.
  - skrebbel 13y ago
    would those experimental users have given up right when the browser reported "Missing </p> tag"?
    - TeMPOraL 13y ago
      They would probably be more encouraged and happier to develop even more because they would understand HTML, which is a simple but powerful concept of block markup, and not some random pile of characters in angle brackets.
    - enqk 13y ago
      That end tags were an option and not mandatory was a feature of the SGML language that HTML was based on. I'm not sure specifically if this was allowed in the description for the P tag, however I suspect it is. So this was entirely by design and probably not all that difficult to implement. That it might not be appealing to some I can understand. I suspect the creators of SGML had very different concerns. I like that HTML has a very humane interface.

osth 13y ago

Amen, OP. herge: You could also use the "data" program from Hobbit's netcat.

zackbrown 13y ago

This sounds similar to an argument you could make for writing all of your code in assembly instead of C (or other higher-level languages.) Yes, there's a layer of obscurity between a program written in C and the resulting assembly: a tool (yes, a tool, spouting abstraction all over the place!) called a compiler. As a core tenet of engineering, we create tools that often leverage abstraction in order to let us tackle more complex problems without wrapping our heads around (or writing code to handle) the details. It's a matter of balance, and we can all choose our own camps along the abstraction <=> ease-of-understanding-every-detail continuum (I'd tend to plop myself toward the former.) Just as we can create a reliable C => assembly compiler, we can create reliable tools to let us leverage abstraction for the better and focus on the bigger picture rather than the smaller.

scott_s 13y ago
I find your analogy curious, because I think it's backwards. If we're going to use C and assembly, the author is arguing for C: easier for humans to read, but we'll have to translate (compile) it so that the machine can read it. Assembly is easier for machines to read (well, not really), but harder for humans to read. My "not really" bit is because the better analogy would be C to machine code. The author's point is that HTTP/1.1 is a text-based protocol. It's inefficient that we have to translate that text into meaningful bits on either end, but it makes it easier for humans to inspect. HTTP/2.0 allows for binary communication, removing the need to translate text, but making it harder for humans to read. It's removing layers of abstraction, not adding them. The introduction is quite clear on this point: The Hypertext Transfer Protocol (HTTP) is a wildly successful protocol. However, the HTTP/1.1 message format is optimized for implementation simplicity and accessibility, not application performance. As such it has several characteristics that have a negative overall effect on application performance. I'm not necessarily saying I agree with the author; I don't know enough about the tradeoffs in this domain to say if the loss of abstraction is worth it. But I do think it's worth understanding his arguments on its merits.
- zackbrown 13y ago
  I think we're addressing different points in the article, especially in that 'readability' and 'complexity' are distinct. Also, I think the author does make valid points, which is why I qualified that we can all choose our preferred places on that continuum of abstraction. My point is that it is sometimes (or from my perspective, often) worth it to deal with more complex (as in higher-level) technologies via tools that allow us to de-abstract those technologies than to focus on using only technologies for which we can understand every detail of their implementation. Trying to understand assembly written by a human vs assembly spat out by a compiler (with its optimizations, etc.) is a very different beast, but thanks to the tools of higher-level programming languages and compilers we rarely need to fiddle around with machine-generated assembly. Similarly, with reliable tools, we would not need to fiddle around with raw HTTP 2.0 workings, and thus rarely would the concern of immediate human-readability be an issue.

Afforess 13y ago

I don't get the fuss. HTTP 2.0 is not a deprecation of HTTP 1.1/1.

jere 13y ago

>At first, this struck me as a very odd thing; text is inefficient, and machines, not humans, are meant to interpret protocols. A binary setup would save bytes—bytes!— and be all-around more manageable by software. >It wasn’t long, however, before I realized the true genius behind this decision. This reminded me very much of a talk by Jonathan Blow where he's talking about how he "nerd raged" while reading the Doom source code. He found a section that wasn't optimized, but in fact not optimizing it had so many other benefits (besides, the performance gain would have been negligible). Starts at 14 minutes: http://the-witness.net/news/2011/06/how-to-program-independent-games/ http://the-witness.net/news/2011/06/how-to-program-independe...

mbq 13y ago

Oh sure, let's keep buffer bugs, waste tons of bandwidth and storage, burn millions of CPU hours in parsing computer-generated text, completely ban encryption and screw it for all non-en_US-speakers so that it could be debugged via misuse of a protocol from 1983 that no one sane uses.

jbert 13y ago
I watched simple text-based internet protocols eat the efficient, ASN.1 OSI protocols for breakfast. From the OSI side of the fence. You can talk about efficiency until you're blue in the face, but human-readability is very useful. I used to be able to pretty much parse a hexdump of an X.400 P1 PDU by eye (certainly if I could reformat in a text editor), but even today I found it useful to eyeball a recalcitrant programs's HTTP request/response cycle by simply catching it under strace and grepping for "HTTP". It's a massive reduction of friction to have human-readable protocols. I used to make the argument that it didn't matter and I was wrong.
- zxcdw 13y ago
  > * but human-readability is very useful.* For who, how often and when exactly? As far as I have understood, we are doing something wrong if we have to deal with debugging protocols which implement an abstraction, rather than just let the abstraction do it's job. This is like saying that "Well for programmers x86 assembly instruction mnemonics are useful compared to machine code bytes!", to which one could say that for an average programmer that makes no sense. I didn't quite care about stuff like this until I got introduced to information theory and really started thinking about what it means to send and receive information. The amount of totally unnecessary waste is astounding.
  - dsr_ 13y ago
    I'm starting to feel like I start all my HN posts with "Hi, I'm a sysadmin." Hi, I'm a sysadmin. Sometimes we have to debug things that are on the other side of the world. When we do that, we need to have a mental model of what we're doing. You're familiar with that as a software developer: you have a mental model of the capabilities of the language that you are using, and you are fitting that in with the mental model of the problem you are solving and the context of existing program code. To sysadmins, protocols are like programming languages. That's why we like text-based protocols: we can fit them in our heads and type them out, slowly, like ancient creaky teletypes that make lots of mistakes and pause in between commands. Meanwhile, we're running down checklists: can we connect? OK, there's no IP-based packet filtering. What does the banner say? Can it support STARTTLS? OK, disconnect and try again with telnet-tls. Hey, it doesn't negotiate... Totally unnecessary waste? Sure, assuming someone has already written a tool which does all the testing for you, and you know that tool exists, and you have access to it right now over your tiny smartphone's 3G connection at 0400 while you thought you were on vacation.

antitrust 13y ago

The original idea behind the web was a publishing platform that, like a classic car, would be easy to explore, maintain and develop. The new vision is the application. Whether it's on a phone, or running as a web site, these are not publishing so much as services. Perhaps it's time to fork protocols: one for publishing, one for interactive applications.

weland 13y ago
This has been on my mind, too, for a long time. Many applications don't fit the semantic framework of the WWW anyway; why not call it a day and fork it? This way the web app crowd will be able to do their somethingJS magic without worrying about what the WWW was meant to be, and the people who use WWW for publishing can get a sane platform back on track.
- hollerith 13y ago
  That would be a big win if you can pull it off. I wonder what is the best way to do it. convincing all owners of web apps to migrate to a new infrastructure strikes me as extremely hard. convincing the millions of 'publishers' of web writings (including all bloggers) to migrate sounds really hard, too.
  - pbsdp 13y ago
    What do you think 'native' platforms are? They're the platforms built to support app development.
- mbrock 13y ago
  This is all very interesting... I feel that the document-centric request-oriented nature of the web has been a powerful influence on the design of "web applications." But we still haven't really figured out how to do it. The web has been full of applications since the beginning, no? There was no golden age of pure document sharing. So much of the web throughout its history has been CGI, ASP, JSP, ad hoc setups to provide dynamic sites. Session cookies, inscrutable URLs, forms with two dozen hidden fields encoding cryptic parameters... (Here's a www-talk thread from 1993 discussing the new script support in NCSA httpd http://1997.webhistory.org/www.lists/www-talk.1993q4/0485.html http://1997.webhistory.org/www.lists/www-talk.1993q4/0485.ht...) But since the formulation of the REST architecture, it seems that more and more people are interested in what the WWW was meant to be. Maybe we're only beginning to understand the possible implications of the WWW design, and to purify that into a powerful conceptual framework. Does that essentially involve a text-based HTTP protocol? I don't think that's the most important part. The REST philosophy seems to say that the sane platform of the web is more crucially about clearly defined paths to typed resources providing a uniform set of actions and discoverable associations. This is not just for publishing -- what makes it so interesting and fruitful is the way it can be used to structure many kinds of applications in the form of published resources... "Calling it a day and forking it" seems in some way to mean giving up on this fruitful encounter between two paradigms. Then you get a binary socket protocol with no resource structure but with a lot of potential for shiny stuff, versus a document-centric protocol that's slower and more restricted... And then all the big players do the shiny thing, and the document protocol is left for enthusiastic hobbyists and legacy applications...

breadbox 13y ago

If you're tempted to argue against this by pointing out that very simple tools can make irrelevant the change to a non-textual protocol -- then you are, I suspect, missing the point. There's a real psychological difference between debugging something that is, at bottom, human-readable versus one that isn't. I'm willing to concede that it isn't really that big of a difference. For the sake of argument I'm even willing to concede that it's largely illusory -- because that doesn't matter. Who cares how irrational it may be? If it's that prevalent in the human psyche, you're just better off not fighting it. Like scripting languages, a text-based protocol doesn't just make it easier for you to get your hands dirty: it practically begs you to. The value of that sort of encouragement to the adoption of a global standard should not be underestimated.

Afforess 13y ago
HTTP 2.0 is not a deprecation of HTTP 1.1/1. You can have your cake and eat it too.
- breadbox 13y ago
  Never thought that it was. This is just another reason why HTTP 2.0 adoption is going to be an uphill battle.

wmf 13y ago

This text vs. binary discussion is pretty high level. HTTP/2.0 has a mux layer (let's temporarily put aside whether that's a good idea) and some of the payloads have to be binary (e.g. JPEGs). There are basically three ways you can do this: A. base64 everything. Obviously this has high overhead. B. Escaping (aka byte stuffing). This is somewhat slow to escape and unescape, the overhead is variable (in rare cases 100%), and it's fragile to read or write by hand. C. Byte counting. This is the most efficient, but extremely inconvenient to write by hand. You could create an efficient "text" mux protocol (basically BEEP), but the result is so non-human-readable/writable IMO that I don't think people would be any happier. HTTP/2.0 is not complex because it's binary; it's binary because it's necessarily already complex enough that text doesn't save you anything.

snuxoll 13y ago
You have many of the same issues with HTTP/1.1 pipelining.

mjn 13y ago

My personal "wow, it's just this?" moment came when, in high school, I figured out how to IRC from a Unix box that didn't have an IRC client installed, just by telnetting to the IRC server. The only particularly annoying part was that you'd get disconnected if you were idling and didn't respond to a PING with a PONG quickly enough. It was also a fun way to learn programming. mIRC at some point added a sockets interface to mIRC script, and after that it was pretty easy to write toy clients for various text-based protocols, doing simple things like checking whether a URL was a 404. I can't say I did anything particularly useful with that, but it was a nice way to learn something about programming.

thibauts 13y ago
I remember just about the same, some 15 years back. Discovering networks and being able to mess with them, even though my programming skills were still very basic. This is a good part of what made me enjoy programming. Exploring this world, feeling it within grasp. I think TAOUP sums up a few important points about text protocols [1] and why going binary is not always wise. Obviously the main drive here is money and I'm sure we all have stories where short term money translates later to large unforseen and/or hidden costs. Bandwidth capacity is growing exponentially so do we really need this, even for the savings ? I'd rather see people invent clever new ways to do wonders with text streams than go back to opaque and obscure data structures that bring back to me rememberances of old days proprietary protocols. After all, if our fathers made the choice of text streams at a time when each byte cost much more than now, there may be good reasons. [1] http://www.catb.org/esr/writings/taoup/html/ch05s01.html http://www.catb.org/esr/writings/taoup/html/ch05s01.html

jperras 13y ago

For some reason, I feel as though the people in this thread against text-based protocols have never used telnet to inspect or debug an HTTP request/response. Or used telnet to help develop a client for something like memcached. The ease of discoverability of a protocol should not be underrated.

josteink 13y ago
Yeah. For some things seeing truly is believing. I guess if you've never had that moment where it all clicked for you, just by looking at traffic, the significance of human-readable protocols is easy to underestimate. But hey, Google says that with binary-compressed headers of sorts, they can fit all their tracking cookies in one TCP-packet and that means you wont feel inconvenienced by the tracking payload they are throwing at you everywhere you go on the internet. You know what? I want that huge bag of tracking cookies to hurt. If you violate my privacy, that should come at a cost. Google: You can take your SPDY and keep nice, old HTTP alone.
rjsw 13y ago
Writing new protocol modules for tcpdump isn't difficult.
- mzs 13y ago
  After that you can see what? You also have to use something like sed, xxd, and nc at least to make a request unless you are really good with bitfields and a hex editor. You want to keep a connection open or use what comes back for he next request? Maybe you can quickly cobble together a crude client in expect and tcl, perl, or python.
- iso-8859-1 13y ago
  A lot more difficult than using netcat though.
flipperypokery 13y ago
I feel as though the people in this thread against (well-designed) binary protocols have never tried implementing a binary protocol. Length-prefixed fields and direct encoding of binary data go a long way to simplifying implementing a protocol. The ease of implementation of a simple binary protocol should not be underrated.

kyrra 13y ago

While I agree that it sucks to lose text based protocols, we already have an underlying protocol that is completely binary in nature and requires tools to look at it (TCP). Same can be said for IP. We've survived a long time with those technologies and we have some awesome tools for working with them. While it's not as easy as ascii based protocols, it's more space efficient, which I'm fine with.

gdwatson 13y ago
But one of the beautiful features of TCP is that it has the properties necessary to treat it as a text stream -- you can build text-based protocols on top of it. Since troubleshooting an application-level protocol seldom requires you to delve into the TCP-level, it works very well in practice. The HTTP 2.0 draft not only makes the application-level protocol binary, but it eschews the simpler stream metaphor in favor of replicating a bunch of that TCP-level complexity in the higher layer. There are reasons for all of this, of course -- it's not being proposed on a lark. But it does have a very real complexity cost.
- lstamour 13y ago
  So don't support HTTP 2.0 if it doesn't benefit you and your use case. It's that simple. In a world where HTML5 says browsers need to pretend to be Netscape Navigator when asked, you'll never lose HTTP 1.x no matter what happens. Never.

lstamour 13y ago

Am I the only one who thinks its ironic that the title references 7-bits? This shows that you're already binary encoded enough that your terminal has to parse it out. Why not just have a dumber terminal? Plus, it's SSL based. SSL means you're already used to not looking at things through telnet. Are you really saying we shouldn't use SSL because "it's not just 7-bit plaintext"? That's the point, after all...

jimktrains2 13y ago
> SSL means you're already used to not looking at things through telnet. Someone has never used `openssl s_client` before. Very few people actually delve into and edit the TLS protocol. People, including devs, often delve into the HTTP protocol and craft requests by hand, or with a simple text-processing script. To most people, TLS is transparent. This HTTP 2.0 protocol is not transparent.
- lstamour 13y ago
  If you're using openssl s_client, why not use another such client? That's my point. The fact is, you type ascii because it's inconvenient to type binary, but you have software interpret bytes, so why not send what output you want, monitor what you want, using new tools. Haven't you ever used Fiddler, Charles Web Proxy, WireShark or anything else that makes binary protocols easier? Oh wait, yes, you have: openssl. My point exactly. You'll be able to unwrap the binary to ASCII representations as much as you like, no question, no harm, no foul. Or it won't get adopted.
  - jimktrains2 13y ago
    Because I'm not working at the TLS layer, I'm working at the HTTP layer. I'm sure if I had to often write TLS handshakes by hand, s_client wouldn't be my tool of choice, just as curl isn't always my tool of choice. Making the protocol I'm working in binary doesn't do me any favors.
    - lstamour 13y ago

weinzierl 13y ago

"I’ve been in this business for two decades, and the only way I know to hand-debug a SOAP transaction is with a hammer and a straitjacket2. Heck, I’ll take an old-school EDI transaction over SOAP any day." SOAP? You lucky. It's 2013 and I'm still knee deep in RMI-IIOP sometimes. Wireshark with GIOP dissector helps, but I’ll take SOAP over CORBA any day:-). Seriously: Your post summarized very well what I was thinking when reading the recent HTTP 2.0 posts. Full Ack.

D9u 13y ago

The site crashes my phone browser. Dolphin Mini. HTC Wildfire. Android 2.2.2 I guess I'll check it out later.

joshaidan 13y ago

I work in telecom, and this makes me think of two protocols I work a lot with, SS7 and SIP. SIP is like HTTP, in fact I believe it was modelled after HTTP and has a lot in common with it. You can troubleshoot SIP issues very easily using a packet inspector like ngrep (using Telnet might be a bit difficult as there are some timers that expire if you don't respond fast enough, but that's besides the point). Then there's a protocol like SS7 which is binary based. It's all structure binary bit fields. Even though it's binary based, and not very human readable, we have tools that decode the bits into a human readable format, which in turn makes it just as easy as SIP to troubleshoot. The question I ask myself is, which one would be easier to implement, text based or binary format? I guess I'd lean towards text based--in fact I have implemented a SIP client myself with success. But have yet to implement any kind of SS7 or binary based protocols. Perhaps experience has something to do with it, would having experience implementing binary protocols make it any bit easier? And to be fair, SIP is also much well more documented then SS7. As well it's easier to test SIP. I'm just starting to work with binary protocols, so it will be interesting how much progress I make. I realize there are also some libraries that help with working with binary/bit field data.

toast0 13y ago
binary protocols can be easier to develop, because generated and parsing the protocol stream is usually straight forward: if there's a variable length field, you will typically get the length first and then the field; no need to scan byte by byte for a terminator(s) or handle escaping of terminator(s). There's often less need to be liberal in what you accept too (but that may depend on how the protocol is versioned) But if you're developing for a protocol with garbage documentation (or no documentation), a text based protocol at least offers some amount of intrinsic documentation.

comex 13y ago

I don't think a simple binary protocol for HTTP would be bad at all. HTTP servers are still going to support text-based connections forever, so it only matters if you're snooping on existing traffic; but the ability to use Wireshark or tcpdump to grab the text out of packets directly is already disappearing because of encryption (SSL), which is such a good thing in general that it far outweighs that drawback. Therefore more specialized tools are required, and those may as well parse such a protocol - it wouldn't be that hard. The compression is a significant benefit, and can't be done well below the level of HTTP. What I don't understand is why we need both SPDY and QUIC. SPDY provides fast multiplexing, compression, and guaranteed SSL. QUIC provides fast multiplexing and guaranteed SSL at the transport(-ish) layer. QUIC does at least one thing that can't be done at the application layer - faster opens - and seems nice to have as a generic base for multiple protocols, so I call it a good thing (and I'd like to see it in the kernel). But the plan seems to be to run SPDY over QUIC - I can't actually find enough information on the Internet (and am too lazy to look through the source) to find out whether packets are going to be double encrypted for the time being, but even if/when that is avoided, the multiplexing and optional encryption seem to be wasteful complexity. I would prefer if HTTP/2.0 were a simple and easy-to-parse protocol providing compression only, expected to be used over QUIC.

wmf 13y ago
As I see it, SPDY/HTTP 2.0 is for the next ten years. QUIC is for the ten years after that.

makmanalp 13y ago

I don't get all the for-telnet arguments. Who uses telnet for this anyway? I'd bet that 90% of telnet-folk do: cake:~ mali$ telnet google.com 80 Trying 74.125.235.8... Connected to google.com. Escape character is '^]'. GET / HTTP/1.1 ....... HTTP/1.1 200 OK Date: Wed, 10 Jul 2013 22:25:02 GMT Expires: -1 Cache-Control: private, max-age=0 Content-Type: text/html; charset=ISO-8859-1 At which point they go "oh cool" and go do something else. For the rest of us who use curl, wget, requests, Chrome / Firefox developer tools every day (who are we kidding, it's everyone, you liars! :P), the binary transformation would be transparent. Hell, if you're going for pure cool-factor, how is pulling out your hex editor less cool? But in reality, you'd never do this. For a non-standardized and obscure protocol where tooling would likely be lacking, I can see why human readability is a good idea. But we're talking about the very protocol that makes up the fabric of the internet. Seriously, why? Give me one good reason.

jimktrains2 13y ago
Have you never had to send a specific request/response to a server? I've often found it easier to store the request in a text file and pipe it instead of using curl or another tool to craft it. When I was a webdev (over the past few years) I did this at least once a month. I'd daily look at the HTTP headers, though. Also, I'd often add custom headers for debugging purposes. Edit: It's also not just telnet, it's being able to use simple scripts to automate requests or responses for many reasons.
- comex 13y ago
  Most likely all HTTP/2.0 servers will continue to support HTTP/1.1 forever, so you will still be able to do this.
  - jimktrains2 13y ago
    And clients will need to support it. I feel like the HTTP/1.0 to 1.1 switch allowed for a good deal of code reüse, but this won't.

zokier 13y ago

I think TLS/SSL is the elephant in the room here. A large part of the web is effectively already using binary protocol (HTTPS), and frankly I'd wish the part was even bigger. Think of HTTP 2.0 as a successor for HTTPS instead of HTTP if it makes you feel better.

sergiosgc 13y ago
Wrong. An encrypted text session is different from a purely for binary session (encrypted or not). The former can be mangled from text by a generic encryption proxy, the latter needs a protocol-specific analyser.
- 9h1d9j809s 13y ago
  I don't get it. Why does it matter if your tool just decrypts SSL or you use a tool that decrypts SSL and additionally unpacks the binary representation of HTTP 2.0? The output is exactly the same - plain old HTTP.
  - denzquix 13y ago
    The point is that HTTP may not be the be-all end-all protocol that we will ever want to use on the open web, and encouraging any potential new protocols to take HTTP's lead and use an arbitrary binary encoding that requires protocol-specific tools to make sense of, is maybe not a decision we should take lightly.

deleted 13y ago

[deleted]

gohrt 13y ago

> You should follow me on Twitter. This meme needs to die. It's like ending every post with "Just kidding, I don't know what I'm talking about, I'm just cargo-culting blog attention-desperate person trying to build my Klout score to impress myself."

breck 13y ago

You may find this thing that we're working on relevant: https://github.com/nudgepad/space https://github.com/nudgepad/space It's a very understandable and very powerful language that I think will at some point be extended to replace HTTP, amongst other things.

jfarmer 13y ago

Binary vs. text is one dimension and it shouldn't be conflated with the overall alleged complexity of the HTTP/2.0 proposal. You can have robust and non-robust binary protocols. I don't buy the "argument from visibility", for example. HTTPS and DNS are both binary protocols and you need tools to parse them in order to get telnet-like visibility. Does that mean when it comes to those protocols "we are forced to rely on other tools to hide the underlying complexity and dumb things down to a point where they are manageable", as the author says?

cmccabe 13y ago

Marco claims that HTTP 2.0 is overly complex, but never gives any reasons why. He claims that text protocols are better than binary, but never really gives any reason for that either. Nostalgia for the days of telnet and gopher is not a reason. Then he goes off to talk about SOAP, which he hates, because "the only way [he] know[s] to hand-debug a SOAP transaction is with a hammer and a straitjacket." But SOAP is text based, which directly contradicts his argument from earlier. The other allegation here is that the new protocol "satisf[ies] what are essentially the needs of a small group of very influential players." Again, Marco just states this with no rationale at all. In fact, the opposite is true. By being more efficient, HTTP 2.0 will serve the needs of everyone. Should startups have to pay for 10 servers when they could have paid for 5 with HTTP 2.0? Apparently Marco thinks they should. It's easier for Google to pay for a few more servers than it is for Joe startup. This should be obvious to everyone, but apparently it's not. The whole binary versus text thing is a complete red herring. If you have ever used wireshark, you know that it makes binary fields easy to read. Binary protocols are easier to develop and standardize. Pretty much every good developer realizes this, and the responses here reflect that. The idea that making something LOOK like (but not actually be) "human text" makes anything better is an idea we should have buried along with COBOL and other mistakes from the past. But I guess not.

jpatokal 13y ago
SOAP is to REST as XML is to JSON. They're all technically 7-bit and notionally human-readable, but SOAP/XML are (well, can be) so verbose as to be near-impossible to parse by hand. Comparing the "Hello World"-style simple examples on Wikipedia is instructive: REST http://example.com/resources/item17 http://example.com/resources/item17 SOAP <?xml version="1.0"?> <soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"> http://www.w3.org/2003/05/soap-envelope"> <soap:Header> </soap:Header> <soap:Body> <m:GetStockPrice xmlns:m="http://www.example.org/stock"> http://www.example.org/stock"> <m:StockName>IBM</m:StockName> </m:GetStockPrice> </soap:Body> </soap:Envelope>
- snogglethorpe 13y ago
  Amen. "Human readable" is not the same thing as human readable.

CrLf 13y ago

I love these discussions about text vs. binary protocols. I especially love the part where people defend binary protocols because they are more "efficient" and don't eat as much bandwith or processing resources. May I remind these nice people that most text-based protocols we rely on today were designed in a time when CPUs were orders of magnitude slower and modems ran at 2400 baud? Talk about processing and bandwidth resources... The resources consumed by having to interpret a text-based protocol pale in comparison with the resources consumed by all the levels of abstraction that modern frameworks/language runtimes that we use today. But wait, these save developer time, so they are a good thing. So does having protocols that can be troubleshooted by a human with minimal tools.