5 ms·

HTTP/2 – How did we get here

215 points by kamranahmed_se 10y ago

pg_is_a_butt 10y ago
uh... HTTP/1 + 1
formula1 10y ago
Really great article. Outlines history, evolution and problems solved by each iteration. One thing they didnt get into was the methods (options, patch, put, delete, etc) i always found that one to be weird as it seemed proprietary. After a bit of research [0], i found that it was to because the creator had identified multiple ways they wanted to modify a document. I find that facinating because there is no "pay" method or "mutate 5 objects in a database and send 2 emails but make sure that the post data is encrypted" method. Perhaps the creator didnt have the forsight in simply specifying whether data should be parsed from the url or from a body. Perhaps the http method still has more room to grow [0] https://www.quora.com/What-is-the-history-of-HTTP-verbs-PUT-GET-POST-and-DELETE https://www.quora.com/What-is-the-history-of-HTTP-verbs-PUT-...
morecoffee 10y ago
How we get here? By pushing the previous protocol to the limit, observing where it breaks down, and fixing those things. We could even predict what will replace HTTP/2 by looking at where it starts to break down. I work with HTTP/2 daily, and there are some pain points when running at high speeds: - Headers are still head of line blocking. You must synchronize sending them to maintain the HPACK table state. At a high number of requests per second, this is a bottle neck - Running over TLS is CPU bottleneck since encrypted messages are sequential. We get around this by making multiple TCP connections. - Long lived HTTP/2 connections will often break because of NAT's (home internet), or changing IP address (mobile). A single dropped packet kills throughput for highspeed links too. All of these are addressed by the QUIC protocol. I suspect that eventually HTTP/2 will be last major protocol over TCP because of most of the aforementioned problems come from running over it.
- the8472 10y ago
  > Running over TLS is CPU bottleneck since encrypted messages are sequential. ChaCha20 takes around 4cycles/byte on a xeon. At 3GHz that's 750MB/s. Or about a 10Gbit pipe fed with a single core, i.e. without any need for parallelization.
  - rdtsc 10y ago
    Is that running in a loop or mixed in a larger application context where cache lines might be flushed and replaced by others. That often make a things behave very differently when running in small benchmarks vs production.
    - the8472 10y ago
      Small message sizes don't seem to slow it down much according to DJB's paper[0]. It doesn't use large s-boxes or a lot of other internal state, unlike AES does. Its cache footprint is fairly small. [0] https://cr.yp.to/snuffle/salsafamily-20071225.pdf#2 https://cr.yp.to/snuffle/salsafamily-20071225.pdf#2 (page 2)
- niftich 10y ago
  > Running over TLS is CPU bottleneck since encrypted messages are sequential. Does TLS off-loading help in this case? > Headers are still head of line blocking. You must synchronize sending them to maintain the HPACK table state Is this still an issue if you only use the static table, and send any other namevalue-pairs as either dynamic non-indexed [1] or more likely, dynamic never-indexed [2]? [1] https://http2.github.io/http2-spec/compression.html#literal.header.without.indexing https://http2.github.io/http2-spec/compression.html#literal.... [2] https://http2.github.io/http2-spec/compression.html#literal.header.never.indexed https://http2.github.io/http2-spec/compression.html#literal....
- yeukhon 10y ago
  The TLS one, is that specific to HTTP/2, or TLS in general? That's a important distinction. NAT one is interesting though.
- 10y ago
deleted 10y ago
[deleted]
marvy 10y ago
One thing that is slightly misleading about the article is that it talks as if HTTP/1.0 first came into being in 1996, and everyone was using version 0.9 before then. In fact, one need only observe that the original spec for adding cookies to HTTP came out in 1994 to realize that HTTP/1.0 must have been widely deployed by then. In fact, it is only the standard that took until 1996 to write up.
- gsnedders 10y ago
  Even then, nobody really implemented 1.0 per-se, because nobody really changed to match what the standard (ex postfacto) said.
  - marvy 10y ago
    Wasn't the spec designed so that pretty much everything was already mostly conformant?
Scaevolus 10y ago
Correction: HTTP/1.0 doesn't have Host headers at all, while HTTP/1.1 requires them. It's one of the easiest ways to distinguish the two versions. Host headers enable virtual hosts (multiple domains on one IP), letting the Internet grow to billions of domains without requiring an IP for each of them.
- marvy 10y ago
  Neat! I thought host was optional, I didn't realize it didn't exist at all.
- fanf2 10y ago
  Host headers were widely deployed before HTTP/1.1 - IIRC MS IE 3 was the first version to support Host: in 1996 - Netscape supported it earlier.
iamspoilt 10y ago
Nice post!
131hn 10y ago
Iv been upset with HTTP/2 / SPDY since the very first beginning (yet i never used it, nor try). Now, thanks to you, i understand why. It's not a replacement/ enhancement of HTTP, it's definitively not a Transport layer. It has no use/gain for webservices, and as single page apps tends to optimise resources delivery (webpack, browserify, scss, stylus & co, very few resources are left to deliver) so no urgent "need" for it here. So, for an _application layer_ it's comming a little late, i guess "block SPA" with intrisected client side & server side (nodejs/ express & spa) apps might gain of it, but it have to be considered as the very first line of the app developpement , the transition/ enhancement cost / gain ratio for existing apps is too hard to plan. Great writing !
- axman6 10y ago
  I actually disagree with both your conclusions - the techniques you mentioned exist to get around painful deficiencies with HTTP/1.1, and are anti patterns in http/2 - sprites exist to reduce the latency of loading many small images, which is now no longer an issue (see httpvshttps.com for a demonstration). Also my brief experiments using HTTP/2 in Haskell involved zero changes to my web service handler code, and maybe one or two extra lines to give paths to the TLS cert files. Web services will benefit massively from HTTP/2 in some domains, I do a lot of web geospatial work and using it for servers which return hundreds of image tiles will be a massive win
Sami_Lehtinen 10y ago
Many people think that the HTTP was the ultimate invention, often doesn't seem to know that it's clearly evolution of Gopher. https://en.wikipedia.org/wiki/Gopher_(protocol) https://en.wikipedia.org/wiki/Gopher_(protocol)

215 points by kamranahmed_se 10y ago

uh... HTTP/1 + 1

Really great article. Outlines history, evolution and problems solved by each iteration. One thing they didnt get into was the methods (options, patch, put, delete, etc) i always found that one to be weird as it seemed proprietary. After a bit of research [0], i found that it was to because the creator had identified multiple ways they wanted to modify a document. I find that facinating because there is no "pay" method or "mutate 5 objects in a database and send 2 emails but make sure that the post data is encrypted" method. Perhaps the creator didnt have the forsight in simply specifying whether data should be parsed from the url or from a body. Perhaps the http method still has more room to grow [0] https://www.quora.com/What-is-the-history-of-HTTP-verbs-PUT-GET-POST-and-DELETE https://www.quora.com/What-is-the-history-of-HTTP-verbs-PUT-...

morecoffee 10y ago

How we get here? By pushing the previous protocol to the limit, observing where it breaks down, and fixing those things. We could even predict what will replace HTTP/2 by looking at where it starts to break down. I work with HTTP/2 daily, and there are some pain points when running at high speeds: - Headers are still head of line blocking. You must synchronize sending them to maintain the HPACK table state. At a high number of requests per second, this is a bottle neck - Running over TLS is CPU bottleneck since encrypted messages are sequential. We get around this by making multiple TCP connections. - Long lived HTTP/2 connections will often break because of NAT's (home internet), or changing IP address (mobile). A single dropped packet kills throughput for highspeed links too. All of these are addressed by the QUIC protocol. I suspect that eventually HTTP/2 will be last major protocol over TCP because of most of the aforementioned problems come from running over it.

the8472 10y ago
> Running over TLS is CPU bottleneck since encrypted messages are sequential. ChaCha20 takes around 4cycles/byte on a xeon. At 3GHz that's 750MB/s. Or about a 10Gbit pipe fed with a single core, i.e. without any need for parallelization.
- rdtsc 10y ago
  Is that running in a loop or mixed in a larger application context where cache lines might be flushed and replaced by others. That often make a things behave very differently when running in small benchmarks vs production.
  - the8472 10y ago
    Small message sizes don't seem to slow it down much according to DJB's paper[0]. It doesn't use large s-boxes or a lot of other internal state, unlike AES does. Its cache footprint is fairly small. [0] https://cr.yp.to/snuffle/salsafamily-20071225.pdf#2 https://cr.yp.to/snuffle/salsafamily-20071225.pdf#2 (page 2)
niftich 10y ago
> Running over TLS is CPU bottleneck since encrypted messages are sequential. Does TLS off-loading help in this case? > Headers are still head of line blocking. You must synchronize sending them to maintain the HPACK table state Is this still an issue if you only use the static table, and send any other namevalue-pairs as either dynamic non-indexed [1] or more likely, dynamic never-indexed [2]? [1] https://http2.github.io/http2-spec/compression.html#literal.header.without.indexing https://http2.github.io/http2-spec/compression.html#literal.... [2] https://http2.github.io/http2-spec/compression.html#literal.header.never.indexed https://http2.github.io/http2-spec/compression.html#literal....
yeukhon 10y ago
The TLS one, is that specific to HTTP/2, or TLS in general? That's a important distinction. NAT one is interesting though.
10y ago

deleted 10y ago

[deleted]

marvy 10y ago

One thing that is slightly misleading about the article is that it talks as if HTTP/1.0 first came into being in 1996, and everyone was using version 0.9 before then. In fact, one need only observe that the original spec for adding cookies to HTTP came out in 1994 to realize that HTTP/1.0 must have been widely deployed by then. In fact, it is only the standard that took until 1996 to write up.

gsnedders 10y ago
Even then, nobody really implemented 1.0 per-se, because nobody really changed to match what the standard (ex postfacto) said.
- marvy 10y ago
  Wasn't the spec designed so that pretty much everything was already mostly conformant?

Scaevolus 10y ago

Correction: HTTP/1.0 doesn't have Host headers at all, while HTTP/1.1 requires them. It's one of the easiest ways to distinguish the two versions. Host headers enable virtual hosts (multiple domains on one IP), letting the Internet grow to billions of domains without requiring an IP for each of them.

marvy 10y ago
Neat! I thought host was optional, I didn't realize it didn't exist at all.
fanf2 10y ago
Host headers were widely deployed before HTTP/1.1 - IIRC MS IE 3 was the first version to support Host: in 1996 - Netscape supported it earlier.

iamspoilt 10y ago

Nice post!

131hn 10y ago

Iv been upset with HTTP/2 / SPDY since the very first beginning (yet i never used it, nor try). Now, thanks to you, i understand why. It's not a replacement/ enhancement of HTTP, it's definitively not a Transport layer. It has no use/gain for webservices, and as single page apps tends to optimise resources delivery (webpack, browserify, scss, stylus & co, very few resources are left to deliver) so no urgent "need" for it here. So, for an _application layer_ it's comming a little late, i guess "block SPA" with intrisected client side & server side (nodejs/ express & spa) apps might gain of it, but it have to be considered as the very first line of the app developpement , the transition/ enhancement cost / gain ratio for existing apps is too hard to plan. Great writing !

axman6 10y ago
I actually disagree with both your conclusions - the techniques you mentioned exist to get around painful deficiencies with HTTP/1.1, and are anti patterns in http/2 - sprites exist to reduce the latency of loading many small images, which is now no longer an issue (see httpvshttps.com for a demonstration). Also my brief experiments using HTTP/2 in Haskell involved zero changes to my web service handler code, and maybe one or two extra lines to give paths to the TLS cert files. Web services will benefit massively from HTTP/2 in some domains, I do a lot of web geospatial work and using it for servers which return hundreds of image tiles will be a massive win

Sami_Lehtinen 10y ago

Many people think that the HTTP was the ultimate invention, often doesn't seem to know that it's clearly evolution of Gopher. https://en.wikipedia.org/wiki/Gopher_(protocol) https://en.wikipedia.org/wiki/Gopher_(protocol)