3 ms·

These are all really good points, let me just address one or two. > We are talking about ~300 bytes of data per header ... are the bandwidth gains really so su

by Lukasa 12y ago

These are all really good points, let me just address one or two. > We are talking about ~300 bytes of data per header ... are the bandwidth gains really so superior to outweigh the CPU costs in having to compress the data? The commonly-cited reason for pursuing header compression is making the very first request of a session, when your initial congestion window is small. The goal is to make the request small enough that it can fit in one round-trip so that you don't have to wait for all the ACKs. Reducing duplication is great and will help for a lot of cases as well, but my impression from the WG has been that it's less important than that first round trip. Note also that HPACK is really quite efficient in terms of CPU cycles. It takes relatively few to compress the headers very heavily. Finally, if you really don't like it, you can turn HPACK off. That gives an out for constrained devices that genuinely can't spare the cycles. > Does anyone know how clients are going to go about requesting HTTP/2 capabilities? Is there a special HTTP/1.1 header? Because Apache will respond to "GET / HTTP/2.0" with an HTTP/1.1 OK response at present. Yes, because there's a draft spec. I invite you to read it[0]. The summary is that you have three options: 1. For plaintext HTTP/2, you use a HTTP Upgrade: header, just as everyone expects you to. 2. For HTTP/2 over TLS you use ALPN, a feature of TLS that lets you request a protocol in the TLS negotiation (to avoid an extra round trip). 3. Alternatively, you might be able to use HTTP Alternative Services (a WIP draft). The behaviour Apache is using is clearly a bug. [0]: https://tools.ietf.org/html/draft-ietf-httpbis-http2 https://tools.ietf.org/html/draft-ietf-httpbis-http2

byuu 12y ago
Thanks for the response! > The goal is to make the request small enough that it can fit in one round-trip so that you don't have to wait for all the ACKs. Thank you, that makes sense. I definitely see how bloated some of these headers can get these days with super-long user-agent strings, x-foo values, accepted modes, etc. If this is really going to make a substantial number of initial requests and responses fit into a single round-trip, then that sounds like a decent rationale. > Note also that HPACK is really quite efficient in terms of CPU cycles. Is there a C implementation of it? I had quite the bitch of a time to implement INFLATE in C, taking about 10KB of super-nasty code at a minimum. I hope that pain isn't repeated here. Something like just plain LZSS would be a real treat because that's maybe five lines of code. Of course, compressing with LZSS isn't very efficient in the naive case. I'm guessing HPACK is probably an entropy coder ... I'll Google around for more on HPACK, thanks for providing the name to me. > Finally, if you really don't like it, you can turn HPACK off. That's really interesting, and good to hear. So that would have to mean the initial request from the client cannot be compressed (eg your Upgrade header), and the server then must decide what to send back. Hopefully we don't end up with any clients that ignore you and start sending their subsequent request headers compressed anyway. > 1. For plaintext HTTP/2, you use a HTTP Upgrade: header, just as everyone expects you to. So is this going to be yet another "your PC boots in 16-bit real mode in 2014", where servers have to keep switching from HTTP/1 to HTTP/2 on each new startup forevermore? :/ I mean, I don't know what the alternative would be other than header compatibility with 1.1 ... In this case, will the server and client be able to leave that initial HTTP/1 Upgrade: connection open, and just start using it as HTTP/2, eg sending data back and forth in turn? (I can't see why not.)
- Lukasa 12y ago
  No problem! > Is there a C implementation of it? Currently there's no good standalone version, because we're still at interop stage. Take a look at nghttp2[0], however. They've got a HPACK layer that I believe can be separated out. I've got a plan to actually do that work for them, but haven't had time to sit down and do it. Nevertheless, theirs is the best implementation around at the moment in terms of compression efficiency. My own HTTP/2 library can optionally use it. > Will all clients respect this, or will they start trying to send you HPACK requests anyway? So we need to be clear on what this means. The actual binary format of headers via HPACK cannot be turned off. However, all the compression can be. This isn't optional: the other party must respect your preferences. > So is this going to be yet another "your PC boots in 16-bit real mode in 2014", where servers have to keep switching from HTTP/1 to HTTP/2 on each new startup forevermore? Hopefully no. Hopefully. You're allowed to start straight away with HTTP/2 if you have some form of 'prior knowledge' that HTTP/2 will work. What 'prior knowledge' actually consists of is not entirely clear-cut, but in principle you can go straight to HTTP/2. Note also that Upgrade: is only a problem for plaintext HTTP/2. If you use TLS, going to HTTP/2 is exactly as fast as going to HTTP/1.1 (because the negotiation is done at the same time as the TLS negotiation). However, servers are clearly going to have to support HTTP/1.1 for a long time to come. This is unfortunate, but unavoidable. [0]: http://nghttp2.org/ http://nghttp2.org/
  - mohamedattahri 12y ago
    When you consider headers per request, we're talking bytes. But at the Internet scale, it's gigantic. Accept: image/gif, image/jpeg, image/pjpeg, application/x-ms-application, application/vnd.ms-xpsdocument, application/xaml+xml, application/x-ms-xbap, application/x-shockwave-flash, application/x-silverlight-2-b2, application/x-silverlight, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, / This is an Accept header on IE8 with Office installed. Back when it had over 55% of market share, Google alone estimated that as much as 38Gb of useless garbage internet traffic was sent its way every single day. And that was back when the Internet population was half was it is today, web apps were were primitive, and mobile and the network constraints were nascent.
pilif 12y ago
>The behaviour Apache is using is clearly a bug. not so sure. RFC2616 in section 3.1 says: The HTTP version of an application is the highest HTTP version for which the application is at least conditionally compliant. So you request HTTP/2, but the server responds in HTTP/1.1 because that's what it understands and tells you so. The server is free to not accept your HTTP/2.0 request, but it's also free to go ahead and respond in HTTP/1.1. The RFC does not mandate a server to not accept requests in versions it doesn't understand. The only thing the RFC says is that you're not allowed to state that your request or response is in a protocol you don't in-fact understand. In-fact, I would argue that, considering Applications that are at least conditionally compliant with this specification SHOULD use an HTTP-Version of "HTTP/1.1" in their messages, and MUST do so for any message that is not compatible with HTTP/1.0 Stating HTTP/1.1 is the only valid thing to do according to the RFC. I know that was likely not the intention, but I would certainly read the RFC that way. Also see section 2.3 in RFC 2145: An HTTP server SHOULD send a response version equal to the highest version for which the server is at least conditionally compliant, and whose major version is less than or equal to the one received in the request. An HTTP server MUST NOT send a version for which it is not at least conditionally compliant. A server MAY send a 505 (HTTP Version Not Supported) response if cannot send a response using the major version used in the client's request. So, again, Apache MAY send a 505 but it's totally allowed to know nothing about HTTP/2.0 yet, parse the request as HTTP/1.x and then respond with a HTTP/1.1 response.

by Lukasa 12y ago

byuu 12y ago

Thanks for the response! > The goal is to make the request small enough that it can fit in one round-trip so that you don't have to wait for all the ACKs. Thank you, that makes sense. I definitely see how bloated some of these headers can get these days with super-long user-agent strings, x-foo values, accepted modes, etc. If this is really going to make a substantial number of initial requests and responses fit into a single round-trip, then that sounds like a decent rationale. > Note also that HPACK is really quite efficient in terms of CPU cycles. Is there a C implementation of it? I had quite the bitch of a time to implement INFLATE in C, taking about 10KB of super-nasty code at a minimum. I hope that pain isn't repeated here. Something like just plain LZSS would be a real treat because that's maybe five lines of code. Of course, compressing with LZSS isn't very efficient in the naive case. I'm guessing HPACK is probably an entropy coder ... I'll Google around for more on HPACK, thanks for providing the name to me. > Finally, if you really don't like it, you can turn HPACK off. That's really interesting, and good to hear. So that would have to mean the initial request from the client cannot be compressed (eg your Upgrade header), and the server then must decide what to send back. Hopefully we don't end up with any clients that ignore you and start sending their subsequent request headers compressed anyway. > 1. For plaintext HTTP/2, you use a HTTP Upgrade: header, just as everyone expects you to. So is this going to be yet another "your PC boots in 16-bit real mode in 2014", where servers have to keep switching from HTTP/1 to HTTP/2 on each new startup forevermore? :/ I mean, I don't know what the alternative would be other than header compatibility with 1.1 ... In this case, will the server and client be able to leave that initial HTTP/1 Upgrade: connection open, and just start using it as HTTP/2, eg sending data back and forth in turn? (I can't see why not.)

Lukasa 12y ago
No problem! > Is there a C implementation of it? Currently there's no good standalone version, because we're still at interop stage. Take a look at nghttp2[0], however. They've got a HPACK layer that I believe can be separated out. I've got a plan to actually do that work for them, but haven't had time to sit down and do it. Nevertheless, theirs is the best implementation around at the moment in terms of compression efficiency. My own HTTP/2 library can optionally use it. > Will all clients respect this, or will they start trying to send you HPACK requests anyway? So we need to be clear on what this means. The actual binary format of headers via HPACK cannot be turned off. However, all the compression can be. This isn't optional: the other party must respect your preferences. > So is this going to be yet another "your PC boots in 16-bit real mode in 2014", where servers have to keep switching from HTTP/1 to HTTP/2 on each new startup forevermore? Hopefully no. Hopefully. You're allowed to start straight away with HTTP/2 if you have some form of 'prior knowledge' that HTTP/2 will work. What 'prior knowledge' actually consists of is not entirely clear-cut, but in principle you can go straight to HTTP/2. Note also that Upgrade: is only a problem for plaintext HTTP/2. If you use TLS, going to HTTP/2 is exactly as fast as going to HTTP/1.1 (because the negotiation is done at the same time as the TLS negotiation). However, servers are clearly going to have to support HTTP/1.1 for a long time to come. This is unfortunate, but unavoidable. [0]: http://nghttp2.org/ http://nghttp2.org/
- mohamedattahri 12y ago
  When you consider headers per request, we're talking bytes. But at the Internet scale, it's gigantic. Accept: image/gif, image/jpeg, image/pjpeg, application/x-ms-application, application/vnd.ms-xpsdocument, application/xaml+xml, application/x-ms-xbap, application/x-shockwave-flash, application/x-silverlight-2-b2, application/x-silverlight, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, / This is an Accept header on IE8 with Office installed. Back when it had over 55% of market share, Google alone estimated that as much as 38Gb of useless garbage internet traffic was sent its way every single day. And that was back when the Internet population was half was it is today, web apps were were primitive, and mobile and the network constraints were nascent.

pilif 12y ago

>The behaviour Apache is using is clearly a bug. not so sure. RFC2616 in section 3.1 says: The HTTP version of an application is the highest HTTP version for which the application is at least conditionally compliant. So you request HTTP/2, but the server responds in HTTP/1.1 because that's what it understands and tells you so. The server is free to not accept your HTTP/2.0 request, but it's also free to go ahead and respond in HTTP/1.1. The RFC does not mandate a server to not accept requests in versions it doesn't understand. The only thing the RFC says is that you're not allowed to state that your request or response is in a protocol you don't in-fact understand. In-fact, I would argue that, considering Applications that are at least conditionally compliant with this specification SHOULD use an HTTP-Version of "HTTP/1.1" in their messages, and MUST do so for any message that is not compatible with HTTP/1.0 Stating HTTP/1.1 is the only valid thing to do according to the RFC. I know that was likely not the intention, but I would certainly read the RFC that way. Also see section 2.3 in RFC 2145: An HTTP server SHOULD send a response version equal to the highest version for which the server is at least conditionally compliant, and whose major version is less than or equal to the one received in the request. An HTTP server MUST NOT send a version for which it is not at least conditionally compliant. A server MAY send a 505 (HTTP Version Not Supported) response if cannot send a response using the major version used in the client's request. So, again, Apache MAY send a 505 but it's totally allowed to know nothing about HTTP/2.0 yet, parse the request as HTTP/1.x and then respond with a HTTP/1.1 response.