3 ms·

Is it the case that HTTP/3, like HTTP/2, will not change the semantics of HTTP in any way? One thing that drives me nuts, for example, is that Expect: 100-cont

Is it the case that HTTP/3, like HTTP/2, will not change the semantics of HTTP in any way? One thing that drives me nuts, for example, is that Expect: 100-continue exists but has no criticality, so a user-agent can't tell at all if the 100 continue will be coming, or if a 2xx, 3xx, 4xx, or 5xx response will be sent immediately. In principle the user-agent could use OPTIONS to detect if the server implements Expect: for the target resource, but there is no guarantee that a server that supports Expect: will note that in responses to OPTIONS. Some clients implement a timeout then begin sending the request body, but suppose when they do so the server's 3xx/4xx/5xx response is already in-flight? Then the server will expect a new request, not a request body, next from the user-agent. What a mess. And, of course, Expect: is a very important feature. It's needed to avoid sending data that cannot be recreated, or without having to cache it in order to resend it if the request failed with a 401, say. What I'd like to see is a hard requirement that OPTIONS must indicate whether the server supports Expect:. I'm not sure much more than that can be done to fix the problem. But obviously, a new transport for HTTP can't change this sort of semantics, as one might be gatewaying to/from HTTP/1.1.

Matthias247 8y ago
Is that really a HTTP/[2|3] problem, or a problem of those servers? As far as I understood the HTTP/2 spec (have read this, but not HTTP/3), it is allowed for servers to send an arbitrary amount of informational headers (incl. 100-continue) before sending the actual headers and response body. So everything to handle these kinds of requests should be in place. If the server ignores the "Expect: 100-continue" there is obviously nothing the other side can do, but that's the same in HTTP/1.1. What I am a bit unsure about is actually how things like "transfer-encoding: chunked" or the various "connection" headers are supposed to be handled in servers/clients/libraries that are supposed to handle HTTP/1.1 and HTTP/2. Is this supposed to be automatically set on the lower level, depending on the actual protocol type. And the application level HTTP client or handler on server just sees the body as a stream? I'm not sure if that happens currently in common libraries, or whether they would forward these things if a client is connected via HTTP/1.1 and not when it's connected via HTTP/2.
- cryptonector 8y ago
  The real is that the server is allowed to defer decisions about authentication and authorization until it has seen some or all of the request body, therefore the server might not respond until it has seen part of the body, but the user-agent wants to know whether it can get an immediate response to a POST/PUT in order to avoid sending the body until it knows the server is ready to sink it, and the user-agent is resource constrained and cannot cache the request body, say: - the user-agent wants an early 2xx/4xx from the server - the server is allowed to defer - the user-agent has no reliable way to force the server to decide early -- the user-agent can only ask nicely for a 100-continue / early 2xx/4xx - and the user-agent has no reliable way to determine if the server does support Expect: 100-continue What the user-agent wants is for the server to send any 3xx/401 responses early, else commit to not-401 by sending 100-continue. The server can still defer 2xx/403 responses, though preferably not 403s, until it sees some/all of the request body. The only fix I can think of is to REQUIRE that OPTIONS responses indicate whether the server supports this or not. An HTTP/2 or 3 gateway to an HTTP/1.1 server (or vice versa) could be configured to indicate support / no support for Expect: 100-continue. In particular, there is no way for a new version of HTTP to give a user-agent a way to force the server to support early decisions. The servers exist already, and the are as they are.
- treve 8y ago
  > What I am a bit unsure about is actually how things like "transfer-encoding: chunked" or the various "connection" headers are supposed to be handled in servers/clients/libraries that are supposed to handle HTTP/1.1 and HTTP/2 As you suspected these headers are 'special' and don't exist in HTTP/2.
- cryptonector 8y ago
  > What I am a bit unsure about is actually how things like "transfer-encoding: chunked" or the various "connection" headers are supposed to be handled in servers/clients/libraries that are supposed to handle HTTP/1.1 and HTTP/2. Is this supposed to be automatically set on the lower level, depending on the actual protocol type. And the application level HTTP client or handler on server just sees the body as a stream? I'm not sure if that happens currently in common libraries, or whether they would forward these things if a client is connected via HTTP/1.1 and not when it's connected via HTTP/2. Section 8.1.2.2 of RFC 7540 [0] covers most of what you ask except that it doesn't explain how chunked transfer-encoding maps to HTTP/2, and section 8.1.3 [1] shows an example of mapping chunked transfer-encoding onto HTTP/2 data frames. Basically, in HTTP/2 all transfers are necessarily "chunked" into DATA frames [2] (necessarily because of multiplexing of channels!), and Content-Length -when present- merely indicates what all the chunks must add up to. [0] https://tools.ietf.org/html/rfc7540#section-8.1.2.2 [1] https://tools.ietf.org/html/rfc7540#section-8.1.3 [2] https://tools.ietf.org/html/rfc7540#section-6.1
jayd16 8y ago
Its annoying but the "check and then attempt" pattern just doesn't cut it. There are going to be unexpected issues so you end up always having to handle exceptions anyway.
- cryptonector 8y ago
  You always have to handle exceptions, but if you can insist on early 3xx/4xx (at least 401) responses, then you can avoid caching any part of the request body. For example, consider something like: $ huge_fast_source | curl --negotiate -x POST ... Here curl might respond to 401s by authenticating to the server, and if it can know that a 401 is coming, then it can simply not read from stdin (thus flow-controlling the huge, fast source) until it has seen 100-continue, which might require a second request. This is very nice. The fact that it can't be had reliably is rather annoying.
  - jayd16 8y ago
    I see your hypothetical but I can't its something I've missed. If reads are not repeatable or can't be lost then you really should caching them. If you can reset your stream, then just start the firehose. Presumably things like a 401 are rare so the waste is minimal. I could see a use case for this when network IO is prohibitively expensive and needs to be minimized. You're right that there isn't a great solution.

Matthias247 8y ago

Is that really a HTTP/[2|3] problem, or a problem of those servers? As far as I understood the HTTP/2 spec (have read this, but not HTTP/3), it is allowed for servers to send an arbitrary amount of informational headers (incl. 100-continue) before sending the actual headers and response body. So everything to handle these kinds of requests should be in place. If the server ignores the "Expect: 100-continue" there is obviously nothing the other side can do, but that's the same in HTTP/1.1. What I am a bit unsure about is actually how things like "transfer-encoding: chunked" or the various "connection" headers are supposed to be handled in servers/clients/libraries that are supposed to handle HTTP/1.1 and HTTP/2. Is this supposed to be automatically set on the lower level, depending on the actual protocol type. And the application level HTTP client or handler on server just sees the body as a stream? I'm not sure if that happens currently in common libraries, or whether they would forward these things if a client is connected via HTTP/1.1 and not when it's connected via HTTP/2.

cryptonector 8y ago
The real is that the server is allowed to defer decisions about authentication and authorization until it has seen some or all of the request body, therefore the server might not respond until it has seen part of the body, but the user-agent wants to know whether it can get an immediate response to a POST/PUT in order to avoid sending the body until it knows the server is ready to sink it, and the user-agent is resource constrained and cannot cache the request body, say: - the user-agent wants an early 2xx/4xx from the server - the server is allowed to defer - the user-agent has no reliable way to force the server to decide early -- the user-agent can only ask nicely for a 100-continue / early 2xx/4xx - and the user-agent has no reliable way to determine if the server does support Expect: 100-continue What the user-agent wants is for the server to send any 3xx/401 responses early, else commit to not-401 by sending 100-continue. The server can still defer 2xx/403 responses, though preferably not 403s, until it sees some/all of the request body. The only fix I can think of is to REQUIRE that OPTIONS responses indicate whether the server supports this or not. An HTTP/2 or 3 gateway to an HTTP/1.1 server (or vice versa) could be configured to indicate support / no support for Expect: 100-continue. In particular, there is no way for a new version of HTTP to give a user-agent a way to force the server to support early decisions. The servers exist already, and the are as they are.
treve 8y ago
> What I am a bit unsure about is actually how things like "transfer-encoding: chunked" or the various "connection" headers are supposed to be handled in servers/clients/libraries that are supposed to handle HTTP/1.1 and HTTP/2 As you suspected these headers are 'special' and don't exist in HTTP/2.
cryptonector 8y ago
> What I am a bit unsure about is actually how things like "transfer-encoding: chunked" or the various "connection" headers are supposed to be handled in servers/clients/libraries that are supposed to handle HTTP/1.1 and HTTP/2. Is this supposed to be automatically set on the lower level, depending on the actual protocol type. And the application level HTTP client or handler on server just sees the body as a stream? I'm not sure if that happens currently in common libraries, or whether they would forward these things if a client is connected via HTTP/1.1 and not when it's connected via HTTP/2. Section 8.1.2.2 of RFC 7540 [0] covers most of what you ask except that it doesn't explain how chunked transfer-encoding maps to HTTP/2, and section 8.1.3 [1] shows an example of mapping chunked transfer-encoding onto HTTP/2 data frames. Basically, in HTTP/2 all transfers are necessarily "chunked" into DATA frames [2] (necessarily because of multiplexing of channels!), and Content-Length -when present- merely indicates what all the chunks must add up to. [0] https://tools.ietf.org/html/rfc7540#section-8.1.2.2 [1] https://tools.ietf.org/html/rfc7540#section-8.1.3 [2] https://tools.ietf.org/html/rfc7540#section-6.1

jayd16 8y ago

Its annoying but the "check and then attempt" pattern just doesn't cut it. There are going to be unexpected issues so you end up always having to handle exceptions anyway.

cryptonector 8y ago
You always have to handle exceptions, but if you can insist on early 3xx/4xx (at least 401) responses, then you can avoid caching any part of the request body. For example, consider something like: $ huge_fast_source | curl --negotiate -x POST ... Here curl might respond to 401s by authenticating to the server, and if it can know that a 401 is coming, then it can simply not read from stdin (thus flow-controlling the huge, fast source) until it has seen 100-continue, which might require a second request. This is very nice. The fact that it can't be had reliably is rather annoying.
- jayd16 8y ago
  I see your hypothetical but I can't its something I've missed. If reads are not repeatable or can't be lost then you really should caching them. If you can reset your stream, then just start the firehose. Presumably things like a 401 are rare so the waste is minimal. I could see a use case for this when network IO is prohibitively expensive and needs to be minimized. You're right that there isn't a great solution.