6 ms·

And this is why I expect HTTP/2 and HTTP/3 to be much more robust in the long term: the implementations are harder to write, and you won’t get anywhere without

And this is why I expect HTTP/2 and HTTP/3 to be much more robust in the long term: the implementations are harder to write, and you won’t get anywhere without reading at least a some spec, whereas HTTP/1 is deceptively simple with therefore a lot of badly incorrect implementations, often with corresponding security problems.

arjvik 4y ago
As someone who has not read the HTTP/1.1 spec, what are some pitfalls that could actually become security issues?
- pwdisswordfish9 4y ago
  Being text-based. Which leads to people constructing protocol messages by printf and therefore tons of injection bugs.
- chrismorgan 4y ago
  The most common proximate cause of security issues in format handling (parsing and emitting) comes from implementations differing in their parsing, or implementations emitting invalid values in a way that will be parsed differently. Probably the most common type of security issue then comes from smuggling values through, bypassing checks or triggering injection. (This is the essence of injection attacks as a broad class.) One of the easiest demonstrations of this in HTTP specifically is called HTTP request smuggling: https://portswigger.net/web-security/request-smuggling https://portswigger.net/web-security/request-smuggling. And the solution for that is pretty much: “stop using a text protocol, they’re too hard to use correctly”.
- jefftk 4y ago
  One of the simplest issues is that headers end with a newline. Most code will not generate a header with an embedded new line, so it's common that software doesn't handle this case, and passes the new line through unmodified. This means that if someone is able to set a custom value for part of a header they can often use that to inject their own custom response header. Or even their own customer response body, since that is also set off with newlines.
superkuh 4y ago
HTTP/3 is written for the use case of large corporations and does not even allow for human persons to use it alone. It requires CA based TLS to set up a connection. So if you want to host a website visitable by a random person you've never communicated with before you have to get continued permission from an incorporated entity running a CA to do so. This is far more of a security problem than all of the bad HTTP 1.1 implementations put together. It is built in corporate control that cannot be bypassed except by not using HTTP/3. It is extremely important that we not let the mega-corp browsers drop HTTP 1.1 and continue to write our own projects for it.
- tambre 4y ago
  Is there anything the spec that actually requires that? AFAIK it's just that major implementators (browsers) have chosen to enforce TLS.
  - superkuh 4y ago
    The language in the RFC that Google and Microsoft forced through the IETF to open-wash it uses "MUST" in capital letters when talking about setting up the HTTP3 endpoint and verifying the cert. https://datatracker.ietf.org/doc/rfc9114/ https://datatracker.ietf.org/doc/rfc9114/ I would be extremely relieved if I am wrong and someone could explain how I am wrong. Like... maybe there's some mechanism to self-sign without CA and use a null cypher? So even if most users would be scared away geeks could click through (like today's status quo with self-signed ssl certs).
    - jefftk 4y ago
      You can still use a self-signed cert with HTTP3 (including rightly scary warnings for visitors) or you can make your own CA and distribute the cert (no scary warnings when people visit your site).
      - mistrial9 4y ago
        I will believe this when I see it, thank you the zealous "you must obey the law" tone of SOME comments here reinforces the worst stereotypes of corporate apparats.. individuals doing the bidding of institutions based on the letter of their "laws" Human history has shown again and again that this ends badly .. HTTP is OK with ME
mgaunard 4y ago
HTTP/2 makes no sense at all. HTTP/3 is just a fix to HTTP/2 so that it makes some sort of sense. Both of these are only concerned with reducing the latency of doing lots of requests to the same server in parallel. Which is only needed by web browsers and nothing else.
chestervonwinch 4y ago
I get what you're saying, but robustness through complexity feels like an odd argument nonetheless.
- chrismorgan 4y ago
  Its counterintuitivity is why I like bringing it up. :-)
gumby 4y ago
But for back compatibility implementors will still have to support HTTP/1, which will likely take more than 50% of the total effort.
dbttdft 4y ago
I don't think I could implement a correct HTTP 1 agent even if I read the specs.
bmitc 4y ago
> whereas HTTP/1 is deceptively simple with therefore a lot of badly incorrect implementations Doesn't that imply that HTTP/1 is deceptively complex?
- ameliaquining 4y ago
  I think the idea is that HTTP/1 is simple in the hello-world 5th-percentile-complexity case, which deceives people into thinking that it's also simple in the real-world 99.9th-percentile-complexity case, which it's not at all.
  - Joker_vD 4y ago
    It's, like, simple in about 80-percentile-complexity case. But the rest 20% take 80% of the work (and re-architecturing). For example, 1xx responses break 1-1 correspondence between requests and responses. Then an "Upgrade" header may mean you need to turn a connection into a dumb byte pipe, ditto for "CONNECT" requests. Then there is a whole business of end-to-end vs. by-hop headers: some of the latter ones will be listed in the "Connection" header (did you know that that is its original purpose, and "close" option is but a hack?) but some of the headers are always hop-by-hop and the proxy is expected to filter them even if they're not listed in "Connection" header — but of course the comprehensive list of such by-hop headers doesn't exist. Then there is pipelining. And handling HTTP/1.0 clients (yep, one of the reasons why OP has "IT'S SET TO HTTP/1.1 AND NOTHING ELSE" in his article) who by their nature cannot support replies in "chunked" transfer-encoding. And handling POST bodies in "chunked" transfer-encoding. And handling trailers if you did not cut "Expect: trailers" in the client's request. And there may be comments in "chunked" encoding. And... multiline headers?.. The list goes on and on. And a decent HTTP-proxy must handle all of that stuff or at least fail gracefully without affecting other clients.

arjvik 4y ago

As someone who has not read the HTTP/1.1 spec, what are some pitfalls that could actually become security issues?

pwdisswordfish9 4y ago
Being text-based. Which leads to people constructing protocol messages by printf and therefore tons of injection bugs.
chrismorgan 4y ago
The most common proximate cause of security issues in format handling (parsing and emitting) comes from implementations differing in their parsing, or implementations emitting invalid values in a way that will be parsed differently. Probably the most common type of security issue then comes from smuggling values through, bypassing checks or triggering injection. (This is the essence of injection attacks as a broad class.) One of the easiest demonstrations of this in HTTP specifically is called HTTP request smuggling: https://portswigger.net/web-security/request-smuggling https://portswigger.net/web-security/request-smuggling. And the solution for that is pretty much: “stop using a text protocol, they’re too hard to use correctly”.
jefftk 4y ago
One of the simplest issues is that headers end with a newline. Most code will not generate a header with an embedded new line, so it's common that software doesn't handle this case, and passes the new line through unmodified. This means that if someone is able to set a custom value for part of a header they can often use that to inject their own custom response header. Or even their own customer response body, since that is also set off with newlines.

superkuh 4y ago

HTTP/3 is written for the use case of large corporations and does not even allow for human persons to use it alone. It requires CA based TLS to set up a connection. So if you want to host a website visitable by a random person you've never communicated with before you have to get continued permission from an incorporated entity running a CA to do so. This is far more of a security problem than all of the bad HTTP 1.1 implementations put together. It is built in corporate control that cannot be bypassed except by not using HTTP/3. It is extremely important that we not let the mega-corp browsers drop HTTP 1.1 and continue to write our own projects for it.

tambre 4y ago
Is there anything the spec that actually requires that? AFAIK it's just that major implementators (browsers) have chosen to enforce TLS.
- superkuh 4y ago
  The language in the RFC that Google and Microsoft forced through the IETF to open-wash it uses "MUST" in capital letters when talking about setting up the HTTP3 endpoint and verifying the cert. https://datatracker.ietf.org/doc/rfc9114/ https://datatracker.ietf.org/doc/rfc9114/ I would be extremely relieved if I am wrong and someone could explain how I am wrong. Like... maybe there's some mechanism to self-sign without CA and use a null cypher? So even if most users would be scared away geeks could click through (like today's status quo with self-signed ssl certs).
  - jefftk 4y ago
    You can still use a self-signed cert with HTTP3 (including rightly scary warnings for visitors) or you can make your own CA and distribute the cert (no scary warnings when people visit your site).
    - mistrial9 4y ago
      I will believe this when I see it, thank you the zealous "you must obey the law" tone of SOME comments here reinforces the worst stereotypes of corporate apparats.. individuals doing the bidding of institutions based on the letter of their "laws" Human history has shown again and again that this ends badly .. HTTP is OK with ME

mgaunard 4y ago

HTTP/2 makes no sense at all. HTTP/3 is just a fix to HTTP/2 so that it makes some sort of sense. Both of these are only concerned with reducing the latency of doing lots of requests to the same server in parallel. Which is only needed by web browsers and nothing else.

chestervonwinch 4y ago

I get what you're saying, but robustness through complexity feels like an odd argument nonetheless.

chrismorgan 4y ago
Its counterintuitivity is why I like bringing it up. :-)

gumby 4y ago

But for back compatibility implementors will still have to support HTTP/1, which will likely take more than 50% of the total effort.

dbttdft 4y ago

I don't think I could implement a correct HTTP 1 agent even if I read the specs.

bmitc 4y ago

> whereas HTTP/1 is deceptively simple with therefore a lot of badly incorrect implementations Doesn't that imply that HTTP/1 is deceptively complex?

ameliaquining 4y ago
I think the idea is that HTTP/1 is simple in the hello-world 5th-percentile-complexity case, which deceives people into thinking that it's also simple in the real-world 99.9th-percentile-complexity case, which it's not at all.
- Joker_vD 4y ago
  It's, like, simple in about 80-percentile-complexity case. But the rest 20% take 80% of the work (and re-architecturing). For example, 1xx responses break 1-1 correspondence between requests and responses. Then an "Upgrade" header may mean you need to turn a connection into a dumb byte pipe, ditto for "CONNECT" requests. Then there is a whole business of end-to-end vs. by-hop headers: some of the latter ones will be listed in the "Connection" header (did you know that that is its original purpose, and "close" option is but a hack?) but some of the headers are always hop-by-hop and the proxy is expected to filter them even if they're not listed in "Connection" header — but of course the comprehensive list of such by-hop headers doesn't exist. Then there is pipelining. And handling HTTP/1.0 clients (yep, one of the reasons why OP has "IT'S SET TO HTTP/1.1 AND NOTHING ELSE" in his article) who by their nature cannot support replies in "chunked" transfer-encoding. And handling POST bodies in "chunked" transfer-encoding. And handling trailers if you did not cut "Expect: trailers" in the client's request. And there may be comments in "chunked" encoding. And... multiline headers?.. The list goes on and on. And a decent HTTP-proxy must handle all of that stuff or at least fail gracefully without affecting other clients.