4 ms·

The best solution would be to make an http version 4 with a non-fragile format, e.g. json. Otherwise we will keep chasing bugs forever.

josephg 5y ago
A lot of the new http bugs aren’t caused by ambiguities in http1 headers, or ambiguities in http2 headers. They happen when an http2 message gets rewritten into http1 and “valid” http header characters (like new lines) show up as header separators in http1. The problem isn’t that we don’t have a good header format. The problem is we have too many.
nmadden 5y ago
Configuring your webserver/reverse proxy to talk HTTP/2 to backend appservers is a good improvement against request smuggling. (If they support it, sadly not guaranteed). The binary format is much less ambiguous. There is a talk by James Kettle about request smuggling with HTTP/2, but it is largely about attacks when the frontend talks HTTP/2 and then converts to HTTP/1.1 to talk to backend servers [1]. That said, it does also highlight some HTTP/2-only quirks, so it’s not completely perfect, but it’s so much better than HTTP/1.1. [1]: https://portswigger.net/kb/papers/rfekn2Uv/HTTP2whitepaper.pdf https://portswigger.net/kb/papers/rfekn2Uv/HTTP2whitepaper.p...
dtech 5y ago
http 2 and 3 have much stricter defined binary formats. JSON would be a step back in terms of spec and performance.
legulere 5y ago
JSON has the problem that different parsers handle multiple occurrences of a object element differently. You need to watch out that header names are only ascii otherwise you could run into string comparisons being different on different platforms.
shakna 5y ago
> a non-fragile format, e.g. json. JSON is a terrible format. Especially for streaming data. Here's a super simple shell script for generating invalid JSON that will blow Python's stack: n="$(python3 -c 'import math; import sys; sys.stdout.write(str(math.floor(sys.getrecursionlimit() - 4)))')" left="$(yes [ | head -n "$n" | tr -d '\n')" echo "$left" | python3 -c 'import json; print(json.loads(input()))' It is invalid JSON. But you cannot tell if it's invalid until either: a) You run out of memory. b) The connection ends. Because JSON is terrible for streaming data.

josephg 5y ago

A lot of the new http bugs aren’t caused by ambiguities in http1 headers, or ambiguities in http2 headers. They happen when an http2 message gets rewritten into http1 and “valid” http header characters (like new lines) show up as header separators in http1. The problem isn’t that we don’t have a good header format. The problem is we have too many.

nmadden 5y ago

Configuring your webserver/reverse proxy to talk HTTP/2 to backend appservers is a good improvement against request smuggling. (If they support it, sadly not guaranteed). The binary format is much less ambiguous. There is a talk by James Kettle about request smuggling with HTTP/2, but it is largely about attacks when the frontend talks HTTP/2 and then converts to HTTP/1.1 to talk to backend servers [1]. That said, it does also highlight some HTTP/2-only quirks, so it’s not completely perfect, but it’s so much better than HTTP/1.1. [1]: https://portswigger.net/kb/papers/rfekn2Uv/HTTP2whitepaper.pdf https://portswigger.net/kb/papers/rfekn2Uv/HTTP2whitepaper.p...

dtech 5y ago

http 2 and 3 have much stricter defined binary formats. JSON would be a step back in terms of spec and performance.

legulere 5y ago

JSON has the problem that different parsers handle multiple occurrences of a object element differently. You need to watch out that header names are only ascii otherwise you could run into string comparisons being different on different platforms.

shakna 5y ago

> a non-fragile format, e.g. json. JSON is a terrible format. Especially for streaming data. Here's a super simple shell script for generating invalid JSON that will blow Python's stack: n="$(python3 -c 'import math; import sys; sys.stdout.write(str(math.floor(sys.getrecursionlimit() - 4)))')" left="$(yes [ | head -n "$n" | tr -d '\n')" echo "$left" | python3 -c 'import json; print(json.loads(input()))' It is invalid JSON. But you cannot tell if it's invalid until either: a) You run out of memory. b) The connection ends. Because JSON is terrible for streaming data.