3 ms·

One of these days, I'm building an integration test suite of all the weird corner cases in HTTP and whatnot to throw at libraries. God help the poor engineer h

One of these days, I'm building an integration test suite of all the weird corner cases in HTTP and whatnot to throw at libraries. God help the poor engineer here. Let's say he or she (reasonably!) looked up the grammar for a URI in RFC 3986[1], the standard that defines them. Well, according to that grammar, the browser is wrong: backslash isn't valid in that position, and the exploiting string isn't a valid production of the URI grammar. But of course, the browser has its own standard — the WHATWG URL spec — which doesn't even define a grammar for a URL, but instead defines a program in pseudocode to parse them. Normally, \ isn't valid after an authority … unless the scheme is http or https or one of several other Chosen One schemes.[2] Uuugh. We've wrought this upon ourselves. Also included in this test suite would be the wild and wonderful things you can type into a browser's URL bar that result in malformed HTTP requests hitting the wire… AFAIK, WHATWG doesn't yet have their own copy of HTTP… [1]: https://tools.ietf.org/html/rfc3986#section-3 https://tools.ietf.org/html/rfc3986#section-3 [2]: https://url.spec.whatwg.org/#hostname-state https://url.spec.whatwg.org/#hostname-state Maybe one should argue that regex should not have accepted the URI, since it's isn't valid according to the RFC.

tal8d 7y ago
I wrote 90% of a strict URI parser over a year ago, I know that RFC way better than I'd like to. I stopped at 90% after discovering the reason why my fuzzer was generating test failures: the ABNF is wrong. I believe it was around the query portion of URI segmentation, where two potential interpretations shared the same order of precedence and there was no rule on the order of execution. I'll finish that project one day... because I still don't know how scared I should be about that sort of ambiguity making it through more than one ABNF parser.
chrismorgan 7y ago
Grammars are overrated—they regularly leave ambiguities and sometimes conflict with the text, despite both being considered normative. I love WHATWG specs with their state machines. They’re all about defining exact behaviour based upon what has already been implemented by extant software, and define comprehensively how every possible input should be parsed. They’re easy to transcribe in code without error. These things get you compatibility, and avoid security bugs where people have implemented different behaviour in different places—as was the case in this instance. As a parser spec, the HTML spec is my favourite, because it makes implementing something that is fiendishly complex genuinely easy. I would love something equivalent for HTTP/1 and MIME messages.
richdougherty 7y ago
As someone who has also worked on various bits of HTTP and URI parsing I have also wished for such a test suite - or even that these RFCs and specs would come bundled with such a suite. There are a lot of weird cases to consider. For example, how many tools correctly handle character encoding in HTTP headers? https://tools.ietf.org/html/rfc8187#section-3.2.3 https://tools.ietf.org/html/rfc8187#section-3.2.3
- tal8d 7y ago
  I attacked it from both directions: an arguably supervised dataset of urls (hundreds of GB from various internet survey projects), and unit testing the insane markov gibberish the parser makes when you cram random data where it doesn't belong and run it in reverse. The dataset was way less helpful for a strictly conforming design, but I imagine it would be useful for a mostly correct parser.
dnpp123 7y ago
You could say pretty much the same for TCP (or even maybe lower down the network stack)

tal8d 7y ago

I wrote 90% of a strict URI parser over a year ago, I know that RFC way better than I'd like to. I stopped at 90% after discovering the reason why my fuzzer was generating test failures: the ABNF is wrong. I believe it was around the query portion of URI segmentation, where two potential interpretations shared the same order of precedence and there was no rule on the order of execution. I'll finish that project one day... because I still don't know how scared I should be about that sort of ambiguity making it through more than one ABNF parser.

chrismorgan 7y ago

Grammars are overrated—they regularly leave ambiguities and sometimes conflict with the text, despite both being considered normative. I love WHATWG specs with their state machines. They’re all about defining exact behaviour based upon what has already been implemented by extant software, and define comprehensively how every possible input should be parsed. They’re easy to transcribe in code without error. These things get you compatibility, and avoid security bugs where people have implemented different behaviour in different places—as was the case in this instance. As a parser spec, the HTML spec is my favourite, because it makes implementing something that is fiendishly complex genuinely easy. I would love something equivalent for HTTP/1 and MIME messages.

richdougherty 7y ago

As someone who has also worked on various bits of HTTP and URI parsing I have also wished for such a test suite - or even that these RFCs and specs would come bundled with such a suite. There are a lot of weird cases to consider. For example, how many tools correctly handle character encoding in HTTP headers? https://tools.ietf.org/html/rfc8187#section-3.2.3 https://tools.ietf.org/html/rfc8187#section-3.2.3

tal8d 7y ago
I attacked it from both directions: an arguably supervised dataset of urls (hundreds of GB from various internet survey projects), and unit testing the insane markov gibberish the parser makes when you cram random data where it doesn't belong and run it in reverse. The dataset was way less helpful for a strictly conforming design, but I imagine it would be useful for a mostly correct parser.

dnpp123 7y ago

You could say pretty much the same for TCP (or even maybe lower down the network stack)