3 ms·

I'm surprised no one has mentioned Ragel: http://www.colm.net/open-source/ragel/ http://www.colm.net/open-source/ragel/ Ragel generates very fast code, is modu

I'm surprised no one has mentioned Ragel: http://www.colm.net/open-source/ragel/ http://www.colm.net/open-source/ragel/ Ragel generates very fast code, is modular by design and has good error handling. It supports both compiled and scripted languages (e.g. can generate both Ruby and C) which is useful if you need a fallback. I've really enjoyed using it to implement the following: - A template language for Ruby: https://github.com/ioquatix/trenni/blob/master/parsers/trenni/template.rl https://github.com/ioquatix/trenni/blob/master/parsers/trenn... - A SGML parser for Ruby: https://github.com/ioquatix/trenni/blob/master/parsers/trenni/markup.rl https://github.com/ioquatix/trenni/blob/master/parsers/trenn... - A HTTP V1 protocol parser: https://github.com/kurocha/async-http/blob/master/source/Async/HTTP/V1/RFC7230.rl https://github.com/kurocha/async-http/blob/master/source/Asy... - A URI parser: https://github.com/kurocha/uri/blob/master/source/URI/RFC3986.rl https://github.com/kurocha/uri/blob/master/source/URI/RFC398... I'll admit, it does take a while to understand how to correctly handle ambiguity when dealing with callbacks/events during parsing, but generally speaking, once you get a bit of experience with how things work, it becomes a very powerful tool in your toolbox.

allengeorge 9y ago
I thought that Ragel was no longer open-source, and supported C/C++ only (admittedly, the second point isn't that big a deal, given the article and its audience).
- justincormack 9y ago
  No. It changed from GPL to MIT license, but did not go closed source. http://www.colm.net/open-source/ragel/ http://www.colm.net/open-source/ragel/
jwilk 9y ago
Cloudbleed was caused by a bug in a Ragel-based parser.
- rhencke 9y ago
  It was not caused by a bug in Ragel, though.
  - wuch 9y ago
    To provide a little bit more context, here is detailed description of this bug [0]. Curiously, they suggest that problem could be avoided by changing the check for EOF to ">=". This is not true at all in case of C, at least as far as language semantics is concerned. When pointer goes two past the end of array you are already in undefined behaviour land. [0] https://blog.cloudflare.com/incident-report-on-memory-leak-caused-by-cloudflare-parser-bug/ https://blog.cloudflare.com/incident-report-on-memory-leak-c...
    - striking 9y ago
      More specifically: > The Ragel code we wrote contained a bug that caused the pointer to jump over the end of the buffer and past the ability of an equality check to spot the buffer overrun.
    - ioquatix 9y ago
      I think it's valid in C/C++ to go 1 past the end of an array: https://stackoverflow.com/questions/988158/take-the-address-of-a-one-past-the-end-array-element-via-subscript-legal-by-the https://stackoverflow.com/questions/988158/take-the-address-...
  - staticassertion 9y ago
    This is always going to lead to the same place - whether it's the developer's fault or the tool's. The author of this post wants uncompromising safety, ragel does not allow for that - programmer error can introduce memory unsafety. They even explicitly call out Cloudbleed.
  - ioquatix 9y ago
    A simple `assert(p<pe);` in a debug build would have caught the issue. I think you could argue that ragel should include something like this by default.
vram22 9y ago
IIRC, Zed Shaw used Ragel to create a parser for Mongrel or Mongrel2 (the web servers). Update: Checked it, he did: https://en.wikipedia.org/wiki/Mongrel_(web_server) https://en.wikipedia.org/wiki/Mongrel_(web_server) https://en.wikipedia.org/wiki/Mongrel2 https://en.wikipedia.org/wiki/Mongrel2

allengeorge 9y ago

I thought that Ragel was no longer open-source, and supported C/C++ only (admittedly, the second point isn't that big a deal, given the article and its audience).

justincormack 9y ago
No. It changed from GPL to MIT license, but did not go closed source. http://www.colm.net/open-source/ragel/ http://www.colm.net/open-source/ragel/

jwilk 9y ago

Cloudbleed was caused by a bug in a Ragel-based parser.

rhencke 9y ago
It was not caused by a bug in Ragel, though.
- wuch 9y ago
  To provide a little bit more context, here is detailed description of this bug [0]. Curiously, they suggest that problem could be avoided by changing the check for EOF to ">=". This is not true at all in case of C, at least as far as language semantics is concerned. When pointer goes two past the end of array you are already in undefined behaviour land. [0] https://blog.cloudflare.com/incident-report-on-memory-leak-caused-by-cloudflare-parser-bug/ https://blog.cloudflare.com/incident-report-on-memory-leak-c...
  - striking 9y ago
    More specifically: > The Ragel code we wrote contained a bug that caused the pointer to jump over the end of the buffer and past the ability of an equality check to spot the buffer overrun.
  - ioquatix 9y ago
    I think it's valid in C/C++ to go 1 past the end of an array: https://stackoverflow.com/questions/988158/take-the-address-of-a-one-past-the-end-array-element-via-subscript-legal-by-the https://stackoverflow.com/questions/988158/take-the-address-...
- staticassertion 9y ago
  This is always going to lead to the same place - whether it's the developer's fault or the tool's. The author of this post wants uncompromising safety, ragel does not allow for that - programmer error can introduce memory unsafety. They even explicitly call out Cloudbleed.
- ioquatix 9y ago
  A simple `assert(p<pe);` in a debug build would have caught the issue. I think you could argue that ragel should include something like this by default.

vram22 9y ago

IIRC, Zed Shaw used Ragel to create a parser for Mongrel or Mongrel2 (the web servers). Update: Checked it, he did: https://en.wikipedia.org/wiki/Mongrel_(web_server) https://en.wikipedia.org/wiki/Mongrel_(web_server) https://en.wikipedia.org/wiki/Mongrel2 https://en.wikipedia.org/wiki/Mongrel2