4 ms·

Honest question: would Ragel be more robust if it were written in Rust and to generate Rust code?

by devy 10y ago

pjmlp 10y ago
Logical errors cannot be avoided regardless of the language, even theorem provers might have incorrect axioms. However memory corruption issues, like in this case (out of bounds corruption), are easily avoidable in any memory safe language.
aidenn0 10y ago
The language Ragel is written in is fairly inconsequential. Not because memory errors don't happen in compilers, but because compilers typically don't get malicious inputs. Generating Rust would be useful, but Ragel already has an ML backend IIRC.
- steveklabnik 10y ago
  There used to be a Rust backend for ragel, but it was well before 1.0 and it bitrotted and then died, as far as I know.
  - erickt 10y ago
    I wrote it [0]. I meant to get it merged upstream, but the author took the repository offline to help build his business, so my fork bitrotted. However, it looks like Rust support may have independently landed upstream [2]? [0]: https://github.com/erickt/ https://github.com/erickt/ [1]: http://www.colm.net/news/2014/10/24/ragel-now-maintained-by-colm-networks.html http://www.colm.net/news/2014/10/24/ragel-now-maintained-by-... [2]: http://www.colm.net/news/2015/09/26/first-dev-version-of-ragel-7-released.html http://www.colm.net/news/2015/09/26/first-dev-version-of-rag...
JoachimSchipper 10y ago
Not really. Safe code generating unsafe-code-with-bugs gets you a vulnerable program, so rewriting Ragel in Rust doesn't help. As to generating Rust - Rust isn't magic, and Rust-that-is-as-fast-as-C can't do a lot more work than C. In particular, you don't want to do memory allocation in the fast path of a network server written for performance (note that Rust uses jemalloc, which is a C memory allocation library; Rust memory allocation isn't going to beat C memory allocation.) As such, a realistic Rust program probably needs to re-use network buffers, at which point language-level security doesn't stop you from sending out "old" data that was still in the buffer. [EDIT: but see the reply https://news.ycombinator.com/item?id=13804831 https://news.ycombinator.com/item?id=13804831 below.] A debug build in Rust could allocate-per-request and detect the problem; but allocate-per-request in C is also likely to detect such overflows (indeed, the CloudFlare server did occasionally crash on this bug), and C programmers can also use e.g. VALGRIND_MAKE_MEM_UNDEFINED() for effort comparable to maintaining an alternate allocate-per-request implementation in Rust. The difference for the fast path - where Rust can't do a lot more work, and where you can focus your effort - just isn't very large. (Using a safe language for e.g. low-performance control-plane interfaces can help, however - but at that point, you may well be arguing e.g. Rust vs. Lua.)
- Manishearth 10y ago
  Rust's version of Ragel actually generates 100% safe Rust code, AFAICT without any additional cost. http://github.com/erickt/stateful http://github.com/erickt/stateful
  - JoachimSchipper 10y ago
    Are you sure? CloudFlare's problem seems to have been running off the end of a logical buffer embedded in a linked list of "physical" buffers. (Sorry, have to run - will try to take a better look later tonight.)
    - Manishearth 10y ago
      In Rust you'd usually use a slice for this anyway, which has bounds checks. Yes, you could use indices instead, but that's not very Rusty (and more annoying to use!), whereas using pointer arithmetic is par for C. Edit: Yes, you could ultimately get it wrong in Rust, too, I just think it's much less likely.
      - JoachimSchipper 10y ago
        So you're thinking of an iterator of slices of char (arrays), right? It still somewhat tricky to get right to me; the writer wants to have access to the underlying physical buffers, so you need a conversion step from list-of-physical-buffers without allocating, etc. But you're probably not wrong that a Rust programmer is more likely to get this right; thanks! (And it's all likelihoods anyway; there's plenty of C code that doesn't get this wrong, and e.g. the VALGRIND_MAKE_MEM_UNDEFINED() I suggested above does reliably - albeit dynamically - find this problem in C.) (Added a reference from my top comment to your reply.)
        Manishearth 10y ago
        You don't need to allocate, Rust slices are safe zero-copy views into an existing buffer. Yes, you can run under valgrind, but dynamic checks ... :) You need to have testcases that hits those code paths, and it's really hard to make "unexpected" testcases that test code paths like these. That's why fuzzing is awesome!
astrodust 10y ago
Ragel itself is not the attack vector here, it's the code that Ragel generates. That's a by-product of the grammar and code you write with it. It's like blaming JavaScript for a bug in a JavaScript interpreter. It's not the fault of the language specification, but the implementation. Ragel's emitted code is partially the product of Ragel, which is working code, and the user-supplied code as part of the Ragel program, which is an unknown quantity. Apparently the side-effect of one of Ragel's directives wasn't fully understood and this wasn't exposed in testing. It sounds like Cloudflare needs to fuzz a lot more aggressively to shake out issues like this before they hit production.
- kibwen 10y ago
  Ragel itself may not be at fault, but it is the case that if it were generating code in a memory-safe language rather than C (including Rust code sans the `unsafe` keyword), then improper use of Ragel wouldn't be capable of dumping random memory contents, which is what cloudbleed exhibited.
  - astrodust 10y ago
    Ragel can emit lots of different kinds of code, Ruby included, so to blame the tool here is mistaken. It's blaming how the tool was configured and used. Emitting Rust might not be a bad idea, but I bet this code-base pre-dates Rust being a viable target.
    - sitkack 10y ago
      When the option arises, it is nice to run the test vectors against the diversity of generated output, C, Ruby, Etc. One of those backends could have very well alerted on the error.
      - astrodust 10y ago
        Although Ragel can emit a variety of code, it's non-trivial to rework your own code that's a part of that from C to Ruby or Rust or whatever. You'd basically have to implement the same thing N times.
pornel 10y ago
If it were exact 1:1 equivalent, generating Rust code operating on a raw buffer with no bounds checks, then no. However, existing parsers in Rust take advantage of its macros and type system to provide more guarantees at compile time, so perhaps a different approach would not allow the logic error that lead to the bug.

Honest question: would Ragel be more robust if it were written in Rust and to generate Rust code?

by devy 10y ago

Honest question: would Ragel be more robust if it were written in Rust and to generate Rust code?

pjmlp 10y ago

Logical errors cannot be avoided regardless of the language, even theorem provers might have incorrect axioms. However memory corruption issues, like in this case (out of bounds corruption), are easily avoidable in any memory safe language.

aidenn0 10y ago

The language Ragel is written in is fairly inconsequential. Not because memory errors don't happen in compilers, but because compilers typically don't get malicious inputs. Generating Rust would be useful, but Ragel already has an ML backend IIRC.

steveklabnik 10y ago
There used to be a Rust backend for ragel, but it was well before 1.0 and it bitrotted and then died, as far as I know.
- erickt 10y ago
  I wrote it [0]. I meant to get it merged upstream, but the author took the repository offline to help build his business, so my fork bitrotted. However, it looks like Rust support may have independently landed upstream [2]? [0]: https://github.com/erickt/ https://github.com/erickt/ [1]: http://www.colm.net/news/2014/10/24/ragel-now-maintained-by-colm-networks.html http://www.colm.net/news/2014/10/24/ragel-now-maintained-by-... [2]: http://www.colm.net/news/2015/09/26/first-dev-version-of-ragel-7-released.html http://www.colm.net/news/2015/09/26/first-dev-version-of-rag...

JoachimSchipper 10y ago

Not really. Safe code generating unsafe-code-with-bugs gets you a vulnerable program, so rewriting Ragel in Rust doesn't help. As to generating Rust - Rust isn't magic, and Rust-that-is-as-fast-as-C can't do a lot more work than C. In particular, you don't want to do memory allocation in the fast path of a network server written for performance (note that Rust uses jemalloc, which is a C memory allocation library; Rust memory allocation isn't going to beat C memory allocation.) As such, a realistic Rust program probably needs to re-use network buffers, at which point language-level security doesn't stop you from sending out "old" data that was still in the buffer. [EDIT: but see the reply https://news.ycombinator.com/item?id=13804831 https://news.ycombinator.com/item?id=13804831 below.] A debug build in Rust could allocate-per-request and detect the problem; but allocate-per-request in C is also likely to detect such overflows (indeed, the CloudFlare server did occasionally crash on this bug), and C programmers can also use e.g. VALGRIND_MAKE_MEM_UNDEFINED() for effort comparable to maintaining an alternate allocate-per-request implementation in Rust. The difference for the fast path - where Rust can't do a lot more work, and where you can focus your effort - just isn't very large. (Using a safe language for e.g. low-performance control-plane interfaces can help, however - but at that point, you may well be arguing e.g. Rust vs. Lua.)

Manishearth 10y ago
Rust's version of Ragel actually generates 100% safe Rust code, AFAICT without any additional cost. http://github.com/erickt/stateful http://github.com/erickt/stateful
- JoachimSchipper 10y ago
  Are you sure? CloudFlare's problem seems to have been running off the end of a logical buffer embedded in a linked list of "physical" buffers. (Sorry, have to run - will try to take a better look later tonight.)
  - Manishearth 10y ago
    In Rust you'd usually use a slice for this anyway, which has bounds checks. Yes, you could use indices instead, but that's not very Rusty (and more annoying to use!), whereas using pointer arithmetic is par for C. Edit: Yes, you could ultimately get it wrong in Rust, too, I just think it's much less likely.
    - JoachimSchipper 10y ago
      So you're thinking of an iterator of slices of char (arrays), right? It still somewhat tricky to get right to me; the writer wants to have access to the underlying physical buffers, so you need a conversion step from list-of-physical-buffers without allocating, etc. But you're probably not wrong that a Rust programmer is more likely to get this right; thanks! (And it's all likelihoods anyway; there's plenty of C code that doesn't get this wrong, and e.g. the VALGRIND_MAKE_MEM_UNDEFINED() I suggested above does reliably - albeit dynamically - find this problem in C.) (Added a reference from my top comment to your reply.)
      - Manishearth 10y ago
        You don't need to allocate, Rust slices are safe zero-copy views into an existing buffer. Yes, you can run under valgrind, but dynamic checks ... :) You need to have testcases that hits those code paths, and it's really hard to make "unexpected" testcases that test code paths like these. That's why fuzzing is awesome!

astrodust 10y ago

Ragel itself is not the attack vector here, it's the code that Ragel generates. That's a by-product of the grammar and code you write with it. It's like blaming JavaScript for a bug in a JavaScript interpreter. It's not the fault of the language specification, but the implementation. Ragel's emitted code is partially the product of Ragel, which is working code, and the user-supplied code as part of the Ragel program, which is an unknown quantity. Apparently the side-effect of one of Ragel's directives wasn't fully understood and this wasn't exposed in testing. It sounds like Cloudflare needs to fuzz a lot more aggressively to shake out issues like this before they hit production.

kibwen 10y ago
Ragel itself may not be at fault, but it is the case that if it were generating code in a memory-safe language rather than C (including Rust code sans the `unsafe` keyword), then improper use of Ragel wouldn't be capable of dumping random memory contents, which is what cloudbleed exhibited.
- astrodust 10y ago
  Ragel can emit lots of different kinds of code, Ruby included, so to blame the tool here is mistaken. It's blaming how the tool was configured and used. Emitting Rust might not be a bad idea, but I bet this code-base pre-dates Rust being a viable target.
  - sitkack 10y ago
    When the option arises, it is nice to run the test vectors against the diversity of generated output, C, Ruby, Etc. One of those backends could have very well alerted on the error.
    - astrodust 10y ago
      Although Ragel can emit a variety of code, it's non-trivial to rework your own code that's a part of that from C to Ruby or Rust or whatever. You'd basically have to implement the same thing N times.

pornel 10y ago

If it were exact 1:1 equivalent, generating Rust code operating on a raw buffer with no bounds checks, then no. However, existing parsers in Rust take advantage of its macros and type system to provide more guarantees at compile time, so perhaps a different approach would not allow the logic error that lead to the bug.