8 ms·

It should be noted that this implementation has been in production since last Summer and is powering the Firewall Rules ( https://blog.cloudflare.com/how-we-mad

by buro9 8y ago

It should be noted that this implementation has been in production since last Summer and is powering the Firewall Rules ( https://blog.cloudflare.com/how-we-made-firewall-rules/ https://blog.cloudflare.com/how-we-made-firewall-rules/ ) engine today. This was one of the first major components to use Rust at our edge and we've been really happy with it. The flexibility this approach (to matching traffic) has given us... whilst we also get the speed, memory safety... is just great. The speed at which we're iterating on the firewall and other systems that use this is a joy to behold. A lot of that speed derives from the confidence in this component. We're also really pleased that whilst we have several proposals for optimisations, none have yet needed to be prioritised as the performance is great.

jimmy1 8y ago
> . The speed at which we're iterating on the firewall and other systems that use this is a joy to behold. How much of this do you attribute directly to Rust versus other factors like seniority of engineering talent, clear communicated requirements, organizational empowerment, etc? If you swapped out Rust for C, what would be the impact on velocity? Is it a 1-4x multiplier, or larger?
- buro9 8y ago
  Engineering resource is finite, the less time spent on working on the engine the more time we can spend working on new features and other systems. Before making this in Rust we experimented with a Go and also a Lua implementation as we investigated the approach. The Rust code took the same time to initially produce and was production ready very early and has been rock solid and required virtually no maintenance since it was put into production. That frees up that engineer to work on other things, whilst also reducing ongoing maintenance that was anticipated and so those engineers are also working on other stuff. I'd attribute a chunk of that to Rust... though it helps if you have great engineers who are familiar with the concepts of parsers, etc working on these things. The same may not be true if someone new to Rust was also new to the concepts needed.
  - ryanworl 8y ago
    What were the differences between the Go and Rust versions that led you to choose the Rust version in the end?
    - buro9 8y ago
      Given that I wrote the Go version and RReverser wrote the Rust version, I'm going to go with: A better engineer wrote the Rust version. But actually I was excited by Rust too. The memory safety, performance (the Rust was faster) and the degree of control over how we could present the FFI to the languages we needed to integrate with, in addition to how readable the Rust was by comparison to the Go code (readable, but so much of it), and then avoiding the GC... the Rust implementation was a convincing winner.
      - int_19h 8y ago
        In terms of readability, how much is it due to Rust having ADTs and pattern matching over them?
        gavia1 8y ago
        You can kind of "fake" ADTs in Go with interface{} and switches, but it isn't all that nice. Rust on the other hand with first-class ADT and pattern matching makes it a pleasure! You also have all the nice advantages of the compiler checking that all branches have been satisfied which catches a lot of bugs at compile time. Go has it's place, and works great in those places, but I don't think this is one of them. I'm pleased we chose Rust and this feels like a great use for it!
    - deleted 8y ago
      [deleted]
  - foldr 8y ago
    It presumably also helps that the Rust version is the third iteration of the code (given that you mention earlier implementations in Go and Lua).
DyslexicAtheist 8y ago
was this project inspired by the Usenix ;login: article Safe Parsers in Rust Changing the World Step by Step ? it seems the philosophy behind it is very similar https://www.usenix.org/publications/login/fall2017 https://www.usenix.org/publications/login/fall2017
- RReverser 8y ago
  Never heard of it, but will check out, thanks.
  - DyslexicAtheist 8y ago
    after re-reading the article I think the similarity is nom. It probably ends there since the usenix thing is quite old in comparison.
    - RReverser 8y ago
      After looking into it, I think I actually did come across it long time ago. As you've noted, the approaches are different, but it's indeed inspiring in the sense of moving to safer languages for parsing in general.

by buro9 8y ago

jimmy1 8y ago

> . The speed at which we're iterating on the firewall and other systems that use this is a joy to behold. How much of this do you attribute directly to Rust versus other factors like seniority of engineering talent, clear communicated requirements, organizational empowerment, etc? If you swapped out Rust for C, what would be the impact on velocity? Is it a 1-4x multiplier, or larger?

buro9 8y ago
Engineering resource is finite, the less time spent on working on the engine the more time we can spend working on new features and other systems. Before making this in Rust we experimented with a Go and also a Lua implementation as we investigated the approach. The Rust code took the same time to initially produce and was production ready very early and has been rock solid and required virtually no maintenance since it was put into production. That frees up that engineer to work on other things, whilst also reducing ongoing maintenance that was anticipated and so those engineers are also working on other stuff. I'd attribute a chunk of that to Rust... though it helps if you have great engineers who are familiar with the concepts of parsers, etc working on these things. The same may not be true if someone new to Rust was also new to the concepts needed.
- ryanworl 8y ago
  What were the differences between the Go and Rust versions that led you to choose the Rust version in the end?
  - buro9 8y ago
    Given that I wrote the Go version and RReverser wrote the Rust version, I'm going to go with: A better engineer wrote the Rust version. But actually I was excited by Rust too. The memory safety, performance (the Rust was faster) and the degree of control over how we could present the FFI to the languages we needed to integrate with, in addition to how readable the Rust was by comparison to the Go code (readable, but so much of it), and then avoiding the GC... the Rust implementation was a convincing winner.
    - int_19h 8y ago
      In terms of readability, how much is it due to Rust having ADTs and pattern matching over them?
      - gavia1 8y ago
        You can kind of "fake" ADTs in Go with interface{} and switches, but it isn't all that nice. Rust on the other hand with first-class ADT and pattern matching makes it a pleasure! You also have all the nice advantages of the compiler checking that all branches have been satisfied which catches a lot of bugs at compile time. Go has it's place, and works great in those places, but I don't think this is one of them. I'm pleased we chose Rust and this feels like a great use for it!
  - deleted 8y ago
    [deleted]
- foldr 8y ago
  It presumably also helps that the Rust version is the third iteration of the code (given that you mention earlier implementations in Go and Lua).

DyslexicAtheist 8y ago

was this project inspired by the Usenix ;login: article Safe Parsers in Rust Changing the World Step by Step ? it seems the philosophy behind it is very similar https://www.usenix.org/publications/login/fall2017 https://www.usenix.org/publications/login/fall2017

RReverser 8y ago
Never heard of it, but will check out, thanks.
- DyslexicAtheist 8y ago
  after re-reading the article I think the similarity is nom. It probably ends there since the usenix thing is quite old in comparison.
  - RReverser 8y ago
    After looking into it, I think I actually did come across it long time ago. As you've noted, the approaches are different, but it's indeed inspiring in the sense of moving to safer languages for parsing in general.