6 ms·

> . The speed at which we're iterating on the firewall and other systems that use this is a joy to behold. How much of this do you attribute directly to Rust v

• buro9 8y ago
  Engineering resource is finite, the less time spent on working on the engine the more time we can spend working on new features and other systems. Before making this in Rust we experimented with a Go and also a Lua implementation as we investigated the approach. The Rust code took the same time to initially produce and was production ready very early and has been rock solid and required virtually no maintenance since it was put into production. That frees up that engineer to work on other things, whilst also reducing ongoing maintenance that was anticipated and so those engineers are also working on other stuff. I'd attribute a chunk of that to Rust... though it helps if you have great engineers who are familiar with the concepts of parsers, etc working on these things. The same may not be true if someone new to Rust was also new to the concepts needed.
  • ryanworl 8y ago
    What were the differences between the Go and Rust versions that led you to choose the Rust version in the end?
    • buro9 8y ago
      Given that I wrote the Go version and RReverser wrote the Rust version, I'm going to go with: A better engineer wrote the Rust version. But actually I was excited by Rust too. The memory safety, performance (the Rust was faster) and the degree of control over how we could present the FFI to the languages we needed to integrate with, in addition to how readable the Rust was by comparison to the Go code (readable, but so much of it), and then avoiding the GC... the Rust implementation was a convincing winner.
      • int_19h 8y ago
        In terms of readability, how much is it due to Rust having ADTs and pattern matching over them?
        • gavia1 8y ago
          You can kind of "fake" ADTs in Go with interface{} and switches, but it isn't all that nice. Rust on the other hand with first-class ADT and pattern matching makes it a pleasure! You also have all the nice advantages of the compiler checking that all branches have been satisfied which catches a lot of bugs at compile time. Go has it's place, and works great in those places, but I don't think this is one of them. I'm pleased we chose Rust and this feels like a great use for it!
    • deleted 8y ago
      [deleted]
  • foldr 8y ago
    It presumably also helps that the Rust version is the third iteration of the code (given that you mention earlier implementations in Go and Lua).
    • RReverser 8y ago
      All of these were written by different people, and languages are quite different as well, so not sure that particular aspect played a role.
  • hathawsh 8y ago
    As I compare Rust and Go, the one thing that keeps me indecisive is that the Rust compiler is slower. Was that ever an issue for you?
    • RReverser 8y ago
      It's not a big issue when you're working on the code continuously, thanks to the incremental compilation, but in general I'd always take slower compilation with more analysis (== lower risk of crashes and faster runtime) over faster compilation any time.
      • Thaxll 8y ago
        Depends, I work on large C++ code base, compile times are really hurting productivity.
        • IWeldMelons 8y ago
          You may try precompiled headers
        • stouset 8y ago
          It's important to keep in mind that there is always going to be a bottleneck on productivity. Sometimes that bottleneck is compilation speed, sometimes it's the language allowing unbounded growth in complexity, sometimes it's ecosystem immaturity, sometimes it's fixing bugs another language would have avoided, and sometimes it's a language's lack of expressivity. But there's always something. It's worth keeping in mind that in some other language or with some other library or using some other toolkit, you'll still wind up with a bottleneck. It'll just be a different one (and maybe a worse one).
        • saghm 8y ago
          Does C++ have a "lower risk of crashes" as GP said though?
        • freedomben 8y ago
          When I worked on a large C++/Qt code base compile times were awful. We ended up setting up distcc and that helped immensely. Worth looking into.
  • stcredzero 8y ago
    Engineering resource is finite Yes. So if one is going to do anything (optimize, write a new server, whatever) one should focus effort where it gets the most bang for the buck. Before making this in Rust we experimented with a Go and also a Lua implementation as we investigated the approach. This is excellent! More companies need to be willing to spike & experiment and change directions on effort up front. In general, the sooner you know, the more money you save.
    • towawayzone 8y ago
      Amen! But I know what my company would say. "Training devs and SRE on a new language is too expensive. And there are too few Rust devs in the job market, and the ones there are are too expensive to hire."
      • meowface 8y ago
        Cloudflare still seems to operate pretty similar to a startup despite now being huge. These days, it seems like a lot more companies that begin as startups are preserving more of that experimental, innovative culture to their later stages. Y Combinator, and others, definitely deserve credit for positively influencing the industry.
        • jgrahamc 8y ago
          A few thoughts: 1. Cloudflare has an unusual structure in that we have a three engineering groups: the core engineering group that builds things that the product management team specify, a totally separate disruption group that works on riskier bets, and a crypto research group. 2. I strongly believe that letting people work in languages they love has a huge advantage. Engineers want to learn new skills (and those that don't we don't want to hire) and so letting them do that means they are happier and do better work. 3. Small teams do more than large ones. The teams that work on Cloudflare products are very small and agile.
      • mikekchar 8y ago
        On the other hand, I have experienced a situation where a fairly young group experienced tech burnout from having to learn too much at once. It's a really tricky balance. I was definitely on the side of "We're programmers! Learning programming languages is like drinking water for us" originally, but now I step back a bit every time we think of introducing new tech. Still, spiking a solution in a couple of different languages seems awesome and I can think of a few projects I've worked on that would definitely have benefited from that.
  • sebcat 8y ago
    > Before making this in Rust we experimented with a Go and also a Lua implementation Lua or LuaJIT?