6 ms·

I've spent the last few months porting the guts of a 100 KLOC PHP command-line utility I wrote to Rust. Thanks to the wonderful Rust documentation it's been a m

• staticassertion 5y ago
  My company has its code basically split in half between Python and Rust. Our Python code uses mypy pretty aggressively. The iteration speed difference is... drastic. Rust far outshines Python in terms of our ability to get something done, maintain it, and iterate on it over time.
  • muglug 5y ago
    That's great for you and your team, but looking at https://github.com/grapl-security/grapl https://github.com/grapl-security/grapl it seems like your needs are pretty different from most web developers.
    • staticassertion 5y ago
      How so? We use many of the crates listed on the linked page - sqlx, rusoto, actix-web, tower/tonic, etc.
      • muglug 5y ago
        Yeah, but the thing you're building is quite a bit more complex than the average consumer-focussed website.
        • staticassertion 5y ago
          I guess, but if you're just building some basic CRUD app that's never going to change, why would iteration speed matter anyway?
          • muglug 5y ago
            It's more a CRUD app that has many features added in the first six months, and fewer features added thereafter, and then a few months go by and a new developer takes over and has to figure out how to add yet another feature. That model favours easy-to-google frameworks with wide ecosystems.
  • fartcannon 5y ago
    Did you mean to write that python beats rust? I find it hard to believe Rust is faster to 'get something done' than Python. Happy to be proven wrong, of course.
    • staticassertion 5y ago
      Nope, it's written correctly as-is. Probably the "time to tests pass" is about equal, but the iteration and maintenance is far better in Rust.
      • melissalobos 5y ago
        > but the iteration... is far better in Rust That is something I would find hard to believe, if you mean iteration of changes. I have coordinated a few rust rewrites, and at least seeing engineers live programming, it takes far far longer to change and compile Rust code than python. Could you talk more about your company's process or how you mean?
        • staticassertion 5y ago
          The time it takes to run `cargo check` is nothing compared to dealing with bugs, terrible tooling, weak errors, etc. All of those areas have been radically better with Rust. As an example, type errors in Rust are radically better than mypy, it's no competition. In general rust's type system is just way better, we have to use Any all over the place because mypy can't even handle a Json type (no recursive types at all). There's nothing that Python really does better in terms of iteration that I can think of. Obviously you don't have to compile code, but `cargo check` is seconds, that is not the bottleneck. We use hypothesis in Python and quickcheck in Rust, cargo is way better than pip, working with loosely typed data is about the same in Rust, maybe a bit better, and working with strictly typed data is wayyyy better. The tooling for working with crates, having an actual lockfile, higher quality libraries, etc. It just all makes Rust more productive. We've run into maybe one rust footgun, which is that binding to _ drops the value immediately. We've run into so many Python footguns, like, did you know that in Python if you want to do circular imports you need to put your imports at the bottom of the file? Or use absolute imports everywhere? Neither of which seem to work 100% of the time by the way, something that makes writing graph data structures quite cumbersome. The other day we ran into a stray comma turning a value into a tuple. The papercuts and footguns across the tooling and language just add up really fast and basically don't exist in Rust. I don't know how people get stuck on the borrow checker, frankly. It's one thing if you're very new to the language, but within weeks it should be very simple to avoid those errors. I maybe hit a "fuck, what" borrow checker issue once a year, in which case I hit up the Discord, they fix it, and I file an issue with rustc so that the error message will get improved. This is based on about 7 years of Rust and Python experience, with 6 years of professional experience in Python and 3 in Rust.
  • bobbylarrybobby 5y ago
    My impression has been that python is easier to iterate on over the course of a week, while Rust is easier to iterate on over the course of a year.
• compumike 5y ago
  You may want to check out Crystal https://crystal-lang.org/ https://crystal-lang.org/ -- you will get all of the positives you describe, while avoiding perhaps 3 out of 4 of your listed negatives. (I think your "ease of iteration" point will still probably win for PHP vs any compiled language.) Also, I'm very curious: how does the KLOC count compare for the PHP and Rust versions?
  • muglug 5y ago
    > how does the KLOC count compare for the PHP and Rust versions I don't have exact numbers since it's not a straight port — the original PHP code was written to analyse a slightly different language than the Rust version I'm building. I'd estimate that Rust requires you to be about 10% more verbose than PHP. Main annoyances vs PHP: no default argument values, and only a single let allowed inside an if conditional.
  • ByteJockey 5y ago
    > I think your "ease of iteration" point will still probably win for PHP vs any compiled language. While I don't think you can beat straight php/js/python for this, there's still a lot of room for improvement over the rust/c++ magnitude of compile times. For example Go/OCaml/Common Lisp (at least, SBCL is a compiler, there are lisp interpreters), all have reasonably quick compile times. And the latter two have repls, which really does help improve the iteration cycle. I haven't had to play with crystal, where does it fall on this spectrum?
• ebingdom 5y ago
  > I sometimes wonder whether people adopt Rust/Haskell/ReasonML for front-end work because they want it to feel like more of an effort than it otherwise would. That's ridiculous. There's a lot of garbage to unpack in that sentence, but for example Rust's borrow checker provides valuable guarantees about your code and eliminates entire classes of bugs. It's not just wasted time. Haskell's purity and lazy semantics allows for equational reasoning and more composable abstractions. ReasonML makes it really hard to accidentally write a bug that crashes the program. Etc. Just because you don't see why those things are valuable doesn't mean they aren't.
• csomar 5y ago
  I disagree on the things you've lost. > - concise, readable code Rust code is significantly more readable. Use the type system to your advantage. > - - really smart type inference Rust has really smart type inference too and only rarely ask for an explicit type. I've found that I prefer annotating my types. It adds to documentation and readability. > - a bunch of time thinking about the borrow checker You can use Arc/Mutex to overcome some of these and review them in a refactoring. > - ease of iteration At the cost of introducing a bug/technical debt.
  • muglug 5y ago
    Rust has smart type checking. The really smart stuff is (IMO) the beauty of the borrow checker. In PHP, modern typecheckers can verify that the first element always exists on an array after an emptiness check: if ($some_arr) { echo reset($arr); } Whereas in Rust you have to explicitly unwrap: if !some_vec.empty() { print!("{}", some_vec.first().unwrap()); // .. more code } The Rust version requires you to use your intuition to figure out that `unwrap()` will never panic here. In my Rust port I consistently rely on the PHP typechecker's knowledge of the equivalent code when using `unwrap()`, because it knows which array fetches are safe. > At the cost of introducing a bug/technical debt. At the cost of shipping a product! ------- Edit: user ibraheemdev proposes this Rust equivalent which more closely matches the idealised pseudocode, and type-checks: if let [first, ..] = &some_vec { print!("{}", first); // .. more code } It's preferable IMO to if let Some(first) = some_vec.first() { because the latter relies on a property of a non-empty vec, and checks that property, rather than checking the non-emptiness of the collection.
    • ibraheemdev 5y ago
      You can usually use pattern matching to get around a check followed by an unwrap. I would rewrite your example as: if let [first, ..] = &some_vec { print!("{}", first); // .. more code }
      • muglug 5y ago
• JodieBenitez 5y ago
  > Things I've lost: > - ease of iteration > - concise, readable code Ouch... then it's a big nope for me.
  • mustache_kimono 5y ago
    When a person converts their 100 KLOC PHP command line utility(!) to Rust and that's not a bigger red flag than their negatives list, it's a world gone mad.
    • muglug 5y ago
      > When a person converts their 100 KLOC PHP command line utility(!) to Rust and that's not a bigger red flag than their negatives list, it's a world gone mad. I'm not converting, I'm just porting a good chunk of it for a slightly different purpose. Not sure what's a red flag -- the original PHP utility has 17 million downloads and people are pretty happy with it.
      • mustache_kimono 5y ago
        It was a joke. It's great that people like your utility, and that you're trying Rust. The source of my amusement was -- I really, really don't see the world the same way as you do. I won't take your negatives point by point, but, for instance, familiarity and readability do really seem to be in the eye of the beholder, because, to my eyes, to call PHP concise and readable, especially as compared to Rust, is a spit take. PHP is emphatically not readable to me. Writing a 100KLOC project in PHP? Ugh. Yeah, you do you, but not this guy.
• juststeve 5y ago
  The 3x speed increase is interesting. Do you have any further details?
  • muglug 5y ago
    Yes — that's after a ton of Rust-centric optimisation. About 25% of the runtime is currently consumed with deallocating memory (there are a lot of heavily-nested data structures getting cleaned up), so there's definitely some more work to be done to reduce cloning.
    • juststeve 5y ago
      what's the total running time? ms/sec/minutes/hours? is it heavy IO bound?
      • muglug 5y ago
        A few minutes. It's not heavily IO-bound — it's mostly memory-bound (it's a type-checking tool for very large codebases). Further (painful) optimisations will likely involve the use of arenas to prevent so many atomic de-allocations.
  • tored 5y ago
    Only 3x speed increase feels like either did PHP a very good job or the port is not optimized for rust yet.
    • muglug 5y ago
      A bit of one, a bit of the other. It's fairly well-optimised. I think the big remaining optimisation will be to de-allocate memory less frequently — the program creates and destroys millions of structs representing types. 20% of the runtime is taken up iterating over and dropping collections of type-related structs.
• neandrake 5y ago
  I imagine that especially for a command line utility you also gained an easier deployment story when using rust, given the static linking and lack of interpreter. Requiring PHP to be installed to use a tool has caused me issues, especially with some of the recent PHP 8 updates introducing breaking changes. It’s a better story for PHP on a server where it’s more centrally managed.
  • muglug 5y ago
    The deployment story of the PHP tool in question is pretty good thanks to the semi-official Composer package management system (it hasn't been officially endorsed, but everybody uses it). People just run `composer install --dev vimeo/psalm` and it works. It can also run on any web server that runs PHP, so I have it running on psalm.dev OfC the Rust equivalent is very easy to package, and also runs reliably in the browser via WASM (currently ~1.5MB compressed).
• darthrupert 5y ago
  >really smart type inference Are you calling php's chaotic mess of type juggling "really smart type inference"?