3 ms·

I'm curious what kind of code gets a 45x speedup by going from python to rust and by that I don't mean the rhetorical or bait-style "i'm curious", no, the lite

I'm curious what kind of code gets a 45x speedup by going from python to rust and by that I don't mean the rhetorical or bait-style "i'm curious", no, the literal I'm curious, cause I'm trying to find use cases such as that these days and I'm often thwarted by the fact that for anything requiring remotely decent speeds, python most of the time already delegates to C extensions and so any rewrite is not as useful

cyberax 2y ago
> I'm curious what kind of code gets a 45x speedup by going from python to rust Pretty much any code that is not just tying together external libraries?
Aeolos 2y ago
Anecdotal experience: we rewrote an image processing algorithm from numpy+scipy to pure rust and got a 50x speedup in release builds, without even spending any effort optimizing the rust side. There are further improvements possible around memory allocation and cachelines, but 2 days for 50x improvement was sufficient to not make it worth investing additional effort. Edit: this was from a team who had _never_ touched Rust before.
IshKebab 2y ago
It is basically reading a massive JSON file containing a few thousand logs and then scanning them with a load of regexes. I was a bit surprised how much faster it was too. Apart from Python being dog slow the only thing I really changed was to use RegexSet which isn't available in Python. I didn't benchmark how much difference that made though; I just used it because it was obviously the right thing to do. That's kind of the point. If you just do the obvious thing in Rust you get very good performance by default. It's the same in C++ but then you're writing C++.
- tialaramex 2y ago
  The actual regular expression implementation in Rust is going to be fast, but one of the things that caught a reddit poster out only recently was that the Rust regex crate's parser doesn't magically cache, so if you sit in a tight loop making the same regex over and over, it'll do all that work over and over, whereas the Python code might take ten times longer to do it once, then caches it, it doesn't take long for that to end up faster. Now, if you're going to use RegexSet you're also smart enough to read "For example, it’s a bad idea to compile the same regex repeatedly in a loop" and say "Yeah, makes sense, I will not repeatedly compile the same regex". But some fraction of Python programmers won't read that - and it'll be very slow.
  - burntsushi 2y ago
    There's plenty of examples like that though. A Python programmer might not know to compile in release mode. They might not use buffering when reading from a file. They might pass around copius copies of Vec<T> instead of &[T]. The list could go on and on.
    - tialaramex 2y ago
      Sure, and there would probably be some value in a tool which can walk them through the easy stuff before they show a real human code which it turns out just wasn't tested with release optimisations or whatever. Still, as I understand it CTRE means if you just "use" the same expression over and over in your inner loop in C++ (with CTRE) it doesn't matter, because the regular expression compilation happened in compilation as part of the type system, your expression got turned into machine code once for the same reason Rust will emit machine code for name.contains(char::is_lowercase) once not somehow re-calculate that each time it's reached - so there is no runtime step to repeat. This is a long way down my "want to have" list, it's below BalancedI8 and the Pattern Types, it's below compile-time for loops, it's below stabilizing Pattern, for an example closer to heart. But it does remind us what's conceivable.
      - burntsushi 2y ago
        IDK how we jumped to CTRE. Python doesn't do CTRE. It's doing caching. In Rust, you use std::sync::LazyLock for that. I don't get what the problem is to be honest. I assume by CTRE you're referring to the CTRE C++ project. That's a totally different can of worms and comes with lots of trade-offs. I wish it were easy to add CTRE to rebar, then I could probably add a lot more color to the trade-offs involved, at least with that specific implementation (but maybe not to "compile time regex" in general).
        tialaramex 2y ago
        I jumped to CTRE because it's another way that you can get the better results. The programmer need have no idea why this works, just like with caches. I agree that there are trade-offs, but nevertheless compile time regex compilation is on my want list, even if a long way down it. I would take compile time arithmetic compilation† much sooner, but since that's an unsolved problem I don't get that choice. † What I mean here is, you type in the real arithmetic you want, the compiler analyses what you wrote and it spits out an approximation in machine code which delivers an accuracy and performance trade off you're OK with, without you needing to be an expert in IEEE floating point and how your target CPU works. Herbie https://herbie.uwplse.org/ https://herbie.uwplse.org/ but as part of the compiler.
tuveson 2y ago
If you heavily rely on the Python standard library, then you’re using a lot of Python code that doesn’t call out to C extensions. Peruse the standard library code, if you want to get a sense of it: https://github.com/python/cpython/tree/main/Lib https://github.com/python/cpython/tree/main/Lib So you can expect any code that heavily relies on the standard library to be slower than the Rust equivalent. A purely interpreted language implementation (not JIT’d) like CPython is almost always going to have a 10x-100x slowdown compared to equivalent code in C/C++/Rust/Go or most other compiled/JIT’d languages. So unless your program spends the vast majority of time in C extensions, it will be much slower.
danudey 2y ago
This is an old example, but - date/time parsing. A coworker of mine years ago was trying to parse out some large logfiles and it was running incredibly slowly (because the log file was huge). Just for fun he profiled the code and found that 90% of the time was spent taking the timestamp ("2019-04-22 15:24:41") into a Python datetime. It was a slow morning, so we went back and forth trying to come up with new methods of optimizing this parsing, including (among other things) creating a dict to map date strings to datetime objects (since there were a lot of repeats). After some even more profiling, I found that most of the slowdown happened because most of Python's strptime() implementation is written in Python so that it can handle timezones correctly; this prevented them from just calling out to the C library's strptime() implementation. Since our timestamps didn't have a timezone specified anyway, I wrote my first ever C module[0] for Python, which simply takes two strings (the format and the timestamp) and runs them through strptime and returns a python datetime of the result. I lost the actual benchmark data before I had a chance to record it somewhere to reproduce, and the Python 3 version in my repo doesn't have as much of a speedup compared to the default Python code, but the initial code that I wrote provided a 47x performance boost to the parsing compared to the built-in Python strptime(). Anyone who had a similar Python script and converted it wholesale to Rust (or C or Golang, probably) would have seen a similarly massive increase in performance. [0] https://github.com/danudey/pystrptime/ https://github.com/danudey/pystrptime/
- jcgrillo 2y ago
  One could argue that writing a timestamp as a string which then has to be parsed is silly and instead it should be delta-of-delta encoded and packed into variable width integers, but even then double integrating and constructing a datetime for each one would still be expensive in python, only less so.
jerf 2y ago
"I'm often thwarted by the fact that for anything requiring remotely decent speeds, python most of the time already delegates to C extensions and so any rewrite is not as useful" Be sure you verify this is the case for whatever you think it is, though. Pure Python is so much slower than compiled languages (not just Rust) that you don't have to do much percentage-wise in pure Python before you've badly fallen behind in performance versus the pure-compiled alternatives. I think this is asserted a lot more often then it is benchmarked. I am reminded of the way people for a long time asserted that the performance of web languages doesn't matter because you spend all your time waiting for the database, so it never mattered. People would just whip this argument out reflexively. It turns out that if you take a non-trivial codebase written in such a language and actually benchmark it, it is often not true, because as applications grow they tend to rapidly outgrow "all my code is just running a SELECT and slamming the results with minimal processing out to the web stream". I hear this a lot less often than I used to, probably through the slow-but-effective process of a lot of individuals learning the hard way this isn't true. I've seen a lot of Python code. Very little of it that was not "data science" was just a bit of scripting around lots of large C-based objects, such that Python wasn't doing much actual work. And even some of that "data science" was falling back to pure Python without realizing because NumPy actually makes that shockingly easy.
- NoGravitas 2y ago
  It would be interesting to me to look at something written in Python and rewritten in Rust with a 40x speed-up, and then rewrite it in something like C# or Common Lisp, and see what the speed-up is. My gut tells me the Rust implementation would use significantly less memory than the CL one, but be only minimally faster, if at all. But my gut has been known to be unreliable in the past.
  - estebank 2y ago
    When comparing a networked service that wrote to disk (a-la magic wormhole) written in Java and in Rust, after 3 iterations of improvements on the respective implementations the throughput was the same for both, CPU utilization was comparable, but memory usage was orders of magnitude lower for Rust. I think that Java will have difficulty closing the gap until project Valhalla (value types in the JVM) is completed, but even then it'll be difficult to bring the ecosystem along to materialize all its benefits.
Etherlord87 2y ago
Conway's Game of Life perhaps is a good example: it's a simple program, with a tight loop and cheap calculation in the loop. Python's loops are slow. I wouldn't be surprised if the speedup was much greater than 45x.
sfink 2y ago
My "I'm gonna learn Rust!" initial project was porting a callgraph traversal tool from Python to Rust. For loading in my input file, the speedup was very disappointing: 1x. (As in, no speedup at all.) Then I learned about the `--release` flag and it instantly became a 40x speedup. So that was nice. Waiting 30s vs <1s puts it well within "anything requiring remotely decent speeds". But it was really about parsing a custom data format, nothing fancy. I haven't done comparison timings of the graph traversals, but everything is basically instantaneous in the Rust version and not in the Python.

cyberax 2y ago

> I'm curious what kind of code gets a 45x speedup by going from python to rust Pretty much any code that is not just tying together external libraries?

Aeolos 2y ago

Anecdotal experience: we rewrote an image processing algorithm from numpy+scipy to pure rust and got a 50x speedup in release builds, without even spending any effort optimizing the rust side. There are further improvements possible around memory allocation and cachelines, but 2 days for 50x improvement was sufficient to not make it worth investing additional effort. Edit: this was from a team who had _never_ touched Rust before.

IshKebab 2y ago

It is basically reading a massive JSON file containing a few thousand logs and then scanning them with a load of regexes. I was a bit surprised how much faster it was too. Apart from Python being dog slow the only thing I really changed was to use RegexSet which isn't available in Python. I didn't benchmark how much difference that made though; I just used it because it was obviously the right thing to do. That's kind of the point. If you just do the obvious thing in Rust you get very good performance by default. It's the same in C++ but then you're writing C++.

tialaramex 2y ago
The actual regular expression implementation in Rust is going to be fast, but one of the things that caught a reddit poster out only recently was that the Rust regex crate's parser doesn't magically cache, so if you sit in a tight loop making the same regex over and over, it'll do all that work over and over, whereas the Python code might take ten times longer to do it once, then caches it, it doesn't take long for that to end up faster. Now, if you're going to use RegexSet you're also smart enough to read "For example, it’s a bad idea to compile the same regex repeatedly in a loop" and say "Yeah, makes sense, I will not repeatedly compile the same regex". But some fraction of Python programmers won't read that - and it'll be very slow.
- burntsushi 2y ago
  There's plenty of examples like that though. A Python programmer might not know to compile in release mode. They might not use buffering when reading from a file. They might pass around copius copies of Vec<T> instead of &[T]. The list could go on and on.
  - tialaramex 2y ago
    Sure, and there would probably be some value in a tool which can walk them through the easy stuff before they show a real human code which it turns out just wasn't tested with release optimisations or whatever. Still, as I understand it CTRE means if you just "use" the same expression over and over in your inner loop in C++ (with CTRE) it doesn't matter, because the regular expression compilation happened in compilation as part of the type system, your expression got turned into machine code once for the same reason Rust will emit machine code for name.contains(char::is_lowercase) once not somehow re-calculate that each time it's reached - so there is no runtime step to repeat. This is a long way down my "want to have" list, it's below BalancedI8 and the Pattern Types, it's below compile-time for loops, it's below stabilizing Pattern, for an example closer to heart. But it does remind us what's conceivable.
    - burntsushi 2y ago
      IDK how we jumped to CTRE. Python doesn't do CTRE. It's doing caching. In Rust, you use std::sync::LazyLock for that. I don't get what the problem is to be honest. I assume by CTRE you're referring to the CTRE C++ project. That's a totally different can of worms and comes with lots of trade-offs. I wish it were easy to add CTRE to rebar, then I could probably add a lot more color to the trade-offs involved, at least with that specific implementation (but maybe not to "compile time regex" in general).
      - tialaramex 2y ago
        I jumped to CTRE because it's another way that you can get the better results. The programmer need have no idea why this works, just like with caches. I agree that there are trade-offs, but nevertheless compile time regex compilation is on my want list, even if a long way down it. I would take compile time arithmetic compilation† much sooner, but since that's an unsolved problem I don't get that choice. † What I mean here is, you type in the real arithmetic you want, the compiler analyses what you wrote and it spits out an approximation in machine code which delivers an accuracy and performance trade off you're OK with, without you needing to be an expert in IEEE floating point and how your target CPU works. Herbie https://herbie.uwplse.org/ https://herbie.uwplse.org/ but as part of the compiler.

tuveson 2y ago

If you heavily rely on the Python standard library, then you’re using a lot of Python code that doesn’t call out to C extensions. Peruse the standard library code, if you want to get a sense of it: https://github.com/python/cpython/tree/main/Lib https://github.com/python/cpython/tree/main/Lib So you can expect any code that heavily relies on the standard library to be slower than the Rust equivalent. A purely interpreted language implementation (not JIT’d) like CPython is almost always going to have a 10x-100x slowdown compared to equivalent code in C/C++/Rust/Go or most other compiled/JIT’d languages. So unless your program spends the vast majority of time in C extensions, it will be much slower.

danudey 2y ago

This is an old example, but - date/time parsing. A coworker of mine years ago was trying to parse out some large logfiles and it was running incredibly slowly (because the log file was huge). Just for fun he profiled the code and found that 90% of the time was spent taking the timestamp ("2019-04-22 15:24:41") into a Python datetime. It was a slow morning, so we went back and forth trying to come up with new methods of optimizing this parsing, including (among other things) creating a dict to map date strings to datetime objects (since there were a lot of repeats). After some even more profiling, I found that most of the slowdown happened because most of Python's strptime() implementation is written in Python so that it can handle timezones correctly; this prevented them from just calling out to the C library's strptime() implementation. Since our timestamps didn't have a timezone specified anyway, I wrote my first ever C module[0] for Python, which simply takes two strings (the format and the timestamp) and runs them through strptime and returns a python datetime of the result. I lost the actual benchmark data before I had a chance to record it somewhere to reproduce, and the Python 3 version in my repo doesn't have as much of a speedup compared to the default Python code, but the initial code that I wrote provided a 47x performance boost to the parsing compared to the built-in Python strptime(). Anyone who had a similar Python script and converted it wholesale to Rust (or C or Golang, probably) would have seen a similarly massive increase in performance. [0] https://github.com/danudey/pystrptime/ https://github.com/danudey/pystrptime/

jcgrillo 2y ago
One could argue that writing a timestamp as a string which then has to be parsed is silly and instead it should be delta-of-delta encoded and packed into variable width integers, but even then double integrating and constructing a datetime for each one would still be expensive in python, only less so.

jerf 2y ago

"I'm often thwarted by the fact that for anything requiring remotely decent speeds, python most of the time already delegates to C extensions and so any rewrite is not as useful" Be sure you verify this is the case for whatever you think it is, though. Pure Python is so much slower than compiled languages (not just Rust) that you don't have to do much percentage-wise in pure Python before you've badly fallen behind in performance versus the pure-compiled alternatives. I think this is asserted a lot more often then it is benchmarked. I am reminded of the way people for a long time asserted that the performance of web languages doesn't matter because you spend all your time waiting for the database, so it never mattered. People would just whip this argument out reflexively. It turns out that if you take a non-trivial codebase written in such a language and actually benchmark it, it is often not true, because as applications grow they tend to rapidly outgrow "all my code is just running a SELECT and slamming the results with minimal processing out to the web stream". I hear this a lot less often than I used to, probably through the slow-but-effective process of a lot of individuals learning the hard way this isn't true. I've seen a lot of Python code. Very little of it that was not "data science" was just a bit of scripting around lots of large C-based objects, such that Python wasn't doing much actual work. And even some of that "data science" was falling back to pure Python without realizing because NumPy actually makes that shockingly easy.

NoGravitas 2y ago
It would be interesting to me to look at something written in Python and rewritten in Rust with a 40x speed-up, and then rewrite it in something like C# or Common Lisp, and see what the speed-up is. My gut tells me the Rust implementation would use significantly less memory than the CL one, but be only minimally faster, if at all. But my gut has been known to be unreliable in the past.
- estebank 2y ago
  When comparing a networked service that wrote to disk (a-la magic wormhole) written in Java and in Rust, after 3 iterations of improvements on the respective implementations the throughput was the same for both, CPU utilization was comparable, but memory usage was orders of magnitude lower for Rust. I think that Java will have difficulty closing the gap until project Valhalla (value types in the JVM) is completed, but even then it'll be difficult to bring the ecosystem along to materialize all its benefits.

Etherlord87 2y ago

Conway's Game of Life perhaps is a good example: it's a simple program, with a tight loop and cheap calculation in the loop. Python's loops are slow. I wouldn't be surprised if the speedup was much greater than 45x.

sfink 2y ago

My "I'm gonna learn Rust!" initial project was porting a callgraph traversal tool from Python to Rust. For loading in my input file, the speedup was very disappointing: 1x. (As in, no speedup at all.) Then I learned about the `--release` flag and it instantly became a 40x speedup. So that was nice. Waiting 30s vs <1s puts it well within "anything requiring remotely decent speeds". But it was really about parsing a custom data format, nothing fancy. I haven't done comparison timings of the graph traversals, but everything is basically instantaneous in the Rust version and not in the Python.