4 ms·

From the benchmarks: > Rust is not that fast because it needs to copy data; I'm surprised. Don't know much about Rust, but isn't it hailed as being competiti

From the benchmarks: > Rust is not that fast because it needs to copy data; I'm surprised. Don't know much about Rust, but isn't it hailed as being competitive with C/C++ ?

iExploder 6y ago
rust has concept of references, maybe it was referring to its particular implementation of the problem
- efaref 6y ago
  The implementations for Rust and Julia are different in a way that means the comparison is not really apples-to-apples. The Julia implementation replaces the input list with a Julia object, which is of course trivial for Julia to iterate over. The Rust implementation is asked to convert the list-of-list-of-floats back to Python types, and then back into Rust types. Of course converting 100M floats back and forth is going to take a non-trivial amount of time. The equivalent in Rust would be to wrap the return value in a PyObject that contains the Rust `Vec<Vec<f64>>`, and the Rust code could then just iterate that directly. It would even be made safe without the GIL as you'd implement your own mutex on the Rust side if you wanted it to be mutable. If you do this then it runs in about 1.2s on my laptop.
deleted 6y ago
[deleted]
PartiallyTyped 6y ago
The culprit here is `a_list: Vec<Vec<f64>>`, which means the list of lists is copied as a whole, hence why the rust version is slower. In general, rust is at least as fast as C or C++, but this seems to be a special case.
- hobofan 6y ago
  It's also repeatedly `push`-ing elements into the `Vec`, which will likely lead to repeated allocation and copying. In a case like this were you pretty clearly know the size beforehand you should crate a Vec of the appropriate size with `Vec::reserve`.
  - Sukera 6y ago
    The julia version also does that though
yakubin 6y ago
You left out the explanation from your quote. The full is: > Rust is not that fast beacuse it needs to copy data; using Pyo3 objects would probably lead to similar results as cython, but with an added library. "It needs to copy data", because it's converting Python objects into Rust objects and back again. As the full quote states, it could be written in a different way, although that would make the code look really weird. In this benchmark there is also no normal C++ involved. There is only Cython using some C++ ints, but operating on Python lists. And that's not handled by writing C++ code that operates on Python structures, but writing Python code in Cython in a "C++ mode". Apples to oranges.
- volta83 6y ago
  > although that would make the code look really weird. Not it wouldn't? Using Pyo3 is arguably the most idiomatic way of interfacing Rust with python. The "added library" cost mentioned as an excuse not to use it is literally the 1 second it takes me to write `cargo add pyo3` . They didn't do this because it would make their claim that "Julia is the better language for extending Python" kind of moot. They started from the conclusion that they wanted to have, and worked their way backwards towards implementation and data that would support it.
  - yakubin 6y ago
    > Not it wouldn't? I'll defer to your knowledge on that. I generally don't work with Python. I've only looked into interfacing C# with Rust and it looked crazy, so that's where I'm coming from. But yes, it's pretty clear that the results are more a consequence of what's in the source files than the languages used (if we can even say that e.g. C++ was used at all).
    - hobofan 6y ago
      > interfacing C# with Rust AFAIK, there is no library that achieves a similar idiomatic interoperability between Rust and C#, like PyO3 for Python.
- remram 6y ago
deleted 6y ago
[deleted]
mhh__ 6y ago
You mean C++. C usually comes out a bit worse because it gives even less information to the optimizer. And thanks to the GNU/LLVM monoculture most languages should perform roughly the same if they have them as a target e.g. I have found that D makes its it ridiculously easy to write highly specialized code that is both readable and visible to the optimizer (I will be blogging about that part later), but I'm sure I could force the same asm out from C++ or Rust etc.

From the benchmarks: > Rust is not that fast because it needs to copy data; I'm surprised. Don't know much about Rust, but isn't it hailed as being competitive with C/C++ ?

iExploder 6y ago

rust has concept of references, maybe it was referring to its particular implementation of the problem

efaref 6y ago
The implementations for Rust and Julia are different in a way that means the comparison is not really apples-to-apples. The Julia implementation replaces the input list with a Julia object, which is of course trivial for Julia to iterate over. The Rust implementation is asked to convert the list-of-list-of-floats back to Python types, and then back into Rust types. Of course converting 100M floats back and forth is going to take a non-trivial amount of time. The equivalent in Rust would be to wrap the return value in a PyObject that contains the Rust `Vec<Vec<f64>>`, and the Rust code could then just iterate that directly. It would even be made safe without the GIL as you'd implement your own mutex on the Rust side if you wanted it to be mutable. If you do this then it runs in about 1.2s on my laptop.

deleted 6y ago

[deleted]

PartiallyTyped 6y ago

The culprit here is `a_list: Vec<Vec<f64>>`, which means the list of lists is copied as a whole, hence why the rust version is slower. In general, rust is at least as fast as C or C++, but this seems to be a special case.

hobofan 6y ago
It's also repeatedly `push`-ing elements into the `Vec`, which will likely lead to repeated allocation and copying. In a case like this were you pretty clearly know the size beforehand you should crate a Vec of the appropriate size with `Vec::reserve`.
- Sukera 6y ago
  The julia version also does that though

yakubin 6y ago

You left out the explanation from your quote. The full is: > Rust is not that fast beacuse it needs to copy data; using Pyo3 objects would probably lead to similar results as cython, but with an added library. "It needs to copy data", because it's converting Python objects into Rust objects and back again. As the full quote states, it could be written in a different way, although that would make the code look really weird. In this benchmark there is also no normal C++ involved. There is only Cython using some C++ ints, but operating on Python lists. And that's not handled by writing C++ code that operates on Python structures, but writing Python code in Cython in a "C++ mode". Apples to oranges.

volta83 6y ago
> although that would make the code look really weird. Not it wouldn't? Using Pyo3 is arguably the most idiomatic way of interfacing Rust with python. The "added library" cost mentioned as an excuse not to use it is literally the 1 second it takes me to write `cargo add pyo3` . They didn't do this because it would make their claim that "Julia is the better language for extending Python" kind of moot. They started from the conclusion that they wanted to have, and worked their way backwards towards implementation and data that would support it.
- yakubin 6y ago
  > Not it wouldn't? I'll defer to your knowledge on that. I generally don't work with Python. I've only looked into interfacing C# with Rust and it looked crazy, so that's where I'm coming from. But yes, it's pretty clear that the results are more a consequence of what's in the source files than the languages used (if we can even say that e.g. C++ was used at all).
  - hobofan 6y ago
    > interfacing C# with Rust AFAIK, there is no library that achieves a similar idiomatic interoperability between Rust and C#, like PyO3 for Python.
remram 6y ago

deleted 6y ago

[deleted]

mhh__ 6y ago

You mean C++. C usually comes out a bit worse because it gives even less information to the optimizer. And thanks to the GNU/LLVM monoculture most languages should perform roughly the same if they have them as a target e.g. I have found that D makes its it ridiculously easy to write highly specialized code that is both readable and visible to the optimizer (I will be blogging about that part later), but I'm sure I could force the same asm out from C++ or Rust etc.