4 ms·

Python extension doesn’t mean C. Rust works perfectly for extensions, it covers a lot of low level c-api integration and it is fast. You can write whole applica

Python extension doesn’t mean C. Rust works perfectly for extensions, it covers a lot of low level c-api integration and it is fast. You can write whole application in rust and use python as a glue language https://github.com/PyO3/pyo3 https://github.com/PyO3/pyo3 Pyo3 library gives you ability to work both diractions. Call python code from rust and call rust code from python.

jsmthrowaway 9y ago
Cython interops just fine with Rust, too.
jerf 9y ago
That sounds like one of the "maze of choices" I mentioned, no? And if you're "writing the whole application in Rust and using Python as a glue language", you don't have the problem that this entire discussion is about, which is when you have Python code that is slow. Python as an extension language is a completely different world. Performance problems there are a much less big deal, because you've already got the option to simply use the fast language with only modestly more complexity, if indeed even that given how nice Rust is once you get used to it. It's when your whole app is in Python that these issues emerge, and "Just write extensions" is an option far less often than portrayed.
- fafhrd91 9y ago
  My point is, you are not limited with using extensions only for optimizing hot loops, in rust you can write application logic as well. I doubt you should do this in C for example
deathanatos 9y ago
PyO3 is a fork of rust-cpython, which has a nasty abort issue[1] which is unfortunately a show-stopper for me. It isn't clear to me if PyO3 is also affected by this issue. [1]: https://github.com/dgrunwald/rust-cpython/issues/59 https://github.com/dgrunwald/rust-cpython/issues/59
- fafhrd91 9y ago
  Pyo3 is not affected by this issue. pyo3 compiles in c-api interface, it doesn't use separate libs for that (python27-sys)
  - deathanatos 9y ago
    > Pyo3 is not affected by this issue. pyo3 compiles in c-api interface, it doesn't use separate libs for that (python27-sys) At some point, it must use a separate "lib" for that. Some of the core functions in the Python API exist in the Python binary; it is debatable whether one considers that a "separate lib", or not, but is isn't possible to compile it into your binary. (Or there would be two of whatever you decide to compile in, and that would be problematic.) I looked into it, since you said it was not be affected. PyO3's own README notes that it is affected by the issue; it lists the same proposed solution as the bug against rust-cpython does. While the solution "works", in the sense that you can build a working module from it, the problem with the solution is that the ergonomics of it are terrible; my understanding is that it completely prevents one from being able to `cargo build`. That said, I was not aware of either `cargo rustc` or `setuptools-rust`; at the time I was looking into it, setuptools lacked the necessary support to implement `setuptools-rust`, so that's nice to see that that has finally occurred. `cargo rustc` alleviates much of the concern around the ergonomics of building the extension, though that'll still be fun to explain to coworkers. The combination of all that would seem to imply that building Rust extensions might finally be somewhat feasible.

jsmthrowaway 9y ago

Cython interops just fine with Rust, too.

jerf 9y ago

That sounds like one of the "maze of choices" I mentioned, no? And if you're "writing the whole application in Rust and using Python as a glue language", you don't have the problem that this entire discussion is about, which is when you have Python code that is slow. Python as an extension language is a completely different world. Performance problems there are a much less big deal, because you've already got the option to simply use the fast language with only modestly more complexity, if indeed even that given how nice Rust is once you get used to it. It's when your whole app is in Python that these issues emerge, and "Just write extensions" is an option far less often than portrayed.

fafhrd91 9y ago
My point is, you are not limited with using extensions only for optimizing hot loops, in rust you can write application logic as well. I doubt you should do this in C for example

deathanatos 9y ago

PyO3 is a fork of rust-cpython, which has a nasty abort issue[1] which is unfortunately a show-stopper for me. It isn't clear to me if PyO3 is also affected by this issue. [1]: https://github.com/dgrunwald/rust-cpython/issues/59 https://github.com/dgrunwald/rust-cpython/issues/59

fafhrd91 9y ago
Pyo3 is not affected by this issue. pyo3 compiles in c-api interface, it doesn't use separate libs for that (python27-sys)
- deathanatos 9y ago
  > Pyo3 is not affected by this issue. pyo3 compiles in c-api interface, it doesn't use separate libs for that (python27-sys) At some point, it must use a separate "lib" for that. Some of the core functions in the Python API exist in the Python binary; it is debatable whether one considers that a "separate lib", or not, but is isn't possible to compile it into your binary. (Or there would be two of whatever you decide to compile in, and that would be problematic.) I looked into it, since you said it was not be affected. PyO3's own README notes that it is affected by the issue; it lists the same proposed solution as the bug against rust-cpython does. While the solution "works", in the sense that you can build a working module from it, the problem with the solution is that the ergonomics of it are terrible; my understanding is that it completely prevents one from being able to `cargo build`. That said, I was not aware of either `cargo rustc` or `setuptools-rust`; at the time I was looking into it, setuptools lacked the necessary support to implement `setuptools-rust`, so that's nice to see that that has finally occurred. `cargo rustc` alleviates much of the concern around the ergonomics of building the extension, though that'll still be fun to explain to coworkers. The combination of all that would seem to imply that building Rust extensions might finally be somewhat feasible.