4 ms·

My new favorite way to write very fast libraries for Python is to just use Rust and Maturin: https://github.com/PyO3/maturin https://github.com/PyO3/maturin I

My new favorite way to write very fast libraries for Python is to just use Rust and Maturin: https://github.com/PyO3/maturin https://github.com/PyO3/maturin It basically automates everything for you. If you use it with Github actions, it will compile wheels for you on each release for every platform and python version you want, and even upload them to PyPi (pip) for you. Everything feels very modern and well thought out. People really care about good tooling in the Rust world.

cozzyd 3y ago
this is indeed what the article advocates for (yes, this particular bit of rust evangelism was not obvious from the headline)
abdullahkhalids 3y ago
I am wanting to convert about a 5 functions/100 lines of my Python project into Rust. I cobbled together the Maturin integration with my project earlier this week, which seems to work on some test functions. But I don't know any Rust! What's the best way to learn enough Rust to do this? My code is basically just some Numpy array manipulation, with some unfortunate for-loops which can't be vectorized, which is the source of the slow speeds.
- aardshark 3y ago
  Open up Chat GPT, paste your functions and ask it to convert them to rust. Go through them 1 by 1, see if you understand and ask questions about anything you don't recognise. Don't expect the output to be perfectly logically correct, you will have to ensure that yourself. I've found Chat GPT to be really excellent for quickly getting myself up to speed with languages that I'm not familiar with.
  - eigenvalue 3y ago
    Yes, that’s my advice as well. Set up vscode with rust analyzer and paste any errors it shows back into the same ChatGPT conversation and it will debug everything for you.
    - abdullahkhalids 3y ago
      Thanks, that sounds good. I will search myself, but is there are a great rust syntax reference doc?
      - eigenvalue 3y ago
        The syntax isn’t really the hard part, it’s the rules around memory ownership and when you need to clone variables. There are lots of gotchas and the compiler is very unforgiving. If you want to convert your 5 functions into rust in a reasonable time instead of spending weeks grappling with the differences, just use ChatGPT and iterate by pasting in the errors from rust analyzer and the compiler. If you actually want to learn rust then that’s a different story and you should probably check out Steve Klabnik‘s book or something like that (or just look at the sample code I linked to in my other comment from my own recent rust library for python).
- eigenvalue 3y ago
  You can see how I did something similar in my library here: https://github.com/Dicklesworthstone/fast_vector_similarity/blob/main/src/lib.rs https://github.com/Dicklesworthstone/fast_vector_similarity/... Basically you use ndarray instead of numpy, try to vectorize anything you can, and for the for loops that can’t be vectorized, you can use rayon to do them in parallel.
IshKebab 3y ago
Maturin is great. I've used it to distribute a Rust program that has absolutely nothing to do with Python. It compiled it fine and once I had navigated the usual mess of Python to find out how to upload packages to PyPi (not to be confused with PyPy), it worked pretty well. I got the idea from CMake, which also has absolutely nothing to do with Python but is best installed via Pip. It's a package manager that basically works and is basically always available on Linux and Mac (among programmers anyway). One of the few areas of Python that doesn't completely suck.

cozzyd 3y ago

this is indeed what the article advocates for (yes, this particular bit of rust evangelism was not obvious from the headline)

abdullahkhalids 3y ago

I am wanting to convert about a 5 functions/100 lines of my Python project into Rust. I cobbled together the Maturin integration with my project earlier this week, which seems to work on some test functions. But I don't know any Rust! What's the best way to learn enough Rust to do this? My code is basically just some Numpy array manipulation, with some unfortunate for-loops which can't be vectorized, which is the source of the slow speeds.

aardshark 3y ago
Open up Chat GPT, paste your functions and ask it to convert them to rust. Go through them 1 by 1, see if you understand and ask questions about anything you don't recognise. Don't expect the output to be perfectly logically correct, you will have to ensure that yourself. I've found Chat GPT to be really excellent for quickly getting myself up to speed with languages that I'm not familiar with.
- eigenvalue 3y ago
  Yes, that’s my advice as well. Set up vscode with rust analyzer and paste any errors it shows back into the same ChatGPT conversation and it will debug everything for you.
  - abdullahkhalids 3y ago
    Thanks, that sounds good. I will search myself, but is there are a great rust syntax reference doc?
    - eigenvalue 3y ago
      The syntax isn’t really the hard part, it’s the rules around memory ownership and when you need to clone variables. There are lots of gotchas and the compiler is very unforgiving. If you want to convert your 5 functions into rust in a reasonable time instead of spending weeks grappling with the differences, just use ChatGPT and iterate by pasting in the errors from rust analyzer and the compiler. If you actually want to learn rust then that’s a different story and you should probably check out Steve Klabnik‘s book or something like that (or just look at the sample code I linked to in my other comment from my own recent rust library for python).
eigenvalue 3y ago
You can see how I did something similar in my library here: https://github.com/Dicklesworthstone/fast_vector_similarity/blob/main/src/lib.rs https://github.com/Dicklesworthstone/fast_vector_similarity/... Basically you use ndarray instead of numpy, try to vectorize anything you can, and for the for loops that can’t be vectorized, you can use rayon to do them in parallel.

IshKebab 3y ago

Maturin is great. I've used it to distribute a Rust program that has absolutely nothing to do with Python. It compiled it fine and once I had navigated the usual mess of Python to find out how to upload packages to PyPi (not to be confused with PyPy), it worked pretty well. I got the idea from CMake, which also has absolutely nothing to do with Python but is best installed via Pip. It's a package manager that basically works and is basically always available on Linux and Mac (among programmers anyway). One of the few areas of Python that doesn't completely suck.