3 ms·
Prototype in Python, but scale in Rust. I do a lot of AI Engineering in Python, but production Data Engineering, and Feature Engineering workloads are done in
by Fiahil 3y ago
Prototype in Python, but scale in Rust.
I do a lot of AI Engineering in Python, but production Data Engineering, and Feature Engineering workloads are done in parallel with Rust. Simple and Fast.
You can even make custom-made python modules with a Rust core when Numpy matrices are not enough to crack your graph-traversing problem efficiently ! (Thanks PyO3 and Maturin)
- echelon 3y ago> Prototype in Python, but scale in Rust. Not even. Some people start building big systems with Java or Go. It's entirely appropriate to start in Rust too once you're more proficient in it.
- Fiahil 3y agoa bit difficult and controversial in Data Science, but okay. I’ve taught data scientists to use docker, I might be able to teach some Rust ! For anything that is not related to DS/ML/Opt, yes, start with Rust. I had tremendous success with distributed Rust microservices using ZeroMQ for backbone communication.
- brigadier132 3y agoSince you mentioned Python / Rust interop I wouldn't be surprised to see this have a Python binding made for it and code will likely be easily translated over when it needs to be productionized.
- jononor 3y agoI would be surprised if it does not happen. If it is not provided officially then very likey someone else will make it. Python is the lingua franca of the ML domain, after all.