5 ms·
What is the use case for embedding Python code in a Rust program?
by ptato 6y ago
What is the use case for embedding Python code in a Rust program?
- djohnston 6y agoMaybe interfacing w an ml lib like pytorch or something?
- RcouF1uZ4gsC 6y agoFor pytorch it would probably be easier to do something based off the pytorch C++ interface.
- djohnston 6y agoSure, I've never even used pytorch. But I'm just saying python has a fair bit of data analysis and ml tools that probably haven't found native homes in rust
- qchris 6y agoI suppose you could, but I believe that there are direct bindings from Rust to Pytorch at https://github.com/LaurentMazare/tch-rs https://github.com/LaurentMazare/tch-rs. I haven't used it personally, but I've heard fairly good things about it's correctness and responsive maintainer.
- danieldk 6y agoI have written a transformer-based (BERT, XLM-RoBERTa) sequence labeler/lemmatizer/dependency parser in Rust with tch-rs [1] and it is great! Some parts do not feel rusty. E.g. dynamic typing of tensors, you get a Tensor rather than Tensor<f32>, but that's more due to how libtorch itself works. But it's a very straightforward API that exposes most of libtorch. The maintainer is also very nice and responsive. I was also amazed how much is implemented in libtorch itself as opposed to the Python wrapper, which makes much of the Torch functionality available to other languages. [1] https://github.com/stickeritis/sticker2 https://github.com/stickeritis/sticker2 https://github.com/stickeritis/sticker-transformers https://github.com/stickeritis/sticker-transformers
- random32840 6y agoSomeone might have jumped that hurdle but the point is it circumvents that class of hurdle. What about the next framework which doesn't have bindings? What about leveraging existing Pytorch code that's written in Python?
- amanzi 6y agoI only had a quick scan through the article, but could this be used to create an executable for some existing Python code?
- choward 6y agoWhat would the point of that be versus wrapping it in a bash script? Obfuscation?
- lmm 6y agoPython is missing a nice simple way to ship native executables. If you have to send people a package to unpack that's an extra step. In theory you can use freeze, but configuring it all is a pain.
- trishmapow2 6y agoI've only used it for small GUIs (a couple of files & dependencies) but `pyinstaller --onefile` is about as easy as it gets.
- londt8 6y agoYou can use pip to install the executable source files. Pip is preinstalled on most Libux distributions and MacOS.
- choward 6y agoThat's exactly what I thought when I read the title. It's very frustrating how many landing pages, project readmes, blog posts, etc that don't answer the question "why?". I usually need to know that before going further when I come across something that I've never heard of before. If they don't have the "why?" somewhere easy to find I just close the tab.
- mehrdadn 6y agoYou know, that might not be unintentional. They might want those who don't see the value to just close the tab and move on. That way only those who see the value keep reading. I imagine it helps avoid having to engage in (often endless) arguments about the validity of the use case.
- deleted 6y ago[deleted]
- elpatoisthebest 6y agoIt's a bit of a guilty pleasure for me to read these kinds of weird things with no practical application. It's just a little refreshing to read about something technically possibly, but frivolous (no offense meant to the OP, of course) just because it can be done.
- owl57 6y ago"Because it can be done" is a very solid answer to "Why?"
- aspenmayer 6y agoIt helps me to step outside my own context and view a familiar use case, problem, or solution with an unfamiliar perspective.
- brainless 6y agoI am quite surprised by your message. This is HN and even before or outside of that, for years I have myself dived into totally random projects simply because "why not?". To try random ideas is part of learning and when you have something to share, just do that. There will always be like minded someone to pick up.
- granzymes 6y agoIf imports worked I could see it being very useful for making graphs. I've heard of people serializing data in other languages and then using Python to plot it.
- qchris 6y agoI can confirm this--Rust's data visualization and plotting libraries are still pretty early stage. You can generally do 2D bar charts and scatter plots pretty well, but once you start jumping into more complex representations or three dimensions, you start running into some serious barriers.
- m-ou-se 6y agoImports work fine. The main reason I wrote this, was to be able to use the Python matplotlib library in Rust: https://twitter.com/m_ou_se/status/1120577172438233088 https://twitter.com/m_ou_se/status/1120577172438233088
- memco 6y agoI’m just throwing out ideas but what if I wanted to take working python code and convert it to rust? Could I use this to start as the baseline and start replacing bits of it and see that it still produces the same output?
- zerkten 6y agoThere was a recent ATP.fm podcast with Chris Lattner of Swift fame. He talked about the topic of embedding Python code in Swift for TensorFlow and it overlaps with your suggestion.
- NobodyNada 6y agoSwift for Tensorflow has implemented Python interoperability, rather than embedded Python code as in this article. A Swift program can import a Python module and interact with Python objects and functions as if they were native Swift objects (minus strong typing, of course). See https://github.com/tensorflow/swift/blob/master/docs/PythonInteroperability.md https://github.com/tensorflow/swift/blob/master/docs/PythonI...
- akiselev 6y agoRust has had Python interop through the Python C API since around 2015 [1]. It's pretty low hanging fruit for any language that supports the C FFI. Rust-cpython has had simple macros for interop for ages [2] and there's even a library that uses serde macros to encode/decode pickled object [3]. This article is just a fun hack using Rust macros. [1] https://github.com/dgrunwald/rust-cpython https://github.com/dgrunwald/rust-cpython [2] http://dgrunwald.github.io/rust-cpython/doc/cpython/macro.py_class.html http://dgrunwald.github.io/rust-cpython/doc/cpython/macro.py... [3] https://docs.rs/serde-pickle/0.6.0/serde_pickle/ https://docs.rs/serde-pickle/0.6.0/serde_pickle/
- deleted 6y ago[deleted]
- rtpg 6y agoPlugins for example? Game scripts? Think of use cases for embedding Lua into a C program.
- charliesome 6y agoThere could be no reasonable use case for something like this, and yet it would still have artistic value and this would still be an interesting article. It's a brilliant hack and we are on Hacker News after all.
- deleted 6y ago[deleted]
- tedmielczarek 6y agoA friend of mine has a similar project called PyOxidizer that's intended for building standalone executables with Python+Rust: https://pyoxidizer.readthedocs.io/en/stable/ https://pyoxidizer.readthedocs.io/en/stable/ His primary use for it is building distributable binaries for Mercurial, which is written primarily in Python.