7 ms·

Pretty cool, especially the potential for using python as a scripting language embedded in rust programs. That said, python makes me wince.

avgcorrection 3y ago
Why does it make you wince?
- mavamaarten 3y ago
  From a non-python dev perspective: I always struggle with dependencies and versions. I have a script in front of me that I want to run, and am often just frustratingly brute-forcing commands to make it work. Do I: python? python3? pip install? pip3 install? python pip install? python pip3 install? python3 pip install? python3 pip3 install? And then everyone mentions "oh just use venv" or "conda" or docker or... It just never ends or it never clearly explains what to do. And all I want to do is run a relatively simple script. Or course I would use a docker image to run a more complex app. But it's frustratingly hard to manage for people who don't use it often. I'm from a java/Kotlin world where things aren't perfect either. But at least dependencies are clearly defined, and the only thing you need is Gradle that updates itself through a wrapper script if needed and pulls all dependencies. And a JVM/JDK which is also configured/installed quite clearly in a specific location.
  - diarrhea 3y ago
    Yeah, it's definitely painful. Over the years, I've probably spent in excess of 100 hours on understanding pip, venv, PYTHONPATH, poetry/pdm, pipfile, pyenv, ... still barely understand how it all works together. https://peps.python.org/pep-0723/ https://peps.python.org/pep-0723/ might make life much simpler for single-script applications.
  - FergusArgyll 3y ago
    for simple scripts, this is the easiest and works. python3 -m venv venv source venv/bin/activate pip install whatever echo "source path/to/venv/bin/activate && python3 path/to/app.py" > myscript mv myscript /usr/local/bin and just use myscript. I use this for a lot of little scripts
    - bomewish 3y ago
      Why not just Path/to/venv/bin/python path/to/app.py? Neat trick though, gonna adopt it!
    - ctippett 3y ago
      I haven't tried this myself, but you could probably save yourself the wrapper script and execute your script directly by adding a shebang pointing to the Python executable from your virtual environment. printf '%s\n%s\n' "#!path/to/venv/bin/python3" "$(cat path/to/app.py)" > /usr/local/bin/myscript
  - cultofmetatron 3y ago
    I feel you. python itself has its warts but its mostly minor. The dependancy hell though.. god damn it confuses me every time. I don't understand why they can't get it together with one consistent dependency system with good ergonomics. ruby does it with gem. elixir handles things reasonably well with mix. rust does it REALLY well with cargo. hell even ocaml has opam. but python? it between conda, pip, pip3, virtualenv and who knows what else. its hard to make something that I knwo will just work on everything.
    - odyssey7 3y ago
      To highlight another dynamically typed, interpreted language ecosystem, npm has been providing a remarkably smooth and uniformly adopted service for the JavaScript community.
  - VagabundoP 3y ago
    Rye[1] is an all in one manager for python projects. Including the python versions and virtualenv, pip etc etc... It separates tool deps from app deps. Its all configured through a pyproject.toml config file. Its still new but works well. I'm transitioning to it from an unholy mess of pyenv, pip installs and other manual hacks. If you're starting a new python project that is more than just a straightforward script I'd use Rye from the get go. [1]https://rye-up.com/ https://rye-up.com/
  - ptx 3y ago
    Part of the problem is that this used to require third-party tools, which gave rise to lots of different tools, but nowadays everything you need is included in Python itself. The simplest way (in the sense of having the fewest components required) is this, using only built-in tools: $ python3 -m venv --upgrade-deps my-virtual-environment $ my-virtual-environment/bin/pip install whatever-third-party-package $ my-virtual-environment/bin/python3 my-script.py This creates an isolated virtual environment in "my-virtual-environment". Use the pip and python3 binaries inside that directory to install packages and run scripts. Done. (The "activate" step suggested in other comments is just a convenience for setting your PATH to refer to the virtual environment implicitly, so that's completely optional.) Note that Debian's python3 package leaves out some parts of the standard Python distribution, but you can install the python3-full package for a complete installation.
    - pletnes 3y ago
      This is a fairly good answer but on windows, there’s no python3, so that’s your first command being broken.
      - ptx 3y ago
        Right, on Windows the recipe needs a few alterations but is essentially the same. This should work: > py -3 -m venv --upgrade-deps my-virtual-environment > my-virtual-environment\Scripts\pip install whatever-third-party-package > my-virtual-environment\Scripts\python my-script.py This assumes that the Python launcher (py.exe) was installed when installing Python, which it is by default.
        IshKebab 3y ago
        See this is the problem. You're already several "ah yes and you need to"s deep.
        ptx 3y ago
        Not really. It's the same 3-step process in both cases.
  - weberer 3y ago
    Thankfully, most distros/OSs don't ship with Python 2 anymore, so `python` will just link to `python3`.
    - IshKebab 3y ago
      Nope. Debian doesn't ship a `python` at all, which I think is the sensible option. You should universally use `python3`. Except on Windows the official distribution is bonkers and has no `python3.exe`. Probably the easiest solution is to install it from the Microsoft store instead which does.
  - klibertp 3y ago
    You seriously consider Gradle - a kitchen sink and a monstrosity scripted in obscure, extremely dynamic language - a better solution just because it has a... wrapper script?! With Gradle, where do you define dependencies? All with version numbers in the `dependencies` block? In a separate file, declaring a hash map of versions and then a hash map of dependencies, and then putting the latter into `ext`, which magically propagates to just about everywhere in the build silently? In a separate file using some plugin that tries to replicate the lockfile functionality baked into most other dependency systems? In a separate TOML file, using the Catalog? ...and that's just about dependencies, I could go on for days about other Gradle features... It's not any better than Python. You're just used to it. The same complexity is there in both Python and Kotlin, you just learned to cope with one better than the other. (Gradle with Kotlin DSL and good IDE support is a little better, mostly because Kotlin devs are not as afraid of touching the build as when it's done in Groovy. It's still 2 orders of magnitude more complex than whatever you use for Python - unless you use zc.buildout or something like that.)
- postepowanieadm 3y ago
  Installing python software is hard. Golang is much easier for and end user.
  - vundercind 3y ago
    Go is what I reach for when I might otherwise write Python but need to actually, like, distribute my program.
- lumb63 3y ago
  Personally, I wince at Python because, while I view it as a quick-and-dirty scripting language, some people write production code in Python. And here we are ten years later, stuck with code that reads a few metrics from the system, packages them into JSON, and sends them to an API for monitoring. The code takes 7 seconds at 100% CPU to resolve the imports, every time it runs.
  - wasmitnetzen 3y ago
    You can write bad code in any language, I fail to see how that's Python's problem.
berkes 3y ago
I was recently looking for some ways to make a rust project dynamically configurable. Yaml/toml/etc are too static or are terrible to describe logic in (yet we do it all the time for CI, infra etc, ugh). WASM would be an option, but overengineered for my case. Ruby (through Artichoke), or Python (through RustPython); but they come with the downside of introducing Ruby or Python. I haven't decided yet, but would prefer Lua for it's simplicity. Or JavaScript (a subset) for how easy it can be limited in scope. Are there any (example) rust programs out there that have embedded scripting, or runtime plugins or addons in a scripting language?
- ddorian43 3y ago
  why not lua?
  - berkes 3y ago
    No particular reason. edit: to be clear, I did mention "I would prefer lua for it's simplicity". I'm exploring all the existing work out there. Lua is certainly an option. From what I can see on the outside, is that lua embedding for rust is immature compared to some other solutions. But maybe that's because the lua-folks are just more honest :)
- leoedin 3y ago
  There's Rhai - https://rhai.rs/ https://rhai.rs/
- shzhdbi09gv8ioi 3y ago
  There's a bunch of scripting languages written in rust that targets "embed-in-rust", for example https://crates.io/crates/rhai https://crates.io/crates/rhai https://crates.io/crates/dyon https://crates.io/crates/dyon https://crates.io/crates/duckscript https://crates.io/crates/duckscript You can use the above links to browse crate "dependents", that is other crates using these dependencies if you are looking for example usage.
- tveita 3y ago
  Your requirements are very close to Starklark - an embeddable programming language intended for configuration with an available Rust implementation. https://docs.rs/crate/starlark/latest https://docs.rs/crate/starlark/latest
repelsteeltje 3y ago
Working on a significant C++ code base (shipped as deb, rpm and windows binaries) and wanting to allow some means to customize busyness logic we decided to integrate Python hooks, a couple of years ago. In hindsight, I'd call that decision a mixed blessing. The code level integration (pybind11) was nice and easy, the issues came later. We found operational problems (multithreading), performance (in particular initialization) and worst: memory leaks and crashes on python VM teardown. Another problem was that python has it's own particular ideas about package management than generally don't rhyme with packages from the OS vendor. I'm not sure what the angle is in this project, but I don't see a happy ending for projects that rely on both pip and cargo when it comes to maintenance and long-term support except for source-level integration. That is, unless there is some well designed and well thought out strategy to combine Rust and Python worlds that doesn't conflict with the RPM or apt view of doing things.
- galangalalgol 3y ago
  Rust bindings into the OS packages aren't nearly as common as in C++, so that could be a saving point. With MUSL builds there can easily be no shared dependencies at all, and with glibc builds it would just be glibc and a couple friends.
  - pipo234 3y ago
    Yes, C++ definitely tends to rely more on binary dependencies, shared objects. Rust will probably have to as well, once frameworks and libraries get to size and maturity that you want your OS vendor to take over responsibility (and compile time!). And note that even MUSL doesn't help create a package that works seamlessly on any Linux OS/release — maybe WASM will (or more accurately: WASI). But I think, the point GP is making is more about the dependencies Python ads to your C++ (or Rust). Ie.: build RustPython project to produce artifact that works on my machine, but relies on my python interpreter, my (which may have C dependencies) as well as a rust edition and cargo. How can I bundle this into a package for a different Linux? Preferably do it following the existing packages and lore of that distro, rather some python and some rustup version?
reedf1 3y ago
I think it's very naive for a programming language to make you "wince", tools are tools.
- wayvey 3y ago
  Agree, would like to know more context
  - spacecadet 3y ago
    Because you have all the power to change the tool or invent a new one.
- blurbleblurble 3y ago
  Have you ever used pip?
- amelius 3y ago
  Python is a great language to Just Get Things Done™. Rust is great if you like puzzles and want to spend your precious time solving the same memory management riddle over and over again. You do get faster and often more robust code though, although in practice the difference is often not meaningful and the extra time invested doesn't pay off.
  - aldanor 3y ago
    Sometimes, Things just can't be Done with Python. E.g., you're parsing dozens terabytes of data, or need response time within a microsecond. So, you're left with a choice of C++, Rust (what else, C#, possibly Zig and a few other). Rust stops being a "puzzle" once you've written enough of it and you just know how to do things. It has one major disadvantage though - returning back to write in languages like C++/Python makes you cringe because they don't have basic language tools like proper sum types or traits that get things done.
    - amelius 3y ago
      Turns out 99% of the time you don't need to parse terabytes or have μs-response times. Meaning you should be programming just about 99% of your time in Python. The Rust memory management puzzles are not really complicated. They just get in the way. > they don't have basic language tools like proper sum types or traits that get things done Sum types and traits are not getting things done.
deleted 3y ago
[deleted]
saltwatercowboy 3y ago
> That said, python makes me wince. O lord of the compiler, lend us thine wisdom.

avgcorrection 3y ago

Why does it make you wince?

mavamaarten 3y ago
From a non-python dev perspective: I always struggle with dependencies and versions. I have a script in front of me that I want to run, and am often just frustratingly brute-forcing commands to make it work. Do I: python? python3? pip install? pip3 install? python pip install? python pip3 install? python3 pip install? python3 pip3 install? And then everyone mentions "oh just use venv" or "conda" or docker or... It just never ends or it never clearly explains what to do. And all I want to do is run a relatively simple script. Or course I would use a docker image to run a more complex app. But it's frustratingly hard to manage for people who don't use it often. I'm from a java/Kotlin world where things aren't perfect either. But at least dependencies are clearly defined, and the only thing you need is Gradle that updates itself through a wrapper script if needed and pulls all dependencies. And a JVM/JDK which is also configured/installed quite clearly in a specific location.
- diarrhea 3y ago
  Yeah, it's definitely painful. Over the years, I've probably spent in excess of 100 hours on understanding pip, venv, PYTHONPATH, poetry/pdm, pipfile, pyenv, ... still barely understand how it all works together. https://peps.python.org/pep-0723/ https://peps.python.org/pep-0723/ might make life much simpler for single-script applications.
- FergusArgyll 3y ago
  for simple scripts, this is the easiest and works. python3 -m venv venv source venv/bin/activate pip install whatever echo "source path/to/venv/bin/activate && python3 path/to/app.py" > myscript mv myscript /usr/local/bin and just use myscript. I use this for a lot of little scripts
  - bomewish 3y ago
    Why not just Path/to/venv/bin/python path/to/app.py? Neat trick though, gonna adopt it!
  - ctippett 3y ago
    I haven't tried this myself, but you could probably save yourself the wrapper script and execute your script directly by adding a shebang pointing to the Python executable from your virtual environment. printf '%s\n%s\n' "#!path/to/venv/bin/python3" "$(cat path/to/app.py)" > /usr/local/bin/myscript
- cultofmetatron 3y ago
  I feel you. python itself has its warts but its mostly minor. The dependancy hell though.. god damn it confuses me every time. I don't understand why they can't get it together with one consistent dependency system with good ergonomics. ruby does it with gem. elixir handles things reasonably well with mix. rust does it REALLY well with cargo. hell even ocaml has opam. but python? it between conda, pip, pip3, virtualenv and who knows what else. its hard to make something that I knwo will just work on everything.
  - odyssey7 3y ago
    To highlight another dynamically typed, interpreted language ecosystem, npm has been providing a remarkably smooth and uniformly adopted service for the JavaScript community.
- VagabundoP 3y ago
  Rye[1] is an all in one manager for python projects. Including the python versions and virtualenv, pip etc etc... It separates tool deps from app deps. Its all configured through a pyproject.toml config file. Its still new but works well. I'm transitioning to it from an unholy mess of pyenv, pip installs and other manual hacks. If you're starting a new python project that is more than just a straightforward script I'd use Rye from the get go. [1]https://rye-up.com/ https://rye-up.com/
- ptx 3y ago
  Part of the problem is that this used to require third-party tools, which gave rise to lots of different tools, but nowadays everything you need is included in Python itself. The simplest way (in the sense of having the fewest components required) is this, using only built-in tools: $ python3 -m venv --upgrade-deps my-virtual-environment $ my-virtual-environment/bin/pip install whatever-third-party-package $ my-virtual-environment/bin/python3 my-script.py This creates an isolated virtual environment in "my-virtual-environment". Use the pip and python3 binaries inside that directory to install packages and run scripts. Done. (The "activate" step suggested in other comments is just a convenience for setting your PATH to refer to the virtual environment implicitly, so that's completely optional.) Note that Debian's python3 package leaves out some parts of the standard Python distribution, but you can install the python3-full package for a complete installation.
  - pletnes 3y ago
    This is a fairly good answer but on windows, there’s no python3, so that’s your first command being broken.
    - ptx 3y ago
      Right, on Windows the recipe needs a few alterations but is essentially the same. This should work: > py -3 -m venv --upgrade-deps my-virtual-environment > my-virtual-environment\Scripts\pip install whatever-third-party-package > my-virtual-environment\Scripts\python my-script.py This assumes that the Python launcher (py.exe) was installed when installing Python, which it is by default.
      - IshKebab 3y ago
        See this is the problem. You're already several "ah yes and you need to"s deep.
        ptx 3y ago
        Not really. It's the same 3-step process in both cases.
- weberer 3y ago
  Thankfully, most distros/OSs don't ship with Python 2 anymore, so `python` will just link to `python3`.
  - IshKebab 3y ago
    Nope. Debian doesn't ship a `python` at all, which I think is the sensible option. You should universally use `python3`. Except on Windows the official distribution is bonkers and has no `python3.exe`. Probably the easiest solution is to install it from the Microsoft store instead which does.
- klibertp 3y ago
  You seriously consider Gradle - a kitchen sink and a monstrosity scripted in obscure, extremely dynamic language - a better solution just because it has a... wrapper script?! With Gradle, where do you define dependencies? All with version numbers in the `dependencies` block? In a separate file, declaring a hash map of versions and then a hash map of dependencies, and then putting the latter into `ext`, which magically propagates to just about everywhere in the build silently? In a separate file using some plugin that tries to replicate the lockfile functionality baked into most other dependency systems? In a separate TOML file, using the Catalog? ...and that's just about dependencies, I could go on for days about other Gradle features... It's not any better than Python. You're just used to it. The same complexity is there in both Python and Kotlin, you just learned to cope with one better than the other. (Gradle with Kotlin DSL and good IDE support is a little better, mostly because Kotlin devs are not as afraid of touching the build as when it's done in Groovy. It's still 2 orders of magnitude more complex than whatever you use for Python - unless you use zc.buildout or something like that.)
postepowanieadm 3y ago
Installing python software is hard. Golang is much easier for and end user.
- vundercind 3y ago
  Go is what I reach for when I might otherwise write Python but need to actually, like, distribute my program.
lumb63 3y ago
Personally, I wince at Python because, while I view it as a quick-and-dirty scripting language, some people write production code in Python. And here we are ten years later, stuck with code that reads a few metrics from the system, packages them into JSON, and sends them to an API for monitoring. The code takes 7 seconds at 100% CPU to resolve the imports, every time it runs.
- wasmitnetzen 3y ago
  You can write bad code in any language, I fail to see how that's Python's problem.

berkes 3y ago

I was recently looking for some ways to make a rust project dynamically configurable. Yaml/toml/etc are too static or are terrible to describe logic in (yet we do it all the time for CI, infra etc, ugh). WASM would be an option, but overengineered for my case. Ruby (through Artichoke), or Python (through RustPython); but they come with the downside of introducing Ruby or Python. I haven't decided yet, but would prefer Lua for it's simplicity. Or JavaScript (a subset) for how easy it can be limited in scope. Are there any (example) rust programs out there that have embedded scripting, or runtime plugins or addons in a scripting language?

ddorian43 3y ago
why not lua?
- berkes 3y ago
  No particular reason. edit: to be clear, I did mention "I would prefer lua for it's simplicity". I'm exploring all the existing work out there. Lua is certainly an option. From what I can see on the outside, is that lua embedding for rust is immature compared to some other solutions. But maybe that's because the lua-folks are just more honest :)
leoedin 3y ago
There's Rhai - https://rhai.rs/ https://rhai.rs/
shzhdbi09gv8ioi 3y ago
There's a bunch of scripting languages written in rust that targets "embed-in-rust", for example https://crates.io/crates/rhai https://crates.io/crates/rhai https://crates.io/crates/dyon https://crates.io/crates/dyon https://crates.io/crates/duckscript https://crates.io/crates/duckscript You can use the above links to browse crate "dependents", that is other crates using these dependencies if you are looking for example usage.
tveita 3y ago
Your requirements are very close to Starklark - an embeddable programming language intended for configuration with an available Rust implementation. https://docs.rs/crate/starlark/latest https://docs.rs/crate/starlark/latest

repelsteeltje 3y ago

Working on a significant C++ code base (shipped as deb, rpm and windows binaries) and wanting to allow some means to customize busyness logic we decided to integrate Python hooks, a couple of years ago. In hindsight, I'd call that decision a mixed blessing. The code level integration (pybind11) was nice and easy, the issues came later. We found operational problems (multithreading), performance (in particular initialization) and worst: memory leaks and crashes on python VM teardown. Another problem was that python has it's own particular ideas about package management than generally don't rhyme with packages from the OS vendor. I'm not sure what the angle is in this project, but I don't see a happy ending for projects that rely on both pip and cargo when it comes to maintenance and long-term support except for source-level integration. That is, unless there is some well designed and well thought out strategy to combine Rust and Python worlds that doesn't conflict with the RPM or apt view of doing things.

galangalalgol 3y ago
Rust bindings into the OS packages aren't nearly as common as in C++, so that could be a saving point. With MUSL builds there can easily be no shared dependencies at all, and with glibc builds it would just be glibc and a couple friends.
- pipo234 3y ago
  Yes, C++ definitely tends to rely more on binary dependencies, shared objects. Rust will probably have to as well, once frameworks and libraries get to size and maturity that you want your OS vendor to take over responsibility (and compile time!). And note that even MUSL doesn't help create a package that works seamlessly on any Linux OS/release — maybe WASM will (or more accurately: WASI). But I think, the point GP is making is more about the dependencies Python ads to your C++ (or Rust). Ie.: build RustPython project to produce artifact that works on my machine, but relies on my python interpreter, my (which may have C dependencies) as well as a rust edition and cargo. How can I bundle this into a package for a different Linux? Preferably do it following the existing packages and lore of that distro, rather some python and some rustup version?

reedf1 3y ago

I think it's very naive for a programming language to make you "wince", tools are tools.

wayvey 3y ago
Agree, would like to know more context
- spacecadet 3y ago
  Because you have all the power to change the tool or invent a new one.
blurbleblurble 3y ago
Have you ever used pip?
amelius 3y ago
Python is a great language to Just Get Things Done™. Rust is great if you like puzzles and want to spend your precious time solving the same memory management riddle over and over again. You do get faster and often more robust code though, although in practice the difference is often not meaningful and the extra time invested doesn't pay off.
- aldanor 3y ago
  Sometimes, Things just can't be Done with Python. E.g., you're parsing dozens terabytes of data, or need response time within a microsecond. So, you're left with a choice of C++, Rust (what else, C#, possibly Zig and a few other). Rust stops being a "puzzle" once you've written enough of it and you just know how to do things. It has one major disadvantage though - returning back to write in languages like C++/Python makes you cringe because they don't have basic language tools like proper sum types or traits that get things done.
  - amelius 3y ago
    Turns out 99% of the time you don't need to parse terabytes or have μs-response times. Meaning you should be programming just about 99% of your time in Python. The Rust memory management puzzles are not really complicated. They just get in the way. > they don't have basic language tools like proper sum types or traits that get things done Sum types and traits are not getting things done.

deleted 3y ago

[deleted]

saltwatercowboy 3y ago

> That said, python makes me wince. O lord of the compiler, lend us thine wisdom.