5 ms·

I guess that depends from your perspective. I'm not a Python developer, but like many people I do want to run Python programs from time to time. I don't really

by arp242 2y ago

I guess that depends from your perspective. I'm not a Python developer, but like many people I do want to run Python programs from time to time. I don't really know Rust, or Cargo, but I never have trouble building any Rust program: "cargo build [--release]" is all I need to know. Easy. Even many C programs are actually quite easy: "./configure", "make", and optionally "make install". "./configure" has a nice "--help". There is a lot to be said about the ugly generated autotools soup, but the UX for people just wanting to build/run it without in-depth knowledge of the system is actually quite decent. cmake is a regression here. With Python, "pip install" gives me an entire screen full of errors about venv and "externally managed" and whatnot. I don't care. I just want to run it. I don't want a bunch of venvs, I just want to install or run the damn program. I've taken to just use "pip install --break-system-packages", which installs to ~/.local. It works shrug. Last time I wanted to just run a project with a few small modifications I had a hard time. I ended up just editing ~/.local/lib/python/[...] Again, it worked so whatever. All of this is really where Python and some other languages/build systems fail. Many people running this are not $language_x programmers or experts, and I don't want to read up on every system I come across. That's not a reasonable demand. Any system that doesn't allow non-users of that language to use it in simple easy steps needs work. Python's system is one such system.

simonw 2y ago
"I don't want a bunch of venvs" That's your problem right there. Virtual environments are the Python ecosystem's solution to the problem of wanting to install different things on the same machine that have different conflicting requirements. If you refuse to use virtual environments and you install more than one separate Python project you're going to run into conflicting requirements and it's going to suck. Have you tried pipx? If you're just installing Python tools (and not hacking on them yourself) it's fantastic - it manages separate virtual environments for each of your installations without you having to think about them (or even know what a virtual environment is).
- arp242 2y ago
  Managing a farm of virtualenvs and mucking about with my PATH doesn't address the user-installable problem at all. And it seems there's a new tool to try every few months that really will fix all problems this time. And maybe if you're a Python developer working on the code every day that's all brilliant. But most people aren't Python developers, and I just want to try that "Show HN" project or whatnot. Give me a single command I can run. Always. For any project. And that always works. If you don't have that then your build system needs work.
  - simonw 2y ago
    "Give me a single command I can run. Always. For any project. And that always works." pipx install X
    - arp242 2y ago
      Right so; I'll try that next time. Thanks. I just go by the very prominent "pip install X" on every pypi page (as well as "pip install .." in many READMEs).
      - simonw 2y ago
        Yeah, totally understand that - pipx is still pretty poorly known by people who are active in Python development! A few of my READMEs start like this: https://github.com/simonw/paginate-json?tab=readme-ov-file#installation https://github.com/simonw/paginate-json?tab=readme-ov-file#i... ## Installation pip install paginate-json Or use pipx (link to pipx site) pipx install paginate-json But I checked and actually most them still don't even mention it. I'll be fixing that in the future.
        arp242 2y ago
        Out of the 3 things I want to install 2 don't work. Both of these seem bugs in pipx so I reported one, but the feedback was borderline hostile and it ended up being closed with "unclear what you want". I'm not even going to bother reporting the other bug. So whatever the goals are, it doesn't really work. And in general pipx does not strike me as a serious project.
- guhidalg 2y ago
  Normal users who just want to run some code shouldn't need to learn why they need a venv or any of its alternatives. Normal users just want to download a package and run some code without having to think about interfering with other packages. Many programming languages package managers give them that UX and you can't blame them for expecting that from Python. The added step of having to think about venvs with Python is not good. It is a non-trivial system that every single Python user is forced to learn, understand, and the continually remember every time they switch from one project to another.
  - nine_k 2y ago
    This is correct. The whole application installation process, including the creation of a venv, installing stuff into it, and registering it with some on-PATH launcher should be one command. BTW pyenv comes relatively close.
  - simonw 2y ago
    I agree with that. Until we solve that larger problem, people need to learn to use virtual environments, or at least learn to install Python tools using pipx.
  - VagabundoP 2y ago
    sudo apt install pipx pipx install package_name Takes care of the venv and the script/app path is added to system path.
- kallapy 2y ago
  I reject the virtual environments and have no issues. On an untrusted machine (see e.g. the recent token leak): /a/bin/python3 -m pip install foo /b/bin/python3 -m pip install bar The whole venv thing is overblown but a fertile source for blogs and discussions. If C-extensions link to installed libraries in site-packages, of course they should use RPATH.
- imtringued 2y ago
  Pythons venvs are a problem to the solution of solving the dependency problem. Consider the following: it is not possible to relocate venvs. In what universe does this make sense? Consider a C++ or Rust binary that would only run when it is placed in /home/simonw/.
vhcr 2y ago
Do you have a problem with Node.js too because it creates a node_modules folder, or is the problem that it is not handled automatically?
- arp242 2y ago
  I don't care about the internals. I care about "just" being able to run it. I find that most JS projects work fairly well: "npm install" maybe followed by "npm run build" or the like. This isn't enforced by npm and I don't think npm is perfect here, but practical speaking as a non-JS dev just wanting to run some JS projects: it works fairly well for almost all JS projects I've wanted to run in the last five years or so. A "run_me.py" that would *Just Work™" is fine. I don't overly care what it does internally as long as it's not hugely slow or depends on anything other than "python". Ideally this should be consistent throughout the ecosystem. To be honest I can't imagine shipping any project intended to be run by users and not have a simple, fool-proof, and low-effort way of running it by anyone of any skill level, which doesn't depend on any real knowledge of the language.
  - sgarland 2y ago
    > To be honest I can't imagine shipping any project intended to be run by users and not have a simple, fool-proof, and low-effort way of running it by anyone of any skill level, which doesn't depend on any real knowledge of the language. This is how we got GH Issues full of inane comments, and blogs from mediocre devs recommending things they know nothing about. I see nothing wrong with not catering to the lowest common denominator.
    - arp242 2y ago
      Like people with actual lives to live and useful stuff to do that's not learning about and hand-holding a dozen different half-baked build systems. But sure, keep up the cynical illusion that everyone is an idiot if that's what you need to go through life.
      - sgarland 2y ago
sgarland 2y ago
This is mostly a curse of Python’s popularity. The reason you can’t pip install with system Python is that it can break things, and when your system is relying on Python to run various tools, that can’t be allowed. No one (sane) is building OS-level scripts with Node. The simplest answer, IMO, is to download the Python source code, build it, and then run make altinstall. It’ll install in parallel with system Python, and you can then alias the new executable path so you no longer have to think about it. Assuming you already have gcc’s tool chain installed, it takes roughly 10-15 minutes to build. Not a big deal.
Galanwe 2y ago
Maybe I am biased, because I learned these things so long ago and I don't realize that it's a pain to learn. But what exactly is so confusing about virtualenvs ? They really not that different from any other packaging system like JS or Rust. The only difference is instead of relying on your current directory to find the the libraries / binaries (and thus requiring you to wrap binaries call with some wrapper to search in a specific path), they rely on you sourcing an `activate` script. That's really just it. Create a Virtualenv: $ virtualenv myenv Activate it, now it is added to your $PATH: $ . myenv/bin/activate There really is nothing more in the normal case. If you don't want to have to remember it, create a global Virtualenv somewhere, source it's activate in your .bashrc, and forget it ever existed.
- imtringued 2y ago
  Only python demands you to source an activation script before doing anything.
  - Galanwe 2y ago
    Yes, though just to illustrate that it's a matter of taste, I do prefer the solution of virtualenvs requiring to source a script that append to PATH, rather than a solution requiring the use of a wrapper that executes in its PATH. I never remember how to run Javascript binaries. Is it npm run ? npm run script ? npx ? I always end up running the links in node_modules/bin
VagabundoP 2y ago
Its more probable that you are trying to install the deps in the system python. And using pip instal xxxxx -u will install them in your user directory rather than the system. I'm pretty sure modern Ubuntu warns you against doing that now anyway. If you're installing for a small script then doing python -m venv little_project in you home dir is straightforward, just active it after [1] I'm using rye[2] now and its very similar to Rust's Cargo, it wraps a bunch of the standard toolchain and manages standalone python versions in the background, so doesn't fall into the trap of linux system python issues. [1]https://docs.python.org/3/library/venv.html https://docs.python.org/3/library/venv.html [2]https://rye.astral.sh/ https://rye.astral.sh/

by arp242 2y ago

simonw 2y ago

"I don't want a bunch of venvs" That's your problem right there. Virtual environments are the Python ecosystem's solution to the problem of wanting to install different things on the same machine that have different conflicting requirements. If you refuse to use virtual environments and you install more than one separate Python project you're going to run into conflicting requirements and it's going to suck. Have you tried pipx? If you're just installing Python tools (and not hacking on them yourself) it's fantastic - it manages separate virtual environments for each of your installations without you having to think about them (or even know what a virtual environment is).

arp242 2y ago
Managing a farm of virtualenvs and mucking about with my PATH doesn't address the user-installable problem at all. And it seems there's a new tool to try every few months that really will fix all problems this time. And maybe if you're a Python developer working on the code every day that's all brilliant. But most people aren't Python developers, and I just want to try that "Show HN" project or whatnot. Give me a single command I can run. Always. For any project. And that always works. If you don't have that then your build system needs work.
- simonw 2y ago
  "Give me a single command I can run. Always. For any project. And that always works." pipx install X
  - arp242 2y ago
    Right so; I'll try that next time. Thanks. I just go by the very prominent "pip install X" on every pypi page (as well as "pip install .." in many READMEs).
    - simonw 2y ago
      Yeah, totally understand that - pipx is still pretty poorly known by people who are active in Python development! A few of my READMEs start like this: https://github.com/simonw/paginate-json?tab=readme-ov-file#installation https://github.com/simonw/paginate-json?tab=readme-ov-file#i... ## Installation pip install paginate-json Or use pipx (link to pipx site) pipx install paginate-json But I checked and actually most them still don't even mention it. I'll be fixing that in the future.
      - arp242 2y ago
        Out of the 3 things I want to install 2 don't work. Both of these seem bugs in pipx so I reported one, but the feedback was borderline hostile and it ended up being closed with "unclear what you want". I'm not even going to bother reporting the other bug. So whatever the goals are, it doesn't really work. And in general pipx does not strike me as a serious project.
guhidalg 2y ago
Normal users who just want to run some code shouldn't need to learn why they need a venv or any of its alternatives. Normal users just want to download a package and run some code without having to think about interfering with other packages. Many programming languages package managers give them that UX and you can't blame them for expecting that from Python. The added step of having to think about venvs with Python is not good. It is a non-trivial system that every single Python user is forced to learn, understand, and the continually remember every time they switch from one project to another.
- nine_k 2y ago
  This is correct. The whole application installation process, including the creation of a venv, installing stuff into it, and registering it with some on-PATH launcher should be one command. BTW pyenv comes relatively close.
- simonw 2y ago
  I agree with that. Until we solve that larger problem, people need to learn to use virtual environments, or at least learn to install Python tools using pipx.
- VagabundoP 2y ago
  sudo apt install pipx pipx install package_name Takes care of the venv and the script/app path is added to system path.
kallapy 2y ago
I reject the virtual environments and have no issues. On an untrusted machine (see e.g. the recent token leak): /a/bin/python3 -m pip install foo /b/bin/python3 -m pip install bar The whole venv thing is overblown but a fertile source for blogs and discussions. If C-extensions link to installed libraries in site-packages, of course they should use RPATH.
imtringued 2y ago
Pythons venvs are a problem to the solution of solving the dependency problem. Consider the following: it is not possible to relocate venvs. In what universe does this make sense? Consider a C++ or Rust binary that would only run when it is placed in /home/simonw/.

vhcr 2y ago

Do you have a problem with Node.js too because it creates a node_modules folder, or is the problem that it is not handled automatically?

arp242 2y ago
I don't care about the internals. I care about "just" being able to run it. I find that most JS projects work fairly well: "npm install" maybe followed by "npm run build" or the like. This isn't enforced by npm and I don't think npm is perfect here, but practical speaking as a non-JS dev just wanting to run some JS projects: it works fairly well for almost all JS projects I've wanted to run in the last five years or so. A "run_me.py" that would *Just Work™" is fine. I don't overly care what it does internally as long as it's not hugely slow or depends on anything other than "python". Ideally this should be consistent throughout the ecosystem. To be honest I can't imagine shipping any project intended to be run by users and not have a simple, fool-proof, and low-effort way of running it by anyone of any skill level, which doesn't depend on any real knowledge of the language.
- sgarland 2y ago
  > To be honest I can't imagine shipping any project intended to be run by users and not have a simple, fool-proof, and low-effort way of running it by anyone of any skill level, which doesn't depend on any real knowledge of the language. This is how we got GH Issues full of inane comments, and blogs from mediocre devs recommending things they know nothing about. I see nothing wrong with not catering to the lowest common denominator.
  - arp242 2y ago
    Like people with actual lives to live and useful stuff to do that's not learning about and hand-holding a dozen different half-baked build systems. But sure, keep up the cynical illusion that everyone is an idiot if that's what you need to go through life.
    - sgarland 2y ago

sgarland 2y ago

This is mostly a curse of Python’s popularity. The reason you can’t pip install with system Python is that it can break things, and when your system is relying on Python to run various tools, that can’t be allowed. No one (sane) is building OS-level scripts with Node. The simplest answer, IMO, is to download the Python source code, build it, and then run make altinstall. It’ll install in parallel with system Python, and you can then alias the new executable path so you no longer have to think about it. Assuming you already have gcc’s tool chain installed, it takes roughly 10-15 minutes to build. Not a big deal.

Galanwe 2y ago

Maybe I am biased, because I learned these things so long ago and I don't realize that it's a pain to learn. But what exactly is so confusing about virtualenvs ? They really not that different from any other packaging system like JS or Rust. The only difference is instead of relying on your current directory to find the the libraries / binaries (and thus requiring you to wrap binaries call with some wrapper to search in a specific path), they rely on you sourcing an `activate` script. That's really just it. Create a Virtualenv: $ virtualenv myenv Activate it, now it is added to your $PATH: $ . myenv/bin/activate There really is nothing more in the normal case. If you don't want to have to remember it, create a global Virtualenv somewhere, source it's activate in your .bashrc, and forget it ever existed.

imtringued 2y ago
Only python demands you to source an activation script before doing anything.
- Galanwe 2y ago
  Yes, though just to illustrate that it's a matter of taste, I do prefer the solution of virtualenvs requiring to source a script that append to PATH, rather than a solution requiring the use of a wrapper that executes in its PATH. I never remember how to run Javascript binaries. Is it npm run ? npm run script ? npx ? I always end up running the links in node_modules/bin

VagabundoP 2y ago

Its more probable that you are trying to install the deps in the system python. And using pip instal xxxxx -u will install them in your user directory rather than the system. I'm pretty sure modern Ubuntu warns you against doing that now anyway. If you're installing for a small script then doing python -m venv little_project in you home dir is straightforward, just active it after [1] I'm using rye[2] now and its very similar to Rust's Cargo, it wraps a bunch of the standard toolchain and manages standalone python versions in the background, so doesn't fall into the trap of linux system python issues. [1]https://docs.python.org/3/library/venv.html https://docs.python.org/3/library/venv.html [2]https://rye.astral.sh/ https://rye.astral.sh/