3 ms·

I do this all the time for python virtualenvs. Instead of calling ‘#!/usr/bin/env python3’ I will call ‘#!venv/bin/python3’. This way I don’t have to worry abou

by mbreese 11mo ago

I do this all the time for python virtualenvs. Instead of calling ‘#!/usr/bin/env python3’ I will call ‘#!venv/bin/python3’. This way I don’t have to worry about whether the environment was activated or not.

zahlman 11mo ago
If you're okay with running the script only from the parent directory of the venv (I guess you have things set up so that's the project root), fine. You never have to "worry about whether the environment was activated", unless your code depends on those environment variables (in which case your shebang trick won't help you). Just specify the path to the venv's Python executable. You aren't really intended to put your own scripts in the venv's bin/ directly, although of course you can. An installer will create them for you, from the entry points defined in pyproject.toml. (This is one of the few useful things that an "editable install" can accomplish; I'm generally fairly negative on that model, however.) If you have something installed in a venv and want to run it regardless of CWD, you can symlink the wrapper script from somewhere on your PATH. (That's what Pipx does.)
- mbreese 11mo ago
  I use venv's to keep library installs out of the global space. My python scripts won't run without those libraries, so running from the project root is pretty much required. I don't use the variables set by activating the venv, so that's not a real concern. I've done this for years to keep my individual projects separate and not changing activations when switching directories. I also make sure to only call `venv/bin/pip` or `venv/bin/python3` when I'm directly calling pip or python. So, yes -- you have to be in the root project directory for this to work, but for me, that's a useful tradeoff. Even when running code from within a docker container, I still use this method, so I make sure that I'm executing from the proper work directory. If I think that I need to run a program (without arguments), I'll have a short shell wrapper that is essentially: #!/bin/bash cd $(dirname $0) venv/bin/python3 myscript.py As far as running a program that's managed by venv/pip, symlinks are essentially what I do. I'll create a new venv for each program, install it in that venv, and then symlink from venv/bin/program to $HOME/.local/bin/. It works very well when you're installing a pip managed program.
  - hebelehubele 11mo ago
    I also use the trick to insert new lookup paths. project_root = os.path.dirname(os.path.abspath(__file__)) sys.path.insert(0, project_root)
inetknght 11mo ago
This is terrible. I hope you're not adding a whole venv to version control. What if the user doesn't have a venv created? What if they created it in a different directory? What if they created a second venv and want to use that instead? What if the user uses `.venv` instead of `venv`? `#!/usr/bin/env python3` solves most of that.
- mbreese 11mo ago
  No! These are programs that are largely meant to have a run.sh or install.sh script run before the main script. If the venv doesn’t exist, the it is created there and requirements installed. The main point is that I’m trading away some flexibility to keep my ENV clean. When I submit jobs on HPC clusters, keeping my environment clean tends to make things easier to troubleshoot. If I’m switching between different programs or commonly piping data between two different programs with their own venvs, it can be easier to just run the associated python binary directly, rather than have to manage different venv environment activations.
  - inetknght 11mo ago
    You can have your cake (use multiple venvs) and eat it (flexibility) too. `source venv/bin/activate` from your .sh files will cause `/usr/bin/env python3` to use the python3 located in your venv. Switching between venvs is easy too. just call `deactivate` when one venv is activated. It drops you out of the venv. You can then cleanly `source venv2/bin/activate`.
    - mbreese 11mo ago
      It feels like you're telling me that I'm holding my phone wrong. I don't like sourcing things into my environment. I've worked this way for years. I think the idea of 'activating' and 'deactivating' an environment is an anti-pattern. But I also work on HPC clusters where all of the configuration about paths is handled by the environment. Because of this, I've learned the hard way that for my workflows, it's far too easy to have the wrong environment loaded with venvs and modules that it's often better to keep things explicitly defined. I don't like magic, so I explicitly state which venv my code (or occasionally other people's code) is loading from. I sometimes will have to run multiple programs (that have their own venvs) and pipe data between them. If I have to source and deactivate different venvs for each tool, it just doesn't work right. I think that's part of the power of virtualenv as a tool -- it's flexible in how it works. I can still use my explicit workflows with the same tooling as everyone else, and you can source your environments and keep happily coding along. For me, that's why I keep using them...

by mbreese 11mo ago

zahlman 11mo ago

If you're okay with running the script only from the parent directory of the venv (I guess you have things set up so that's the project root), fine. You never have to "worry about whether the environment was activated", unless your code depends on those environment variables (in which case your shebang trick won't help you). Just specify the path to the venv's Python executable. You aren't really intended to put your own scripts in the venv's bin/ directly, although of course you can. An installer will create them for you, from the entry points defined in pyproject.toml. (This is one of the few useful things that an "editable install" can accomplish; I'm generally fairly negative on that model, however.) If you have something installed in a venv and want to run it regardless of CWD, you can symlink the wrapper script from somewhere on your PATH. (That's what Pipx does.)

mbreese 11mo ago
I use venv's to keep library installs out of the global space. My python scripts won't run without those libraries, so running from the project root is pretty much required. I don't use the variables set by activating the venv, so that's not a real concern. I've done this for years to keep my individual projects separate and not changing activations when switching directories. I also make sure to only call `venv/bin/pip` or `venv/bin/python3` when I'm directly calling pip or python. So, yes -- you have to be in the root project directory for this to work, but for me, that's a useful tradeoff. Even when running code from within a docker container, I still use this method, so I make sure that I'm executing from the proper work directory. If I think that I need to run a program (without arguments), I'll have a short shell wrapper that is essentially: #!/bin/bash cd $(dirname $0) venv/bin/python3 myscript.py As far as running a program that's managed by venv/pip, symlinks are essentially what I do. I'll create a new venv for each program, install it in that venv, and then symlink from venv/bin/program to $HOME/.local/bin/. It works very well when you're installing a pip managed program.
- hebelehubele 11mo ago
  I also use the trick to insert new lookup paths. project_root = os.path.dirname(os.path.abspath(__file__)) sys.path.insert(0, project_root)

inetknght 11mo ago

This is terrible. I hope you're not adding a whole venv to version control. What if the user doesn't have a venv created? What if they created it in a different directory? What if they created a second venv and want to use that instead? What if the user uses `.venv` instead of `venv`? `#!/usr/bin/env python3` solves most of that.

mbreese 11mo ago
No! These are programs that are largely meant to have a run.sh or install.sh script run before the main script. If the venv doesn’t exist, the it is created there and requirements installed. The main point is that I’m trading away some flexibility to keep my ENV clean. When I submit jobs on HPC clusters, keeping my environment clean tends to make things easier to troubleshoot. If I’m switching between different programs or commonly piping data between two different programs with their own venvs, it can be easier to just run the associated python binary directly, rather than have to manage different venv environment activations.
- inetknght 11mo ago
  You can have your cake (use multiple venvs) and eat it (flexibility) too. `source venv/bin/activate` from your .sh files will cause `/usr/bin/env python3` to use the python3 located in your venv. Switching between venvs is easy too. just call `deactivate` when one venv is activated. It drops you out of the venv. You can then cleanly `source venv2/bin/activate`.
  - mbreese 11mo ago
    It feels like you're telling me that I'm holding my phone wrong. I don't like sourcing things into my environment. I've worked this way for years. I think the idea of 'activating' and 'deactivating' an environment is an anti-pattern. But I also work on HPC clusters where all of the configuration about paths is handled by the environment. Because of this, I've learned the hard way that for my workflows, it's far too easy to have the wrong environment loaded with venvs and modules that it's often better to keep things explicitly defined. I don't like magic, so I explicitly state which venv my code (or occasionally other people's code) is loading from. I sometimes will have to run multiple programs (that have their own venvs) and pipe data between them. If I have to source and deactivate different venvs for each tool, it just doesn't work right. I think that's part of the power of virtualenv as a tool -- it's flexible in how it works. I can still use my explicit workflows with the same tooling as everyone else, and you can source your environments and keep happily coding along. For me, that's why I keep using them...