4 ms·

Please consider consolidating python dependency management instead of fragmenting it: https://github.com/mamba-org/mamba https://github.com/mamba-org/mamba In

Please consider consolidating python dependency management instead of fragmenting it: https://github.com/mamba-org/mamba https://github.com/mamba-org/mamba In other words, bring the thinking here. Whether it's new thinking, or decade old thinking, it's well time to converge. We've had decades of trying it like Heinz catsup varieties. Worth trying more wood behind fewer arrows.

kstrauser 2y ago
While I don't know which (if any) Python package manager will "win", I'm confident it won't be Conda or similar. I hope and pray not, anyway.
- matrss 2y ago
  Yes, the entire workflow of conda/mamba is a liability. Having environments detached from directories encourages bad practices (reusing environments for unrelated things, installing everything into the same environment and getting very broken dependency resolutions, that stuff) and it doesn't even have a lock file format, so environments can't be saved short of dumping its entire directory. Iff it must be the conda ecosystem, pixi (https://pixi.sh/latest/ https://pixi.sh/latest/) is a much better pick. But conda-style packages (or anything with install time dependency resolution really) also have the issue of being completely untestable. That makes them unsuitable at least for user-facing package management, if you care about your packages not randomly breaking without warning. I'd rather see every language converge on a single package manager that implements functional package management, i.e. guix or nix. One can dream...
  - skeledrew 2y ago
    I've been using a conda+poetry flow for years and it's worked very well. Particularly because envs aren't tied to projects. I tried pixi for a few days in a project months ago and it was just breakage and untenable limitations all around. I prefer the freedom of using per project envs when I want, and shared envs other times.
    - matrss 2y ago
      What are you using each of them for in that workflow? I've found that there really only are two kinds of packages I want to install: those I want globally (e.g. CLI tools like git, git-annex, DataLad, whatever shell enhancing you want, etc.) and project-specific dependencies (anything required to get going with it on a freshly installed system with the smallest amount of assumptions possible). The former is sufficiently addressed by `pixi global` and other package managers, the latter by pixi projects. Notably, conda environments are a bad fit for both (not global, not really updatable, not automatically tied to a project, not rebuildable due to missing lock files, ...).
      - thejevans 2y ago
        Conda is nice for those of us stuck in government or similar roles with really locked down systems. Without conda, I'd be stuck with a global Python 3.9 and without the ability to install the non-python dependencies like ssl required for various libraries or to run jupyter locally.
        kstrauser 2y ago
        Do they provide or permit Conda, but not other tooling?
        matrss 2y ago
        Pixi is installable under the same conditions and uses the same repositories as conda, so unless you are specifically allowed to use conda and forbidden to use anything else this isn't an argument for conda.
      - skeledrew 2y ago
        Conda for env management, particularly because it also allows for non-Python dependencies (one of my recent projects involved a browser automation solution, and I chose to install Firefox in a conda env as Ubuntu now only has a transitional package that points to a snap for it, and the snap install causes a lot of problems from an automation angle); I also have a few shared envs, for quick tests and general utils (I also use pipx, but it's restricted to packages with a CLI interface). Poetry for project management.
notatallshaw 2y ago
> Please consider consolidating python dependency management instead of fragmenting it: https://github.com/mamba-org/mamba https://github.com/mamba-org/mamba Mamba doesn't even interact with the official python packing ecosystem... It is purely a conda replacement and conda packaging is not Python packaging (it's a more fundamental fragmentation than choosing a particular tool). So weird choice to not fragment Python dependency management. If you depend on both conda and standard Python packaging (e.g. PyPI.org) you can use Pixi: https://github.com/prefix-dev/pixi?tab=readme-ov-file#pixi-package-management-made-easy https://github.com/prefix-dev/pixi?tab=readme-ov-file#pixi-p.... Pixi attempts to bridge the conda world and the Python package world, so it's possible to rely more deeply on standards based packaging but still use conda tooling when you need it.

kstrauser 2y ago

While I don't know which (if any) Python package manager will "win", I'm confident it won't be Conda or similar. I hope and pray not, anyway.

matrss 2y ago
Yes, the entire workflow of conda/mamba is a liability. Having environments detached from directories encourages bad practices (reusing environments for unrelated things, installing everything into the same environment and getting very broken dependency resolutions, that stuff) and it doesn't even have a lock file format, so environments can't be saved short of dumping its entire directory. Iff it must be the conda ecosystem, pixi (https://pixi.sh/latest/ https://pixi.sh/latest/) is a much better pick. But conda-style packages (or anything with install time dependency resolution really) also have the issue of being completely untestable. That makes them unsuitable at least for user-facing package management, if you care about your packages not randomly breaking without warning. I'd rather see every language converge on a single package manager that implements functional package management, i.e. guix or nix. One can dream...
- skeledrew 2y ago
  I've been using a conda+poetry flow for years and it's worked very well. Particularly because envs aren't tied to projects. I tried pixi for a few days in a project months ago and it was just breakage and untenable limitations all around. I prefer the freedom of using per project envs when I want, and shared envs other times.
  - matrss 2y ago
    What are you using each of them for in that workflow? I've found that there really only are two kinds of packages I want to install: those I want globally (e.g. CLI tools like git, git-annex, DataLad, whatever shell enhancing you want, etc.) and project-specific dependencies (anything required to get going with it on a freshly installed system with the smallest amount of assumptions possible). The former is sufficiently addressed by `pixi global` and other package managers, the latter by pixi projects. Notably, conda environments are a bad fit for both (not global, not really updatable, not automatically tied to a project, not rebuildable due to missing lock files, ...).
    - thejevans 2y ago
      Conda is nice for those of us stuck in government or similar roles with really locked down systems. Without conda, I'd be stuck with a global Python 3.9 and without the ability to install the non-python dependencies like ssl required for various libraries or to run jupyter locally.
      - kstrauser 2y ago
        Do they provide or permit Conda, but not other tooling?
      - matrss 2y ago
        Pixi is installable under the same conditions and uses the same repositories as conda, so unless you are specifically allowed to use conda and forbidden to use anything else this isn't an argument for conda.
    - skeledrew 2y ago
      Conda for env management, particularly because it also allows for non-Python dependencies (one of my recent projects involved a browser automation solution, and I chose to install Firefox in a conda env as Ubuntu now only has a transitional package that points to a snap for it, and the snap install causes a lot of problems from an automation angle); I also have a few shared envs, for quick tests and general utils (I also use pipx, but it's restricted to packages with a CLI interface). Poetry for project management.

notatallshaw 2y ago

> Please consider consolidating python dependency management instead of fragmenting it: https://github.com/mamba-org/mamba https://github.com/mamba-org/mamba Mamba doesn't even interact with the official python packing ecosystem... It is purely a conda replacement and conda packaging is not Python packaging (it's a more fundamental fragmentation than choosing a particular tool). So weird choice to not fragment Python dependency management. If you depend on both conda and standard Python packaging (e.g. PyPI.org) you can use Pixi: https://github.com/prefix-dev/pixi?tab=readme-ov-file#pixi-package-management-made-easy https://github.com/prefix-dev/pixi?tab=readme-ov-file#pixi-p.... Pixi attempts to bridge the conda world and the Python package world, so it's possible to rely more deeply on standards based packaging but still use conda tooling when you need it.