2 ms·

> Instead of creating virtual environments using venv, you can use a more sophisticated tool like Poetry which enables you to manage you environments in a much

> Instead of creating virtual environments using venv, you can use a more sophisticated tool like Poetry which enables you to manage you environments in a much better manner. It seems pretty emblematic of the shitshow that is Python environment management that the top-rated comment is recommending something I've never heard of. I know pyenv and pyvenv (not the same thing!) and virtualenv and venv (not the same thing!) and pipenv, but apparently those are all now out of date and I should be moving to this sixth new tool. I'm sure this time it will be fixed for good.

adamckay 3y ago
Poetry is an all-in-one tool that does dependency management using `pyproject.toml` files and its own `poetry.lock` files, as well as package building, virtualenv management, and also has hooks for entrypoints and scripts. It's rather a useful tool and I'm personally using it for dependency management and packing for all my projects moving forwards, though for venvs I'm using `pyenv` and the `pyenv-virtualenv` plugin. Once you figure out your workflow it can be quite nice, but it's figuring it out that's a huge mess in Python at the minute. Hopefully PEP 582 (Python local packages directory) solves it a bit...
- kvnhn 3y ago
  PEP 582 was rejected, FYI. https://peps.python.org/pep-0582/ https://peps.python.org/pep-0582/
  - adamckay 3y ago
    Bugger. Thanks for letting me know (embarrassingly I did load up the PEP page to make sure I remembered the right number but I didn't check its status). Was hoping that it would make things simpler for smaller projects and newbie developers but the rejection reason is solid.
polski-g 3y ago
Actually PDM is the best python package manager tool!
- just_boost_it 3y ago
  I was going to say that too, but then I was like "yep, that's exactly the problem!"
paulddraper 3y ago
Yes, but unironically
zwaps 3y ago
Funnily enough, Poetry only just very recently started to properly support PyTorch (you know, which seemingly 90% of Python usecases nowadays require). Like, it would not work across different setups with Arm, Cpu and Gpu machines, which is the point of poetry. For years, youd need to use one of a few hacks that had serious downsides. Our deployments still install pytorch stuff separately in every container. I think this is still one of the top (open) issues on the repo. Also, for sufficiently sophisticated projects, poetry was known to lead to hour long dependency resolving sessions. For example, if you feel like using different pypy sources (eg internal registry, plus some wheels etc), poetry will try to install every package from every of these source registry meaning creating a lock file takes now hours. Not sure if this was ever resolved, but at my company, poetry is know as the thing that never works.
- slau 3y ago
  I’ve created about 50 Python projects in the last 3 years that are in production, not a single one uses PyTorch.

adamckay 3y ago

Poetry is an all-in-one tool that does dependency management using `pyproject.toml` files and its own `poetry.lock` files, as well as package building, virtualenv management, and also has hooks for entrypoints and scripts. It's rather a useful tool and I'm personally using it for dependency management and packing for all my projects moving forwards, though for venvs I'm using `pyenv` and the `pyenv-virtualenv` plugin. Once you figure out your workflow it can be quite nice, but it's figuring it out that's a huge mess in Python at the minute. Hopefully PEP 582 (Python local packages directory) solves it a bit...

kvnhn 3y ago
PEP 582 was rejected, FYI. https://peps.python.org/pep-0582/ https://peps.python.org/pep-0582/
- adamckay 3y ago
  Bugger. Thanks for letting me know (embarrassingly I did load up the PEP page to make sure I remembered the right number but I didn't check its status). Was hoping that it would make things simpler for smaller projects and newbie developers but the rejection reason is solid.

polski-g 3y ago

Actually PDM is the best python package manager tool!

just_boost_it 3y ago
I was going to say that too, but then I was like "yep, that's exactly the problem!"

paulddraper 3y ago

Yes, but unironically

zwaps 3y ago

Funnily enough, Poetry only just very recently started to properly support PyTorch (you know, which seemingly 90% of Python usecases nowadays require). Like, it would not work across different setups with Arm, Cpu and Gpu machines, which is the point of poetry. For years, youd need to use one of a few hacks that had serious downsides. Our deployments still install pytorch stuff separately in every container. I think this is still one of the top (open) issues on the repo. Also, for sufficiently sophisticated projects, poetry was known to lead to hour long dependency resolving sessions. For example, if you feel like using different pypy sources (eg internal registry, plus some wheels etc), poetry will try to install every package from every of these source registry meaning creating a lock file takes now hours. Not sure if this was ever resolved, but at my company, poetry is know as the thing that never works.

slau 3y ago
I’ve created about 50 Python projects in the last 3 years that are in production, not a single one uses PyTorch.