4 ms·

Yes, it is nice to see the python 3 migration is pretty much done, it really could have happened literally a decade ago! While we're all here: which is the pac

Yes, it is nice to see the python 3 migration is pretty much done, it really could have happened literally a decade ago! While we're all here: which is the package manager to use?

anttihaapala 4y ago
> which is the package manager to use? pip
- dharmab 4y ago
  My team is using Poetry. Did you know pip can execute arbitrary code when installing packages?
  - teruakohatu 4y ago
    I use poetry but regardless as soon as you import a library arbitrary code is executed.
  - matusf 4y ago
    [dead]
  - uranusjr 4y ago
    Just fyi Poetry is still using pip under the hood. Also (unrelated to Poetry’s pip usage) if you ever build a dependency from source (the situation where pip executes arbitrary code), Poetry also executes arbitrary code.
- a2800276 4y ago
  Unless you are doing scientific computing or using Windows in which case you should use anaconda. Oh and if you are on a Mac you you use brew if you want to install things like tensorflow or imagemagick etc. You see there's only one thing to do it TM :)
  - jacquesm 4y ago
    Anaconda works very well on Linux.
    - a2800276 4y ago
      I agree. Yet still most people doing scientific computing will end up with 17 installations on their system. I was just being facetious about the state of package management :)
jacquesm 4y ago
I've sort of bypassed the whole package manager thingy by using 'Anaconda', it's served me very well so far by insulating me from those particular gotchas. https://www.anaconda.com/ https://www.anaconda.com/ Some would consider it cheating but I haven't got enough hours in the day. If I were to write some ad copy for Anaconda it would be 'in several years of frequent use this particular user didn't have to resolve a single conflict, it all just worked'. Which runs counter to just about all of my other experiences with Python and its package managers/distributions/toolchains, some of those stories would end with 'and as a result the whole server was hosed starting from what should have been a routine certificate update' and other such nice adventures.
- deleted 4y ago
  [deleted]
- logicallee 4y ago
  got it - thanks!
aloisklink 4y ago
If you're installing a command-line tool globally, I usually go for the system package manager (e.g. `apt` on debian), or `pip` if I want a newer version. With pip, make sure to use the `--user` flag so things aren't installed with root. If you're writing Python code, `poetry` has been the least bad package manager in my experience. It has a lock-file, so it should help avoid supply-side attacks (like the PyTorch one from a few days ago [1]), and it means your local environment and CI/CD environment should have the same packages installed. I'd also recommend setting the [`virtualenvs.in-project`][2] setting to `true`, to store the `.venv` in the same folder as your code, so that when you delete your code, all of the downloaded packages get deleted too. Otherwise they'll just stick around in your user directory and use up a lot of disk space. [1]: https://pytorch.org/blog/compromised-nightly-dependency/ https://pytorch.org/blog/compromised-nightly-dependency/ [2]: https://python-poetry.org/docs/configuration/#virtualenvsin-project https://python-poetry.org/docs/configuration/#virtualenvsin-...

Yes, it is nice to see the python 3 migration is pretty much done, it really could have happened literally a decade ago! While we're all here: which is the package manager to use?

anttihaapala 4y ago

> which is the package manager to use? pip

dharmab 4y ago
My team is using Poetry. Did you know pip can execute arbitrary code when installing packages?
- teruakohatu 4y ago
  I use poetry but regardless as soon as you import a library arbitrary code is executed.
- matusf 4y ago
  [dead]
- uranusjr 4y ago
  Just fyi Poetry is still using pip under the hood. Also (unrelated to Poetry’s pip usage) if you ever build a dependency from source (the situation where pip executes arbitrary code), Poetry also executes arbitrary code.
a2800276 4y ago
Unless you are doing scientific computing or using Windows in which case you should use anaconda. Oh and if you are on a Mac you you use brew if you want to install things like tensorflow or imagemagick etc. You see there's only one thing to do it TM :)
- jacquesm 4y ago
  Anaconda works very well on Linux.
  - a2800276 4y ago
    I agree. Yet still most people doing scientific computing will end up with 17 installations on their system. I was just being facetious about the state of package management :)

jacquesm 4y ago

I've sort of bypassed the whole package manager thingy by using 'Anaconda', it's served me very well so far by insulating me from those particular gotchas. https://www.anaconda.com/ https://www.anaconda.com/ Some would consider it cheating but I haven't got enough hours in the day. If I were to write some ad copy for Anaconda it would be 'in several years of frequent use this particular user didn't have to resolve a single conflict, it all just worked'. Which runs counter to just about all of my other experiences with Python and its package managers/distributions/toolchains, some of those stories would end with 'and as a result the whole server was hosed starting from what should have been a routine certificate update' and other such nice adventures.

deleted 4y ago
[deleted]
logicallee 4y ago
got it - thanks!

aloisklink 4y ago

If you're installing a command-line tool globally, I usually go for the system package manager (e.g. `apt` on debian), or `pip` if I want a newer version. With pip, make sure to use the `--user` flag so things aren't installed with root. If you're writing Python code, `poetry` has been the least bad package manager in my experience. It has a lock-file, so it should help avoid supply-side attacks (like the PyTorch one from a few days ago [1]), and it means your local environment and CI/CD environment should have the same packages installed. I'd also recommend setting the [`virtualenvs.in-project`][2] setting to `true`, to store the `.venv` in the same folder as your code, so that when you delete your code, all of the downloaded packages get deleted too. Otherwise they'll just stick around in your user directory and use up a lot of disk space. [1]: https://pytorch.org/blog/compromised-nightly-dependency/ https://pytorch.org/blog/compromised-nightly-dependency/ [2]: https://python-poetry.org/docs/configuration/#virtualenvsin-project https://python-poetry.org/docs/configuration/#virtualenvsin-...