3 ms·
My ELI7 understanding is that python installs required modules (libraries) at the system level by default, and doesn't have good support for installing modules
by maliker 6y ago
My ELI7 understanding is that python installs required modules (libraries) at the system level by default, and doesn't have good support for installing modules into local directories like node/C/java/etc. does, which leads to conflicts. Users have to find a set of modules that support all of the scripts on their system. The current workaround is to isolate each script's requirements, but there are a ton of competing tools to do this (virtualenv, pipenv, poetry, etc.), so it's hard to choose the "best" isolation solution.
One reason people like PEP 582 is that, by being something built into the interpreter, it will reduce a lot of the confusion over which isolation tool to use.
Part of the problem, which local installs won't completely solve, is that the python interpreter is still changing fast enough that modules are having trouble keeping up. E.g. tensorflow is still working on python3.9 support [1], and now we'll see python3.10 in the next couple weeks. Even with local installs we'll have to be careful to use the right interpreter.
Another piece of the problem, which is also kind of a good thing, is the huge size of the python module ecosystem. I've got python scripts with over a hundred requirements. It saves me from writing a lot of code, but resolving the requirements is a challenge. The pip package manager has recently added a dependency resolver, which has helped a little make sure everything can work together.
The only surefire ways I can think of to make this huge module ecosystem interoperate cleanly are (1) have every module support every other version of every other module and interpreter (insane amount of work, won't happen) or (2) have simpler interop interfaces like the unix shell convention of only passing strings between programs, but that would take away a lot of the ease-of-use of passing rich objects around between modules.
[1] https://github.com/tensorflow/tensorflow/issues/44485 https://github.com/tensorflow/tensorflow/issues/44485