4 ms·

This isn't really usable if you develop with the latest packages for dependencies. While I could generate an rpm for my python module, Fedora and CentOS/RHEL ar

by emidln 11y ago

This isn't really usable if you develop with the latest packages for dependencies. While I could generate an rpm for my python module, Fedora and CentOS/RHEL are going to have incompatible dependencies. I could package up my dependencies too, but then I'll break system software (a lot of fedora's tooling is written in Python). A much better solution would involve making languages like Python accept the Python version as a dependency while making building the PYTHONPATH more flexible with common tooling. Making PYTHONPATH building better would enable installation with namespace+version, such that we could have: /usr/lib/pymodules/requests/1.0.0/... /usr/lib/pymodules/requests/1.2.1/... /usr/lib/pymodules/python/2.7.10/... /usr/lib/pymodules/python/3.5.0/... Python in particular is still a bit challenging in that the compiler assumes it can spit out byte-compiled code all over the place (.pyc), but if someone was going through the tooling effort, they'd presumably find a way (or get something merged if it doesn't exist), to add a shadow directory of .pyc files that could be keyed by version and cached somewhere user-writeable (~/.pymodules for example).

talideon 11y ago
The purpose of PEP 3147 is to fix some of those problems: https://www.python.org/dev/peps/pep-3147/ https://www.python.org/dev/peps/pep-3147/ Addendum: I'd might as well mention that PEP 3147 is implemented from Python 3.2 onwards, so for Python 3, some of the problem is solved. You still have the issue of being able to install different version of the same library.
- emidln 11y ago
  That only really works in a world where all of the users on the machine that can run Python also can write to shared library directories. That seems kinda awkward from a security perspective unless you're already using containerization (in which case you don't care about any of this). This is certainly better than it was (.pyc files just didn't co-exist sanely across python versions), but it doesn't provide a method for running multiple versions of libraries between different apps.
  - talideon 11y ago
    > That only really works in a world where all of the users on the machine that can run Python also can write to shared library directories. Far from it. It's up to the system package manager to generate the .pyc files for the globally installed version of the libraries. __pycache__ directories allow this to be done cleanly for any installed interpreter that supports them. It has all the information it needs, and users need not get involved. Unless the user is running a copy of Python that wasn't installed by the package manager, there should be no need for them to cause the .pyc files to be generated. Besides, .pyc files are an optimisation so that parsing and bytecode compilation doesn't have to repeatedly happen, and aren't even strictly required: the result is slower startup, but it couldn't prevent the code from being run. > it doesn't provide a method for running multiple versions of libraries between different apps Indeed, and I pointed that out in my comment in the addendum. Virtual environments are still pretty much the state of the art in that regard, unfortunately. It's possible that some PYTHONPATH magic might help with that to some degree, but that would be messy.
mordocai 11y ago
Not that it is ready for primetime yet, but guix solves this problem well by allowing each user/application to have its own dependencies of different versions while allowing them to still share if they want the same version. May sound weird, but take a look: https://www.gnu.org/software/guix/ https://www.gnu.org/software/guix/ The main distribution will only ever have free/open source software in it but it is trivial to make your own packages/package repositories and share them since it is just a guile code file.
snuxoll 11y ago
Correct me if I'm wrong, but isn't multi-versioning one of the issues that eggs solve?

by emidln 11y ago

talideon 11y ago

The purpose of PEP 3147 is to fix some of those problems: https://www.python.org/dev/peps/pep-3147/ https://www.python.org/dev/peps/pep-3147/ Addendum: I'd might as well mention that PEP 3147 is implemented from Python 3.2 onwards, so for Python 3, some of the problem is solved. You still have the issue of being able to install different version of the same library.

emidln 11y ago
That only really works in a world where all of the users on the machine that can run Python also can write to shared library directories. That seems kinda awkward from a security perspective unless you're already using containerization (in which case you don't care about any of this). This is certainly better than it was (.pyc files just didn't co-exist sanely across python versions), but it doesn't provide a method for running multiple versions of libraries between different apps.
- talideon 11y ago
  > That only really works in a world where all of the users on the machine that can run Python also can write to shared library directories. Far from it. It's up to the system package manager to generate the .pyc files for the globally installed version of the libraries. __pycache__ directories allow this to be done cleanly for any installed interpreter that supports them. It has all the information it needs, and users need not get involved. Unless the user is running a copy of Python that wasn't installed by the package manager, there should be no need for them to cause the .pyc files to be generated. Besides, .pyc files are an optimisation so that parsing and bytecode compilation doesn't have to repeatedly happen, and aren't even strictly required: the result is slower startup, but it couldn't prevent the code from being run. > it doesn't provide a method for running multiple versions of libraries between different apps Indeed, and I pointed that out in my comment in the addendum. Virtual environments are still pretty much the state of the art in that regard, unfortunately. It's possible that some PYTHONPATH magic might help with that to some degree, but that would be messy.

mordocai 11y ago

Not that it is ready for primetime yet, but guix solves this problem well by allowing each user/application to have its own dependencies of different versions while allowing them to still share if they want the same version. May sound weird, but take a look: https://www.gnu.org/software/guix/ https://www.gnu.org/software/guix/ The main distribution will only ever have free/open source software in it but it is trivial to make your own packages/package repositories and share them since it is just a guile code file.

snuxoll 11y ago

Correct me if I'm wrong, but isn't multi-versioning one of the issues that eggs solve?