5 ms·

Ideally, your language package manager should know how to build (candidate) packages for common system package managers (eg '''setup.py bdist_rpm''' or '''setup

Ideally, your language package manager should know how to build (candidate) packages for common system package managers (eg '''setup.py bdist_rpm''' or '''setup.py bdist_debian'''). That way, the technical end user can install the package in a way their package manager knows about. If your package becomes widely used, it also simplifies (but does not remove) the job of the distribution's packager to build an official package. This doesn't work so well for micro-dependencies, though. The TeTeX distrib. in Fedora is bad enough, I'd hate to see the same done with node packages.

emidln 11y ago
This isn't really usable if you develop with the latest packages for dependencies. While I could generate an rpm for my python module, Fedora and CentOS/RHEL are going to have incompatible dependencies. I could package up my dependencies too, but then I'll break system software (a lot of fedora's tooling is written in Python). A much better solution would involve making languages like Python accept the Python version as a dependency while making building the PYTHONPATH more flexible with common tooling. Making PYTHONPATH building better would enable installation with namespace+version, such that we could have: /usr/lib/pymodules/requests/1.0.0/... /usr/lib/pymodules/requests/1.2.1/... /usr/lib/pymodules/python/2.7.10/... /usr/lib/pymodules/python/3.5.0/... Python in particular is still a bit challenging in that the compiler assumes it can spit out byte-compiled code all over the place (.pyc), but if someone was going through the tooling effort, they'd presumably find a way (or get something merged if it doesn't exist), to add a shadow directory of .pyc files that could be keyed by version and cached somewhere user-writeable (~/.pymodules for example).
- talideon 11y ago
  The purpose of PEP 3147 is to fix some of those problems: https://www.python.org/dev/peps/pep-3147/ https://www.python.org/dev/peps/pep-3147/ Addendum: I'd might as well mention that PEP 3147 is implemented from Python 3.2 onwards, so for Python 3, some of the problem is solved. You still have the issue of being able to install different version of the same library.
  - emidln 11y ago
    That only really works in a world where all of the users on the machine that can run Python also can write to shared library directories. That seems kinda awkward from a security perspective unless you're already using containerization (in which case you don't care about any of this). This is certainly better than it was (.pyc files just didn't co-exist sanely across python versions), but it doesn't provide a method for running multiple versions of libraries between different apps.
    - talideon 11y ago
      > That only really works in a world where all of the users on the machine that can run Python also can write to shared library directories. Far from it. It's up to the system package manager to generate the .pyc files for the globally installed version of the libraries. __pycache__ directories allow this to be done cleanly for any installed interpreter that supports them. It has all the information it needs, and users need not get involved. Unless the user is running a copy of Python that wasn't installed by the package manager, there should be no need for them to cause the .pyc files to be generated. Besides, .pyc files are an optimisation so that parsing and bytecode compilation doesn't have to repeatedly happen, and aren't even strictly required: the result is slower startup, but it couldn't prevent the code from being run. > it doesn't provide a method for running multiple versions of libraries between different apps Indeed, and I pointed that out in my comment in the addendum. Virtual environments are still pretty much the state of the art in that regard, unfortunately. It's possible that some PYTHONPATH magic might help with that to some degree, but that would be messy.
- mordocai 11y ago
  Not that it is ready for primetime yet, but guix solves this problem well by allowing each user/application to have its own dependencies of different versions while allowing them to still share if they want the same version. May sound weird, but take a look: https://www.gnu.org/software/guix/ https://www.gnu.org/software/guix/ The main distribution will only ever have free/open source software in it but it is trivial to make your own packages/package repositories and share them since it is just a guile code file.
- snuxoll 11y ago
  Correct me if I'm wrong, but isn't multi-versioning one of the issues that eggs solve?

emidln 11y ago

This isn't really usable if you develop with the latest packages for dependencies. While I could generate an rpm for my python module, Fedora and CentOS/RHEL are going to have incompatible dependencies. I could package up my dependencies too, but then I'll break system software (a lot of fedora's tooling is written in Python). A much better solution would involve making languages like Python accept the Python version as a dependency while making building the PYTHONPATH more flexible with common tooling. Making PYTHONPATH building better would enable installation with namespace+version, such that we could have: /usr/lib/pymodules/requests/1.0.0/... /usr/lib/pymodules/requests/1.2.1/... /usr/lib/pymodules/python/2.7.10/... /usr/lib/pymodules/python/3.5.0/... Python in particular is still a bit challenging in that the compiler assumes it can spit out byte-compiled code all over the place (.pyc), but if someone was going through the tooling effort, they'd presumably find a way (or get something merged if it doesn't exist), to add a shadow directory of .pyc files that could be keyed by version and cached somewhere user-writeable (~/.pymodules for example).

talideon 11y ago
The purpose of PEP 3147 is to fix some of those problems: https://www.python.org/dev/peps/pep-3147/ https://www.python.org/dev/peps/pep-3147/ Addendum: I'd might as well mention that PEP 3147 is implemented from Python 3.2 onwards, so for Python 3, some of the problem is solved. You still have the issue of being able to install different version of the same library.
- emidln 11y ago
  That only really works in a world where all of the users on the machine that can run Python also can write to shared library directories. That seems kinda awkward from a security perspective unless you're already using containerization (in which case you don't care about any of this). This is certainly better than it was (.pyc files just didn't co-exist sanely across python versions), but it doesn't provide a method for running multiple versions of libraries between different apps.
  - talideon 11y ago
    > That only really works in a world where all of the users on the machine that can run Python also can write to shared library directories. Far from it. It's up to the system package manager to generate the .pyc files for the globally installed version of the libraries. __pycache__ directories allow this to be done cleanly for any installed interpreter that supports them. It has all the information it needs, and users need not get involved. Unless the user is running a copy of Python that wasn't installed by the package manager, there should be no need for them to cause the .pyc files to be generated. Besides, .pyc files are an optimisation so that parsing and bytecode compilation doesn't have to repeatedly happen, and aren't even strictly required: the result is slower startup, but it couldn't prevent the code from being run. > it doesn't provide a method for running multiple versions of libraries between different apps Indeed, and I pointed that out in my comment in the addendum. Virtual environments are still pretty much the state of the art in that regard, unfortunately. It's possible that some PYTHONPATH magic might help with that to some degree, but that would be messy.
mordocai 11y ago
Not that it is ready for primetime yet, but guix solves this problem well by allowing each user/application to have its own dependencies of different versions while allowing them to still share if they want the same version. May sound weird, but take a look: https://www.gnu.org/software/guix/ https://www.gnu.org/software/guix/ The main distribution will only ever have free/open source software in it but it is trivial to make your own packages/package repositories and share them since it is just a guile code file.
snuxoll 11y ago
Correct me if I'm wrong, but isn't multi-versioning one of the issues that eggs solve?