3 ms·

Deprecation of bdist_egg Uploads to PyPI

3 points by woodruffw 3y ago

PaulHoule 3y ago
That PEP doesn’t really explain the situation which is that egg is a major reason why pip is so FUBAR. With an egg you can’t know what dependencies a package asks for without running a setup script that may itself bring dependencies in and do who knows what in the process. Maven has a correct dependency resolution algorithm for Java because it can download the metadata for all the JAR files and solve the dependencies before it has to install them. I’m pretty sure you could do the same for Python but so long as eggs are supported you are stuck with pip’s “start installing stuff and hope for the best” strategy where it frequently paints itself into the corner. I’m looking forward to a day where pip ‘just works’. Note egg can inspect the environment, install alternate dependencies, compile C code, etc. Python really needs a mechanism better than it has for optional or alternate dependencies. For instance I should be able to tell it to substitute Intel’s MKL scikit-learn for the default scikit-learn by simply telling pip that it is a substitute.
- woodruffw 3y ago
  FWIW, I think you're mixing eggs up with source distributions ("sdists"): eggs are much closer to wheels in that they're a "built" copy of the package rather than a collection of source that requires dynamic code execution for metadata generation. The main reason they're being removed is because, unlike wheels, they were never really standardized and have effectively been a vestige of the packaging ecosystem since ~2016.
  - PaulHoule 3y ago
    They still compute the dependency list programmatically with the setup.py.
    - woodruffw 3y ago
      Hum, for some reason I thought it was part of the pre-baked `EGG-INFO`. Apologies if I was wrong! (Either way, Python's dynamic metadata generation will unfortunately remain present: sdists fundamentally require it even with PEP 517/8, so deprecating eggs doesn't unlock a change there.)
      - PaulHoule 3y ago
        I went and looked at the docs and there is supposed to be a PKG-INFO in the egg which has the ‘Requires’ statement for dependencies so it looks like I was wrong but boy those docs are bad. (Had to look at several to find the answer)

3 points by woodruffw 3y ago

That PEP doesn’t really explain the situation which is that egg is a major reason why pip is so FUBAR. With an egg you can’t know what dependencies a package asks for without running a setup script that may itself bring dependencies in and do who knows what in the process. Maven has a correct dependency resolution algorithm for Java because it can download the metadata for all the JAR files and solve the dependencies before it has to install them. I’m pretty sure you could do the same for Python but so long as eggs are supported you are stuck with pip’s “start installing stuff and hope for the best” strategy where it frequently paints itself into the corner. I’m looking forward to a day where pip ‘just works’. Note egg can inspect the environment, install alternate dependencies, compile C code, etc. Python really needs a mechanism better than it has for optional or alternate dependencies. For instance I should be able to tell it to substitute Intel’s MKL scikit-learn for the default scikit-learn by simply telling pip that it is a substitute.

woodruffw 3y ago
FWIW, I think you're mixing eggs up with source distributions ("sdists"): eggs are much closer to wheels in that they're a "built" copy of the package rather than a collection of source that requires dynamic code execution for metadata generation. The main reason they're being removed is because, unlike wheels, they were never really standardized and have effectively been a vestige of the packaging ecosystem since ~2016.
- PaulHoule 3y ago
  They still compute the dependency list programmatically with the setup.py.
  - woodruffw 3y ago
    Hum, for some reason I thought it was part of the pre-baked `EGG-INFO`. Apologies if I was wrong! (Either way, Python's dynamic metadata generation will unfortunately remain present: sdists fundamentally require it even with PEP 517/8, so deprecating eggs doesn't unlock a change there.)
    - PaulHoule 3y ago
      I went and looked at the docs and there is supposed to be a PKG-INFO in the egg which has the ‘Requires’ statement for dependencies so it looks like I was wrong but boy those docs are bad. (Had to look at several to find the answer)