5 ms·

PEP 427 – The Wheel Binary Package Format 1.0

144 points by myko 14y ago

regularfry 14y ago
I really really really really want this to work out so I can justify trying to port it to ruby...
apawloski 14y ago
For those confused by the relatively unhelpful title change, this is an official binary format that can be used for cross-platform distribution.
- codemac 14y ago
  Can they at least post what the title was? This is just getting absurd.
- shill 14y ago
  A few of the title changes lately have been unhelpful. It would be nice to see the original title in small print somewhere.
- lawnchair_larry 14y ago
  Yeah, an overwhelming majority have expressed frustration at this title changing, but they insist on doing it routinely. I understand fixing editorialized titles, but they are not doing that. In most cases, they change it to a far more useless title. Often the article title will have some implied context, and that is completely lost when it is posted to a general aggregation site like HN. This is just one of many cases where changing the title makes no sense at all.
  - aw3c2 14y ago
    Maybe mods could post the original title and their change as comment for people to vote on as feedback. That would also mean we would finally know who the mods are...
pestaa 14y ago
In the header the following line confused me very much: Content-Type: text/x-rst Took me a short while to figure out it actually marks the type of the PEP rather than the documented format.
polarix 14y ago
So here's my question: Are things in a wheel allowed to rely on ANY site-packages? IF not, this is spectacular. If so, another unfortunate attempt in the long long struggle.
- jessaustin 14y ago
  I'm not sure if this answers your question, but the format includes PKG-INFO, which includes a "Requires" field. How do you envision disallowing dependencies for python packages working? Would all packages have to contain all of their dependencies? I don't think that would ever happen in python-land. It would break lots of things, for dubious benefit.
  - polarix 14y ago
    Thanks. Seems like that may solve the problem, if people actually use it.
    - Schiphol 14y ago
      A question to those who know more: for those of us who only target Linux, isn't it better to leave these things to distro package managers?
      - michaelhoffman 14y ago
        Only if you want to limit the people who can install your software to those with one of the package managers and configurations you target, and who have root.
        dalke 14y ago
        To give a specific example of what michaelhoffman wrote, I have multiple Python virtual installations because my full regression suite tests for about 12 different installation possibilities, including for nice error reporting if a given module isn't installed. There's no way to do that using the standard package managers, except by the much more complicated solution of having multiple OS instances available.
deleted 14y ago
[deleted]
StavrosK 14y ago
Can someone explain the situation with eggs and all the installation utilities? After years of programming in python, I still don't understand why everyone makes such a huge deal of installation, which is basically copying things somewhere in the PYTHONPATH.
- lloeki 14y ago
  First, this does not require code to execute (which setuptools require). Second, the problem is to know what to copy, or rather what to get. even when it's pure python it can be a problem (e.g py2 vs py3) and it's even more of one with e.g C extensions, where you have a choice of building it from source (thus requiring gcc and all build dependencies such as headers) or getting a pre-built binary tied to a given platform. Ruby gems support that since quite some time, and it's a bliss to install e.g the libv8 gem in a few seconds vs a lengthy build. [0]: https://rubygems.org/gems/libv8/versions https://rubygems.org/gems/libv8/versions
  - StavrosK 14y ago
    Oh, so it's more of an improvement rather than "eggs are horribly broken". I think I'm confused with the "easy_install vs pip" thing, where I also don't see much of a difference. Easy binary retrieval sounds fantastic, though.
- addlepate 14y ago
  I'm pretty new to Python and Django, and maybe someone more experienced can explain it better, but installation seems really brittle to me. Every few weeks I run into some silly problem like, I need this package, but it depends on an older version of distutils or setup-whatever, so trying to install it fails until I get that, but then I have to uninstall that after getting the package or the rest of the packages I want won't install right... kind of a nightmare. Usually it doesn't take a super long time to resolve but there's never any rhyme or reason, just install and uninstall and easy_install and pip and so on until it's happy so I can get on with my work. It seems like this proposal would help matters. I hope.
- slurgfest 14y ago
kklimonda 14y ago
Hmm... How is it different from binary eggs?
- michaelhoffman 14y ago
  There's a section in the document which compares wheel with egg.
- dalke 14y ago
  Does http://www.python.org/dev/peps/pep-0427/#comparison-to-egg http://www.python.org/dev/peps/pep-0427/#comparison-to-egg answer your question? If not, what would?
zdw 14y ago
So, where's the wheel to .deb/.rpm/.msi/.pkg converter? For real ease of distribution, you have to support a destination platform's native package format. See also: http://xkcd.com/927/ http://xkcd.com/927/
- inopinatus 14y ago
  I've long thought the better mechanism is to hook into the destination platform's native package system. A la BSDPAN. [Such that installing your language-specific package causes appropriate entries to appear in the system's global database of installed packages. Thus making it easier to identify candidates for upgrade, ensure dependencies are correct, remove redundant libs and so forth]
  - endgame 14y ago
    Hear, hear. Though I believe the best way to make that happen is to make your language play nice with automake. Nobody believes me.
    - klodolph 14y ago
      I believe you. This is why I don't use CMake / SCons / Jam / etc -- they're nice for developers but bad for the package maintainers and sysadmins.
      - mpyne 14y ago
        CMake is widely distributed now if only because of KDE, so the package maintainers and sysadmins have figured it out one way or the other already.
    - shadowmint 14y ago
      Even fink uses cmake to build and install a whole bunch of packages now. The future, it's not with automake.
    - jack12 14y ago
      I'd love to see support for writing directly into distribution's package managers' databases added to things like pip, npm, gems, etc. too but I'm curious what playing nicely with automake gains here. I'm not too experienced with it, and I don't quite see where it would fit into that process. I think of automake as being more part of the build process. But if things worked as you and inopinatus suggest, and you have taught something like pip 2.0 to bypass the rpm package manager and instead add records directly to the RPM database for a binary wheel package that pip has installed, aren't you kind of routing around any problems of a distribution's compatibility with a language's custom build process entirely? Or did you mean to reply one level up? Though, again, I'm not sure where automake would help with converting one binary package format into a distribution-specific binary format.
mcantelon 14y ago
Why the name "wheel"?
- slurgfest 14y ago
  Maybe it's a nod to how PyPI used to be called "the cheese shop" (after the Monty Python sketch)
- thruflo 14y ago
  Because ‘newegg’ was taken. Python packaging - reinvented. A container for cheese. From https://bitbucket.org/aragilar/wheel/src/38ab0c4bca96/docs/index.rst?at=default https://bitbucket.org/aragilar/wheel/src/38ab0c4bca96/docs/i...
  - mcantelon 14y ago
    Aha! Makes sense.
slurgfest 14y ago
Good, let's just kill eggs now
deleted 14y ago
[deleted]
markov_twain 14y ago
Literally reinventing the wheel

144 points by myko 14y ago

I really really really really want this to work out so I can justify trying to port it to ruby...

For those confused by the relatively unhelpful title change, this is an official binary format that can be used for cross-platform distribution.

codemac 14y ago
Can they at least post what the title was? This is just getting absurd.
shill 14y ago
A few of the title changes lately have been unhelpful. It would be nice to see the original title in small print somewhere.
lawnchair_larry 14y ago
Yeah, an overwhelming majority have expressed frustration at this title changing, but they insist on doing it routinely. I understand fixing editorialized titles, but they are not doing that. In most cases, they change it to a far more useless title. Often the article title will have some implied context, and that is completely lost when it is posted to a general aggregation site like HN. This is just one of many cases where changing the title makes no sense at all.
- aw3c2 14y ago
  Maybe mods could post the original title and their change as comment for people to vote on as feedback. That would also mean we would finally know who the mods are...

pestaa 14y ago

In the header the following line confused me very much: Content-Type: text/x-rst Took me a short while to figure out it actually marks the type of the PEP rather than the documented format.

polarix 14y ago

So here's my question: Are things in a wheel allowed to rely on ANY site-packages? IF not, this is spectacular. If so, another unfortunate attempt in the long long struggle.

jessaustin 14y ago
I'm not sure if this answers your question, but the format includes PKG-INFO, which includes a "Requires" field. How do you envision disallowing dependencies for python packages working? Would all packages have to contain all of their dependencies? I don't think that would ever happen in python-land. It would break lots of things, for dubious benefit.
- polarix 14y ago
  Thanks. Seems like that may solve the problem, if people actually use it.
  - Schiphol 14y ago
    A question to those who know more: for those of us who only target Linux, isn't it better to leave these things to distro package managers?
    - michaelhoffman 14y ago
      Only if you want to limit the people who can install your software to those with one of the package managers and configurations you target, and who have root.
      - dalke 14y ago
        To give a specific example of what michaelhoffman wrote, I have multiple Python virtual installations because my full regression suite tests for about 12 different installation possibilities, including for nice error reporting if a given module isn't installed. There's no way to do that using the standard package managers, except by the much more complicated solution of having multiple OS instances available.

deleted 14y ago

[deleted]

StavrosK 14y ago

Can someone explain the situation with eggs and all the installation utilities? After years of programming in python, I still don't understand why everyone makes such a huge deal of installation, which is basically copying things somewhere in the PYTHONPATH.

lloeki 14y ago
First, this does not require code to execute (which setuptools require). Second, the problem is to know what to copy, or rather what to get. even when it's pure python it can be a problem (e.g py2 vs py3) and it's even more of one with e.g C extensions, where you have a choice of building it from source (thus requiring gcc and all build dependencies such as headers) or getting a pre-built binary tied to a given platform. Ruby gems support that since quite some time, and it's a bliss to install e.g the libv8 gem in a few seconds vs a lengthy build. [0]: https://rubygems.org/gems/libv8/versions https://rubygems.org/gems/libv8/versions
- StavrosK 14y ago
  Oh, so it's more of an improvement rather than "eggs are horribly broken". I think I'm confused with the "easy_install vs pip" thing, where I also don't see much of a difference. Easy binary retrieval sounds fantastic, though.
addlepate 14y ago
I'm pretty new to Python and Django, and maybe someone more experienced can explain it better, but installation seems really brittle to me. Every few weeks I run into some silly problem like, I need this package, but it depends on an older version of distutils or setup-whatever, so trying to install it fails until I get that, but then I have to uninstall that after getting the package or the rest of the packages I want won't install right... kind of a nightmare. Usually it doesn't take a super long time to resolve but there's never any rhyme or reason, just install and uninstall and easy_install and pip and so on until it's happy so I can get on with my work. It seems like this proposal would help matters. I hope.
slurgfest 14y ago

kklimonda 14y ago

Hmm... How is it different from binary eggs?

michaelhoffman 14y ago
There's a section in the document which compares wheel with egg.
dalke 14y ago
Does http://www.python.org/dev/peps/pep-0427/#comparison-to-egg http://www.python.org/dev/peps/pep-0427/#comparison-to-egg answer your question? If not, what would?

zdw 14y ago

So, where's the wheel to .deb/.rpm/.msi/.pkg converter? For real ease of distribution, you have to support a destination platform's native package format. See also: http://xkcd.com/927/ http://xkcd.com/927/

inopinatus 14y ago
I've long thought the better mechanism is to hook into the destination platform's native package system. A la BSDPAN. [Such that installing your language-specific package causes appropriate entries to appear in the system's global database of installed packages. Thus making it easier to identify candidates for upgrade, ensure dependencies are correct, remove redundant libs and so forth]
- endgame 14y ago
  Hear, hear. Though I believe the best way to make that happen is to make your language play nice with automake. Nobody believes me.
  - klodolph 14y ago
    I believe you. This is why I don't use CMake / SCons / Jam / etc -- they're nice for developers but bad for the package maintainers and sysadmins.
    - mpyne 14y ago
      CMake is widely distributed now if only because of KDE, so the package maintainers and sysadmins have figured it out one way or the other already.
  - shadowmint 14y ago
    Even fink uses cmake to build and install a whole bunch of packages now. The future, it's not with automake.
  - jack12 14y ago
    I'd love to see support for writing directly into distribution's package managers' databases added to things like pip, npm, gems, etc. too but I'm curious what playing nicely with automake gains here. I'm not too experienced with it, and I don't quite see where it would fit into that process. I think of automake as being more part of the build process. But if things worked as you and inopinatus suggest, and you have taught something like pip 2.0 to bypass the rpm package manager and instead add records directly to the RPM database for a binary wheel package that pip has installed, aren't you kind of routing around any problems of a distribution's compatibility with a language's custom build process entirely? Or did you mean to reply one level up? Though, again, I'm not sure where automake would help with converting one binary package format into a distribution-specific binary format.

mcantelon 14y ago

Why the name "wheel"?

slurgfest 14y ago
Maybe it's a nod to how PyPI used to be called "the cheese shop" (after the Monty Python sketch)
thruflo 14y ago
Because ‘newegg’ was taken. Python packaging - reinvented. A container for cheese. From https://bitbucket.org/aragilar/wheel/src/38ab0c4bca96/docs/index.rst?at=default https://bitbucket.org/aragilar/wheel/src/38ab0c4bca96/docs/i...
- mcantelon 14y ago
  Aha! Makes sense.

slurgfest 14y ago

Good, let's just kill eggs now

deleted 14y ago

[deleted]

markov_twain 14y ago

Literally reinventing the wheel