3 ms·

In Python, though, there seem to be competing managers (easy_install and pip) and separate ways to package libraries for uploading. I’m also hearing names like

by dkarl 16y ago

In Python, though, there seem to be competing managers (easy_install and pip) and separate ways to package libraries for uploading. I’m also hearing names like setuptools, distutils, distribute and virtualenv, and I have to say that the whole ecosystem isn’t too clear to me yet. And the documentation tends to assume I know what all the above mean already! This also bugged me the last time I wrote any Python code. I decided the sensible response was to go write Scala code until the situation resolved itself. Has it? (I hope Python never acquires the problem C has, where its users are scattered so far and wide that every halfway valid tool attracts enough satisfied users to sustain itself, preventing the community from being coherent enough to advance in any direction at all. Guido needs to be a Ghengis who keeps all the tribes pulled together and galloping in the same direction, not scattering into disconnected groups.)

StavrosK 16y ago
Those names are not all disjoint. pip has superseded easy_install, so you should use the former, distribute is a way to write setup packages (pip only installs them), and virtualenv is a way to avoid messing up your system with packages by installing them to a single, self-contained directory. There's really no fragmentation whatsoever that I've seen. Install a package using your preferred way, in the end everything installs fine whether it's built with setuptools, distutils or distribute (or just copy the files to your site-packages directory). I haven't read the entire blog post yet, so I haven't gotten to the part you quote, but that's my two cents.
- cdavid 16y ago
  Actually, there is fragmentation: pip does not supersedes easy_install, since easy_install does things that pip cannot do (like installing from something that is not a source tarball). Also, distribute being a fork of setuptools, with neither distribute/setuptools being a superset of each other, means more fragmentation as well. Also, the affirmation that everything install fine whether you use setuptools/distribute/distutils is simply not true. We get almost daily bug reports/user complaints in numpy/scipy because of installation issues mostly caused by distutils, or people using pip/easy_install (which does not work with numpy distutils extensions). The situation is confusing for a lot of users, especially the ones who are not super familiar with python.
cdavid 16y ago
Python packaging is indeed a mess. The scipy community in particular is acutely aware of the issue - distutils and all the cruft on the top (setuptools, pip, etc...) are not robust and extremely adverse to extensibility. This is a complex issue, though, especially if you want to play well with native tools on each platform. It does not help that python packaging tools have been created by accretion instead of begin designed around simple, robust principles.
- cookietime 16y ago
  For anyone who wants to help Python wrt packaging, get behind [Tarek] and his work on distutils2. That's the future of Python packaging (with Guido's blessing) [^1]. More info on python packaging temporarily at http://guide.python-distribute.org/ http://guide.python-distribute.org/ . There's also a [packaging discussion group] which seems to be primarily focused on distutils2 development. [Tarek]: http://ziade.org/ http://ziade.org/ [^1]: http://tarekziade.wordpress.com/2010/03/03/the-fate-of-distutils-pycon-summit-packaging-sprint-detailed-report/ http://tarekziade.wordpress.com/2010/03/03/the-fate-of-distu... [packaging discussion group]: http://groups.google.com/group/the-fellowship-of-the-packaging/topics?gvc=2 http://groups.google.com/group/the-fellowship-of-the-packagi...
  - cdavid 16y ago
    I am aware of distutils2 - the consensus in the scipy community is that it won't improve the situation at best, and will most likely make it even worse.
    - cookietime 16y ago
      Sorry, didn't mean to address you personally. Edited my post. Also, will study your Oct. 13 blog post on the matter to learn more about the issues.
      - cdavid 16y ago
        ah, sorry for the misunderstanding. Note that I don't pretend to be authoritative on the subject (although I think it is fair to say I am within the scipy community).

by dkarl 16y ago

StavrosK 16y ago

Those names are not all disjoint. pip has superseded easy_install, so you should use the former, distribute is a way to write setup packages (pip only installs them), and virtualenv is a way to avoid messing up your system with packages by installing them to a single, self-contained directory. There's really no fragmentation whatsoever that I've seen. Install a package using your preferred way, in the end everything installs fine whether it's built with setuptools, distutils or distribute (or just copy the files to your site-packages directory). I haven't read the entire blog post yet, so I haven't gotten to the part you quote, but that's my two cents.

cdavid 16y ago
Actually, there is fragmentation: pip does not supersedes easy_install, since easy_install does things that pip cannot do (like installing from something that is not a source tarball). Also, distribute being a fork of setuptools, with neither distribute/setuptools being a superset of each other, means more fragmentation as well. Also, the affirmation that everything install fine whether you use setuptools/distribute/distutils is simply not true. We get almost daily bug reports/user complaints in numpy/scipy because of installation issues mostly caused by distutils, or people using pip/easy_install (which does not work with numpy distutils extensions). The situation is confusing for a lot of users, especially the ones who are not super familiar with python.

cdavid 16y ago

Python packaging is indeed a mess. The scipy community in particular is acutely aware of the issue - distutils and all the cruft on the top (setuptools, pip, etc...) are not robust and extremely adverse to extensibility. This is a complex issue, though, especially if you want to play well with native tools on each platform. It does not help that python packaging tools have been created by accretion instead of begin designed around simple, robust principles.

cookietime 16y ago
For anyone who wants to help Python wrt packaging, get behind [Tarek] and his work on distutils2. That's the future of Python packaging (with Guido's blessing) [^1]. More info on python packaging temporarily at http://guide.python-distribute.org/ http://guide.python-distribute.org/ . There's also a [packaging discussion group] which seems to be primarily focused on distutils2 development. [Tarek]: http://ziade.org/ http://ziade.org/ [^1]: http://tarekziade.wordpress.com/2010/03/03/the-fate-of-distutils-pycon-summit-packaging-sprint-detailed-report/ http://tarekziade.wordpress.com/2010/03/03/the-fate-of-distu... [packaging discussion group]: http://groups.google.com/group/the-fellowship-of-the-packaging/topics?gvc=2 http://groups.google.com/group/the-fellowship-of-the-packagi...
- cdavid 16y ago
  I am aware of distutils2 - the consensus in the scipy community is that it won't improve the situation at best, and will most likely make it even worse.
  - cookietime 16y ago
    Sorry, didn't mean to address you personally. Edited my post. Also, will study your Oct. 13 blog post on the matter to learn more about the issues.
    - cdavid 16y ago
      ah, sorry for the misunderstanding. Note that I don't pretend to be authoritative on the subject (although I think it is fair to say I am within the scipy community).