4 ms·
The issue with using pip and package managers in a similar vein is that they're not distributing complete packages, which usually means that stuff is compiled w
by dom0 9y ago
The issue with using pip and package managers in a similar vein is that they're not distributing complete packages, which usually means that stuff is compiled when installing some application. End users don't have compilers, and ideally you'd want to avoid them on servers as well. Often this also needs some or many OS-level depencies, like libraries and headers.
There are of course many other issues with pip and setup.py, far too many for a comment here.
But to give just one example; distutils/setuptools try to play god ^W build system with things like Extension. This works, badly. For example, most compilers that aren't GCC or MSVC compatible are not supported at all. Finding libraries? Not supported. Having proper dependency graph detection of files? Nope. If some "invisible" dependency of an output was modified, won't recompile. Ups. It pretty much behaves like a Makefile written by a naive person, just without the "-jN" flag.
- dom0 9y agoNow I know that the PyPA guys do a ton of work to make all this stuff less brittle, and indeed, Python packaging has improved a lot over the years. But overall it's still brittle, complex, difficult to debug and reproduce, and generally falls apart randomly with a frequency that does not inspire trust.
- mrout 9y agoThere's not really a lot you can do about this though, is there? It's not that hard to test that things work, or to see the failures. Comments I've seen like 'there should be a server that installs every package on its own and makes sure its tests pass' have a bit of an issue: we already know the issues exist. Having a big list of them in one place doesn't really give us a lot more information. What's the solution? If you need to compile native dependencies, how do you take PyPI and Python packaging in general and make those native dependencies able to be portably compiled, reliably, on any platform? Things that are straight up unworkable: * Bundling native dependencies as binaries * Specifying a single version of your native dependencies (the 'shrinkwrap' approach) and every library having its own
- pwang 9y ago> Building native dependencies as binaries Yep. http://conda.readthedocs.org http://conda.readthedocs.org
- greglindahl 9y agoWhat? I said pip works great in my environment, not that it works great in yours. By all means, if you're concerned about those cases, use native packages or whatever you like.
- dom0 9y agoI don't intend to contradict you, I'm just providing my angle on where the original comment came from.
- ubernostrum 9y agoPython does now have a distributable binary format: wheels (.whl files). A wheel contains everything already built for the target operating system/architecture, and installation consists solely of moving files into their destinations. Also, I don't have the raw numbers to know for certain, but I suspect packages which have compiled extensions are more rare than pure-Python packages.