3 ms·
No, no no. No 'alternately'. The python community needs to pick ONE standard and, uh...standardize on it.
by ForHackernews 11y ago
No, no no. No 'alternately'. The python community needs to pick ONE standard and, uh...standardize on it.
- dvanduzer 11y agoThis isn't the issue. Wheels and Conda are almost completely orthogonal. Conda is a higher level abstraction than pip. Wheels is operating at the lowest level of abstraction. Multiple package management tools are relatively fine. Having multiple competing format standards (egg vs wheel vs lots of other options) would be a disaster. (For reference, there's .deb versus .rpm, but also dpkg versus apt-get versus yum versus up2date versus...)
- shoo 11y agoFrom a continuum.io blog post about Conda [1]: > Having been involved in the python world for so long, we are all aware of pip, easy_install, and virtualenv, but these tools did not meet all of our specific requirements. The main problem is that they are focused around Python, neglecting non-Python library dependencies, such as HDF5, MKL, LLVM, etc., which do not have a setup.py in their source code and also do not install files into Python’s site-packages directory. I haven't used Conda, but I totally get their main point here. Python packages sometimes depend upon non-Python packages. Any Python-specific packaging solution that cannot express that e.g. a Python package depends upon a native library does not really solve this problem. I don't regard packaging, dependency management, and isolation as programming-language-dependent problems. [1] http://www.continuum.io/blog/conda http://www.continuum.io/blog/conda
- dvanduzer 11y agoI haven't evaluated Conda either, but the wheel vs egg distinction is categorically different from managing build chains for huge tools like Hadoop.
- ngoldbaum 11y agoReplace "sometimes" with "nearly always" in the scientific world.
- IanOzsvald 11y ago+1 that conda solves "most of our packaging problems" in the science/data science world (I've lived within it for 1.5 years, it really simplifies things). I use conda on Linux, Mac and Windows for r&d and deployment for tools including numpy, sklearn, matplotlib along with flask, pymongo etc.
- e12e 11y agoConda certainly sounds ambitious -- but I'm doubtful how useful it really is. As I see it, it mainly fills two needs: the lack of a package manager on OS X and the lack of a package manager on Windows. Windows is getting a package manager now, and OS X arguably have the App Store. Everywhere else (read: Linux/BSD) -- most of what's packaged by Conda, is already packaged by the vendor. In short: I don't think it's the best way forward (but it may very well be the best way for many usecases/users right now and in the near future).
- shoo 11y agoI agree. There's probably also some element of "it isn't the best general solution, but it's super convenient for some particular group of people, e.g. python users".
- cossatot 11y agoA big part of Conda's usefulness is not system-wide package management a la apt or yum or whatever, but being able to have reproducible, isolated environments for scientific computing or other times when reproducibility is very important, and/or when you need multiple versions of packages for compatibility with other packages. I believe it uses virtualenv for this, but I may be wrong. Regardless, having this feature (as well as no interference with the system Python) is one of the killer features for me, and others I'm sure.
- e12e 11y agoThis is a great point. Still, whole system snapshots in the form of a vm disk image, vagrant etc might be a better way to serve that use-case. Perhapssome form of appkiance levering something like guix or coreOS?
- cossatot 11y agoAn additional benefit of Conda is that building or installing packages that require a C/Fortran compiler can be done without using sudo. This is really nice when you're working on a cluster or other shared computer resources, and don't want to have to bother the sysadmin.
- analog31 11y agoAs I understand it, it's still the same standard. The distributions that I've used, unpack themselves to look just like a mainstream installation, including pip. So, if you have to install a missing package, you still do so in the same way. The distributions just give you a head start.
- ubernostrum 11y agoThere are at least two major classes of people using Python. On the one hand, there are people who write, distribute and deploy Unix-y-type software -- system utilities, web applications, service daemons, etc. These users write for users who are like themselves, or who are sysadmins of Unix-y systems. Their concern is with automatable, repeatable, command-line-friendly packaging, distribution and deployment. They benefit from wheels because it saves them a potentially-time-consuming build step during installation, and will generally prefer wheels or tarballs as a distribution format. On the other hand, there are people who use Python as a scientific computing and number-crunching stack. Many of them do not write and distribute their own applications; they distribute the results of their code. Many of them run on desktop-type machines using Windows. What they primarily want is a single .exe they can download and run which will provide Python plus all the numeric/scientific libraries in one install, plus (increasingly) IPython Notebook as a way to share their work. There is no single standard which works well for both of these groups.
- e12e 11y ago> There is no single standard which works well for both of these groups. Sure there is. A single exe that bootstraps python and leverage wheels to install what the scientific user needs. In fact, with so many new tools comming to windows lately (as in the past decade) -- that exe could probably be a powershell-script. And an equivalent shell-script could probably do the same for more unix-y platforms. Or one might make a script that builds an installer that bundles up things downloaded via the wheels framework.
- pekk 11y agoI'm sorry if things aren't working well for you. The Python community already did pick a standard, which is PyPI, pip and wheels. All this is covered in PEPs. Unfortunately, the creators of Conda present their own tool to compete with the standard with a complete fork of all of Python's packaging, right down to virtualenv, which really only works for the packages that they wanted to support. That isn't the fault of the broader Python community. There is nothing anyone else can do to prevent that.