8 ms·
Package management is probably, today, a very well defined problem. Isn't it time to solve this once and for all? Or are we forever doomed to have multiple pack
by sergiosgc 12y ago
Package management is probably, today, a very well defined problem. Isn't it time to solve this once and for all? Or are we forever doomed to have multiple package managers for the dot product of [operating system]X[development platform]?
It itches me that php has pear and composer, node has npm, ruby has gems and python has pip. It itches that deb and rpm solve the exact same problem, not to mention all the other smaller package formats/managers like pacman, yaourt or portage. It looks as pretty obvious that it is the same problem, albeit attacked via some different angles (source build versus binary distribution notably).
It even looks like it could be cross-platform, solving in one simple swoop at least unix systems (namely linux along with the BSDs) and, in a dream world, non-unix systems (windows, notably).
Daydreaming, yes. The status quo is too well established. But it justifies my jaded look regarding Guix: Late comer to a party that is already stale and over.
- lucian1900 12y agoNix (and by extension Guix) are meant to be this solution you're looking for. The problem, as with most unified solutions, is adoption.
- sergiosgc 12y agoThen that is the very first thing they must state. Instead, the opening line is: "GNU Guix is the functional package manager for the GNU system and a distribution thereof." In addition, this problem can only be solved by luring established platforms, and this can only be achieved by providing a transition path. It should be able to coexist with other package managers (composer, or pip, or npm or, if shooting for the moon, rpm and deb).
- _delirium 12y agoHaving multiple system package managers coexisting is pretty thorny. If very carefully delineated so they manage completely independent bits of software installed in their own prefixes, I could see it working (with some careful path management). But if you're actually installing/upgrading system software with it, having some mix of apt, rpm, and guix managing your packages sounds like a recipe for pain.
- davexunit 12y agoMuch like how many of us use pip, gem, npm, composer, etc. on our Debian/Red Hat/etc. machines, you can also use Guix in the same way. For example, you are deploying a production web application and Guix had newer versions of some dependencies than your stable host system. You could use Guix to create a profile (a collection of installed packages, a bundle in Ruby terminology) that includes those dependencies for your web application to use. It's possible to use pip and friends on a Guix system, too. I personally do all of my Guix hacking on a Debian machine. They play very nicely together.
- akerl_ 12y agoI'm always confused to see this point brought up. Whenever there are lots of options, folks talk about the harms of fragmentation and wasted effort and focus. Whenever there aren't a lot of options, it seems like the same crowd demands disruption and competing choices.
- sergiosgc 12y agoDifferent people, probably. I'm all for consolidation whenever the problem space is very well defined, and divergence whenever the problem space is ill defined. This is a case of the former.
- abenga 12y agoThis is a pretty common point of view, but I wonder, how would this be enforced in a free-and-open software development ecosystem? Who would go around stopping people from launching their own take on a solution to the same problem apart from the "blessed" solution? In a world where everyone is free to choose which problem they'd like to solve and how to go about it, and it is impossible for everyone to agree about even the most trivial of options, you cannot ever have only one text editor, package manager, init system, etc. Seeing how well most developers welcome change in their favorite OSes, editors, etc, I imagine a world where every developer bends their efforts towards the one true chosen solution, and are not allowed or able to choose to "waste" their efforts on another solution wouldn't yield as vibrant a software ecosystem as we have now.
- sergiosgc 12y ago> This is a pretty common point of view, but I wonder, how would this be enforced in a free-and-open software development ecosystem? It wouldn't be enforced. The beauty of OSS is that there is no enforcing of solutions. Even when you touch something as core as the init system or the graphics system, the solution must prevail by technical merits. There is marketing, but thankfully this is a world of technical evaluation first, selling second and never one of enforcement. Change is more difficult, but so are mandated blunders.
- unterstrom 12y agoMandatory XKCD comic: http://xkcd.com/927/ http://xkcd.com/927/
- taylanub 12y agoYou beat me to it. :-)
- bryanlarsen 12y agoI would argue that it's far from a well defined problem. deb/rpm have very fundamental differences from gem/pip/npm etc. CoreOS takes a very different approach to packaging, and so does Nix/guix. If it truly was a well defined problem, everybody would be using RPM. All the package managers you mentioned were created because somebody perceived a problem or limitation with previous formats.
- sergiosgc 12y ago> I would argue that it's far from a well defined problem. deb/rpm have very fundamental differences from gem/pip/npm etc. I know the internals of pear, pip, rpm and deb. I don't recognize fundamental differences between them, although this evaluation depends on the definition of fundamental. At the very core, all of them catalog files on the filesystem, script configuration of files once written and trigger reactions from other packages to the appearance of a new package. The key difference is one of domain. Development platform specific package managers do their operations from inside the development language, with the niceties that come about from that (better APIs to change the development environment). This is not fundamental. As for being well defined, I state the problem as: If you lock in a room, for a couple of hours, the lead developers of deb and pip, would they be able to come up with a unified package manager? It's, for me, clear that they would. Rinse and repeat, and you get a full unified spec.
- bryanlarsen 12y ago"It's, for me, clear that they would." I highly doubt it. The deb developer is a sysadmin. The pip developer is a programmer. That's a big gap. For example, you can install many different versions of a gem simultaneously, which is very useful to a programmer. A deb can only install a single version at a time. That's best from a sysadmin perspective.
- sergiosgc 12y agoYou can have multiple versions of the same deb package. Look for installed Linux kernels on your system and you'll likely find an example right there. This is actually a good example of the good definition of the problem space. All corner cases have been dug up. It's not a priori obvious that an operating system needs multiple versions of a package, but in some cases it does. And the package manager supports it.
- 1ris 12y agoI hope this solves the fad that every new programming language writes its own package manager.
- davidw 12y agoAdopting a GPL'ed tool written in Guile to install packages for their own languages? I don't see it as likely.
- 1ris 12y agoGPL'ed means everybody can potentially use it. It's a stand alone tool the user has already installed, it does not have to be distributed, or liked against. And I don't see why these people wouldn't write a few lines a guile. Right now they usually write a few lines of shell.
- cpa 12y agoIt's well defined, but there's no good solution to this problem yet! See: http://www.well-typed.com/blog/2014/09/how-we-might-abolish-cabal-hell-part-1/ http://www.well-typed.com/blog/2014/09/how-we-might-abolish-... It's true for every language out there, not just Haskell.
- taylanub 12y agoGuix is not late in that there aren't many purely functional package managers out there. It's the second after Nix if I'm not mistaken, and it doesn't reinvent the purely functional wheel either; it builds on top of existing core parts of Nix. Why not use Nix? Most importantly because it's not GNU; NIH is sadly a real problem for GNU. But Nix also uses C++/Perl and a stand-alone package recipe language; Guix uses Scheme as much as possible. ISTR Guix also has some features that Nix lacks, such as unprivileged package management. Don't quote me on it though; mayhaps it's trivial to add to Nix or has already been done. Cooperating with existing infrastructure is, I imagine, very difficult for a purely functional package manager, given the whole "purely functional" aspect. :-) It's also extremely unlikely that Debian will just give up apt, and RedHad will just give up rpm. And every programming language platform seems to have its own package manager precisely because of that reason: there is no one package manager on all systems, but a proglang platform wants to offer package management on all systems, so each bake their own implemented in and special-made for the language. It's a sad state, but that's the world of software for you. http://xkcd.com/927/ http://xkcd.com/927/
- seagreen 12y agoNix does have unprivileged package management via a CLI tool, though the absolutely fantastic describe-the-whole-system-in-one-file wonder that's configuration.nix[1] takes elevated privileges. [1] E.g: https://github.com/Fuuzetsu/nix-project-defaults/blob/master/nixos-config/configuration.nix https://github.com/Fuuzetsu/nix-project-defaults/blob/master...
- Dylan16807 12y ago>Guix is not late in that there aren't many purely functional package managers out there. Why does almost anyone not involved in writing it care what language the package manager is written in? >Cooperating with existing infrastructure is, I imagine, very difficult for a purely functional package manager, given the whole "purely functional" aspect. :-) I don't follow. Why is that difficult? >http://xkcd.com/927/ http://xkcd.com/927/ You can have different implementations of the same standard. To make your own standard is a step beyond.
- 12y ago
- agumonkey 12y agoPackage management may be defined but maybe not well named. More and more when I see package I see language stack, lexical scope and such... Maybe the Systems vs Languages split is detrimental to the solution here.
- dragonwriter 12y ago> Package management is probably, today, a very well defined problem. I don't think it is. I think "package management" is a name for a general class of solutions that addresses a range of problems (varying, at a minimum, along a continuum between "I want a way to pull a set of implementations of particular capabilities on any of a variety of different systems for which the appropriate implementations of those capabilities may be different" to "I want be able to reproduce from a list the same set of implementations across different machines".) There's also a range of priorities about certainty of results versus freedom to mix and match and ease of contributing to the globally accessible repository. I very much don't think the problem addressed by package management is "well-defined", and I'm not even sure it is coherent in the sense that were the problems different people look to package management to solve all collected, there would be a single workable solution that would address all of them optimally.
- stinkytaco 12y agoThe imperfect analogy I've used is the comparison between paints and pencils. They perform similar general functions (placing something on a media) and even use a similar set of tools (a long stick held in the hand). But the end result and even the goal could be totally different (I may want to hang a painting on a wall in a frame and I may want my pencil drawings to be reproduced a book). It's not perfect, but it helps non-users (i.e. managers) understand that one tool does not fit all.
- Doji 12y agoNix is attempting to be the solution you're looking for. Look at the nixpkgs repository: https://github.com/NixOS/nixpkgs https://github.com/NixOS/nixpkgs You will see packages for tons of stuff, including: * Python packages from PyPi: https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/python-packages.nix * Haskell packages from Hackage: https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/haskell-packages.nix * Perl packages from CPAN: https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/perl-packages.nix * NodeJS packages: https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/node-packages.nix * Lua Packages https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/lua-packages.nix All these packages in one place, all installed through the same system and automatically built and tested on the Hydra build farm. Nix users have developed tools which automate (as much as possible) the creation of these packages. I really believe NixOS is the future of the Linux distro, and it's tragic that it's still relatively obscure. It revolutionizes the OS in a myriad of ways, getting everything into the same package repository is just the beginning. For instance, it's fully configuration managed out of the box, using the Nix package manager as its configuration mangement system. Puppet, ansible, saltstack, chef, etc. are all made obsolete. Sure, it still has some rough edges, but it's a great community to contribute to and is improving at a impressive clip. If you're interested in the other ways NixOS is better than whatever you're currently using, I refer you to the website: https://nixos.org/ https://nixos.org/
- SSLy 12y agoJust one question: how much it takes to run Bumblebee, zsh and Steam on that?
- foobarqux 12y agoHow easy is it to build your own packages from, say, the tip of the master branch of github repository? How automated is building the dependencies? Building packages on Debian like systems is a huge pain. Arch is much better but I need Debian/Ubuntu-like reliability.
- Doji 12y agoVery easy. git clone git://github.com/NixOS/nixpkgs.git cd nixpkgs nix-build -A packagename Building the dependencies is completely automated. For more information, see the nixpkgs contributor's guide: http://nixos.org/nixpkgs/manual/ http://nixos.org/nixpkgs/manual/ Note that the above command will simply download cached binaries where they are available, but if you make any changes to the build scripts nix will notice rebuild locally instead. This gives you the benefits of both source based and binary based package managers.
- kachnuv_ocasek 12y ago>are we forever doomed to have multiple package managers for the dot product of [operating system]X[development platform]? Did you, by any chance, mean Cartesian or direct product?
- VLM 12y agoDon't forget the classic non package maintainer mistake of thinking policy = package file type/format. Its pretty trivial to "force" a rpm to install on a Debian box. That doesn't mean that magically any of it will follow Debian policy, any of the paths will be correct, any of the dependencies will be correct. But technically files can be thrown onto the filesystem, even if they can't actually be used where they land. And vice versa, if you shove a deb onto a Redhat box. My biggest "NIH" surprise is GNU has this and the emacs package system and at first glance they have nothing in common. The idea of installing a local copy of Rails or Leiningen off Emacs MELPA is kinda weird. I wonder if emacs twenty...something will use a user level version of this system. That would be interestingly consistent.
- taylanub 12y agoGuix is a big effort at making a purely functional package manager with a large recipe database and all kinds of bells and whistles you want in a full operating system. Emacs's minimal package manager is to install Elisp libraries (and nothing else) on every platform where Emacs runs, including MS Windows, and you can be happy it does dependency tracking. The two can't really be compared. Maybe in 10 years Guix will run on all platforms and have its components separated in such a way that it's sensible for Emacs to ship a version of Guix with it for Elisp package management, but don't hold your breath. So I don't think NIH applies there.
- rythie 12y agoI'm not sure it really matters that rpm and deb solve the same problem unless you use different distros. However npm, gem, pip, composer mostly solve the same problem, but a different one to rpm, in that we want per-project dependancies rather than per-system and want them to be commitable. Also many project combine languages so it would be good if these were combined.