8 ms·

We have this discussion literally every time. Different languages have different conventions and build systems, and packaging a library for every operating syst

by eudox 12y ago

We have this discussion literally every time. Different languages have different conventions and build systems, and packaging a library for every operating system package manager is just not feasible. A single package, for a single package manager that runs everywhere and is tightly integrated with the language, is the only reasonable solution.

dubcanada 12y ago
I see where he is coming from though. When I look at github repos and they have more random config files in their home directory then actual source files it starts to scare me.
pnathan 12y ago
IMO,a multiplexing system should be done: a stack emits a given standard output. Which then gets read by a operating system package builder and ginned into the OS standard format.
- TylerE 12y ago
  That's never going to work. How is yum or apt ever going to grok the 5+ python virtualenvs I'm working with at any given time?
tikhonj 12y ago
My point is not to use the system package manager, but to replace the various language-specific ones with, say, Nix. It should be flexible enough to work with the different languages' build systems and conventions, while reusing all the existing systems for managing dependencies, conflicts and so on. Very little of all that logic is language specific, and it's traditionally done poorly by language-specific package managers. Basically, I do agree on using a single package manager. I just don't think that the need to be integrated with the language is severe enough to warrant writing a new one for each language! It's much less effort for everyone involved to set integration up for an existing package manager instead. In other words: replace Pip/Cabal/Babel/whatever with Nix, maybe with some plugins for the needs of your specific language. You would even still ship it with your language (optionally), so the only change would be maintaining a plugin, if necessary, instead of a whole manager.
- munificent 12y ago
  > to replace the various language-specific ones with, say, Nix. So now, to use packages for programming language X, I also have to install Nix, be running a Unix-based OS (sorry Windows folks!), and have whatever other dependencies and toolchain Nix requires installed. This is the core problem. A programming language needs a package manager today. Code reuse is too important for a language to succeed with out a nice user experience to get and share code and deal with transitive dependencies. That puts a package manager on the critical path for a language. Given that, it's important that the package manager itself have minimal dependencies. It would suck, for example, if a package manager for Python was written in Ruby and required Ruby to be installed! The only thing most languages' package managers are willing to depend on is the language itself and its core libraries. That means for every programming language, you get a new package manager. It's not ideal, but it seems to be the best solution we've found given those constraints.
  - pmahoney 12y ago
    I don't really disagree with your points, but "the best solution we've found given those constraints" include a lot of pain. Something like Nix comes with a lot of different pain, but I suspect many pain points could be lessened if language-specific package managers were designed with consideration for more generic tools. For example, rubygems cannot properly express dependencies on C libraries. Rubygems that include C extensions run the compilation step during "gem install" [1] (which typically assumes the presence of a C compiler, etc.). A gem like libxml-ruby assumes the presence of libxml development headers. (Nokogiri punted and includes an entire libxml source tree in its gem which leads to its own problems.) Nix has some means to convert rubygems into nix packages, and it has a list of overlay dependencies to apply to various common gems (e.g. libxml-ruby gets enhanced with that dependency on libxml). Language-specific package managers handle simple cases well, but other cases make increasingly large assumptions about the external world that complicate automation and confound repeatability. [1] There is support for binary gems, but I don't know of anyone who uses these in practice, nor do I know of a repository of pre-compiled gems.
  - untothebreach 12y ago
    One idea I had was to leverage nix channels to use Nix as the package manager _only for language packages_. Instead of downloading the default set of packages, you subscribe to the "Wren" (;-)) channel with `$ nix-channel --add http://github.com/wren/nixpkgs` http://github.com/wren/nixpkgs` and now you have all the local packages for your language. It could basically "be" the PyPI or RubyGems for a language.
rdtsc 12y ago
> A single package, for a single package manager that runs everywhere and is tightly integrated with the language, is the only reasonable solution. Well yeah if your product is uni-language. If it is not then you are installing python pips, ruby gems, java jars, make && make installs for C++ code. And some stuff is uni-language, granted. A lot of Github demo projects and libraries are uni-langue And then they might have cross-dependencies, some have transitive dependencies. And you might have to uninstall some old package and install new one, except you can't make make & make install just dumps files on the file system and doesn't know how to properly clean up after itself. Then in many cases uni-platform is common. That calls for using the OS package manager. Say if your services sit on tops of CentOS 6.5 then why run around sprinkling files all over the file systems via shell scripts or re-implement an adhoc package manager in bash when you can just use RPMs. I have seen this all too many time "RPM writers are stupid, look at this complexity, I'll do it quicker". In the end they come up with something messier, less reliable, not-quite-working-right re-implementation of RPMs.

by eudox 12y ago

dubcanada 12y ago

I see where he is coming from though. When I look at github repos and they have more random config files in their home directory then actual source files it starts to scare me.

pnathan 12y ago

IMO,a multiplexing system should be done: a stack emits a given standard output. Which then gets read by a operating system package builder and ginned into the OS standard format.

TylerE 12y ago
That's never going to work. How is yum or apt ever going to grok the 5+ python virtualenvs I'm working with at any given time?

tikhonj 12y ago

My point is not to use the system package manager, but to replace the various language-specific ones with, say, Nix. It should be flexible enough to work with the different languages' build systems and conventions, while reusing all the existing systems for managing dependencies, conflicts and so on. Very little of all that logic is language specific, and it's traditionally done poorly by language-specific package managers. Basically, I do agree on using a single package manager. I just don't think that the need to be integrated with the language is severe enough to warrant writing a new one for each language! It's much less effort for everyone involved to set integration up for an existing package manager instead. In other words: replace Pip/Cabal/Babel/whatever with Nix, maybe with some plugins for the needs of your specific language. You would even still ship it with your language (optionally), so the only change would be maintaining a plugin, if necessary, instead of a whole manager.

munificent 12y ago
> to replace the various language-specific ones with, say, Nix. So now, to use packages for programming language X, I also have to install Nix, be running a Unix-based OS (sorry Windows folks!), and have whatever other dependencies and toolchain Nix requires installed. This is the core problem. A programming language needs a package manager today. Code reuse is too important for a language to succeed with out a nice user experience to get and share code and deal with transitive dependencies. That puts a package manager on the critical path for a language. Given that, it's important that the package manager itself have minimal dependencies. It would suck, for example, if a package manager for Python was written in Ruby and required Ruby to be installed! The only thing most languages' package managers are willing to depend on is the language itself and its core libraries. That means for every programming language, you get a new package manager. It's not ideal, but it seems to be the best solution we've found given those constraints.
- pmahoney 12y ago
  I don't really disagree with your points, but "the best solution we've found given those constraints" include a lot of pain. Something like Nix comes with a lot of different pain, but I suspect many pain points could be lessened if language-specific package managers were designed with consideration for more generic tools. For example, rubygems cannot properly express dependencies on C libraries. Rubygems that include C extensions run the compilation step during "gem install" [1] (which typically assumes the presence of a C compiler, etc.). A gem like libxml-ruby assumes the presence of libxml development headers. (Nokogiri punted and includes an entire libxml source tree in its gem which leads to its own problems.) Nix has some means to convert rubygems into nix packages, and it has a list of overlay dependencies to apply to various common gems (e.g. libxml-ruby gets enhanced with that dependency on libxml). Language-specific package managers handle simple cases well, but other cases make increasingly large assumptions about the external world that complicate automation and confound repeatability. [1] There is support for binary gems, but I don't know of anyone who uses these in practice, nor do I know of a repository of pre-compiled gems.
- untothebreach 12y ago
  One idea I had was to leverage nix channels to use Nix as the package manager _only for language packages_. Instead of downloading the default set of packages, you subscribe to the "Wren" (;-)) channel with `$ nix-channel --add http://github.com/wren/nixpkgs` http://github.com/wren/nixpkgs` and now you have all the local packages for your language. It could basically "be" the PyPI or RubyGems for a language.

rdtsc 12y ago

> A single package, for a single package manager that runs everywhere and is tightly integrated with the language, is the only reasonable solution. Well yeah if your product is uni-language. If it is not then you are installing python pips, ruby gems, java jars, make && make installs for C++ code. And some stuff is uni-language, granted. A lot of Github demo projects and libraries are uni-langue And then they might have cross-dependencies, some have transitive dependencies. And you might have to uninstall some old package and install new one, except you can't make make & make install just dumps files on the file system and doesn't know how to properly clean up after itself. Then in many cases uni-platform is common. That calls for using the OS package manager. Say if your services sit on tops of CentOS 6.5 then why run around sprinkling files all over the file systems via shell scripts or re-implement an adhoc package manager in bash when you can just use RPMs. I have seen this all too many time "RPM writers are stupid, look at this complexity, I'll do it quicker". In the end they come up with something messier, less reliable, not-quite-working-right re-implementation of RPMs.