5 ms·

So many useful tools are packaged in language-specific package management schemes, and they are a pain to deal with in conjunction with a system package manager

So many useful tools are packaged in language-specific package management schemes, and they are a pain to deal with in conjunction with a system package manager. I'm speaking of nice command-line applications that work like Unix utilities, that simply parse STDIN and write to STDOUT, but for some reason I have to install PyPI/NPM/gem/whatever to install them, and figure out what language-specific idiosyncrasies I have to massage to get them to work (venv, rvm, and whatnot). Like the author, I want language implementers to split the packaging realm: anything that's a pure source dependency can and should be packaged with language-specific tools, but anything that runs on the system should be distributed in some sort of universally executable format, such as a launcher shell script with an application directory in /usr/bin. Otherwise you're in this current mess where anyone writing a system package manager has to deal with language runtimes and fight with the language-specific package manager to handle distribution/installation/removal/rollback/versioning in a coherent way with the rest of their platform. Moreover, the user has to make the decision between installing a package from their system vendor or from the language-specific repository. So I guess what I'm advocating for is for the "language package manager (LPM)" concept to go away. If you need to distribute source dependencies, that should be handled on a per-project and not per-language-environment or per-system basis. If you need runtime dependencies, distribute them as a tarball in a standard directory format and let system vendors package them like the rest of the software they manage. The JVM world has good things going with Maven and Gradle. I never have to use some Java command-line package manager to install a global dependency; rather, I'm declaring what my project needs and letting the build tools handle the rest. I never have to deal with conflicting dependency versions between projects or language versions. And binary Java applications are easily available via my system's package manager. There is just no need for the middle layer.

klodolph 11y ago
What would you say is the difference then, between Maven / Gradle and say, NPM? I haven't used JVM tools, but what you are describing sounds like how NPM works. You specify that your project needs library X, version 4.5, in your package.json file and then NPM handles the rest.
- matzipan 11y ago
  Used both, difference is non-existant. Not sure what parent means.
- dlandis 11y ago
  It is similar, but you need to have npm installed first don't you? With gradle you typically interact with it through a shell script (aka gradle wrapper) and that downloads the correct version of gradle if it isn't already present in the project directory and then gradle downloads the dependencies. The person who sets up the project would generate the wrapper script using gradle.
  - aikah 11y ago
    > It is similar, but you need to have npm installed first don't you? Same thing for gradle, you need to install it at first place. There is absolutely no difference except the language that is being used. And yes you can use shell scripts with npm too, except your shell scripts aren't going to run on windows.
    - dikaiosune 11y ago
      Does npm have this? https://docs.gradle.org/current/userguide/gradle_wrapper.html https://docs.gradle.org/current/userguide/gradle_wrapper.htm... I think that's what GP was referring to -- it's not a shell script for automating use of the tool, it's a shell script for automating acquisition and initialization of the tool.
      - aikah 11y ago
        When you fetch a package with npm, the package will either be : a library(installed locally) or a command line tool (installed globally). You can also run any number of tasks per project (npm run <task>), described in the npm manifest, or execute tasks before or after installing, and before publishing a library.
        11y ago
cgm616 11y ago
These were my thoughts entirely while reading the article. If I want a command line utility, I really don't want to install npm then mess with any problems with that. I want to simply 'brew install' everything like that and be done. Sure, if I need the source of something for compilation or testing, i.e. 'pip install numpy', then having that handled by pip + virtualenv makes sense. Besides that use case, brew should handle everything to reduce conflicts.
_pmf_ 11y ago
> The JVM world has good things going with Maven and Gradle. Gradle internally uses Maven. Leiningen (the Clojure build tool) also uses Maven, and the nice thing here (while being rife with other problems) is that it pulls the correct language version in as a dependency (i.e. it pulls in Clojure in the required version).
lmm 11y ago
The language package manager can have so much richer information than the system one, is the thing. The system package manager only understands a crude, lowest-common-denominator notion of dependencies, and everything goes via the system repo which is never going to be remotely up-to-date with the language package repo. Finally most system package managers won't let you have multiple versions of a library installed at all, which makes using more than 2 programs at the same time virtually impossible (because they usually want different versions of some dependent library or other).
- davexunit 11y ago
  >The language package manager can have so much richer information than the system one, is the thing. This is simply not true. Language package managers cannot describe the entire dependency graph of a piece of software because they can't handle things that aren't written in that language. For example, there are plenty of Ruby gems that require a C compiler and C shared libraries in order to work, but those dependencies cannot be encoded in that system. System package managers can describe the entire graph, thus providing richer information. >Finally most system package managers won't let you have multiple versions of a library installed at all, which makes using more than 2 programs at the same time virtually impossible (because they usually want different versions of some dependent library or other). Yes, this is a big issue with the status quo system package managers. Fortunately, there's hope on the horizon in the form of functional package management. Functional package managers such as GNU Guix and Nix allow for an arbitrary number of variants of the same software to coexist without conflict. Additional benefits include unprivileged package management, transactional upgrades and roll backs, reproducible builds, and no central point of trust for binaries (anything can be built from source). Language package managers are good for quickly sharing code written purely in that language with other developers, but unacceptable for anything else.
  - sdboyer 11y ago
    > Language package managers are good for quickly sharing code written purely in that language with other developers, but unacceptable for anything else. Right, so, what almost all of the FLOSS world is doing, almost all of the time. Miiiight be worth optimizing for that case. Especially if a proper PDM would compose well with larger systems...which it would!
drewcrawford 11y ago
The problem is that, as a software developer, I am not going to learn 3 PMs to distribute my free software (yum, apt, homebrew) nor am I going to navigate the politics of Fedora, Debian, Ubuntu, etc. for getting my free software included in their repositories and/or keeping it current. I have things to do, like write software. Language PMs are "good enough" to solve the problems of the people actually writing the software, and in most cases that is really all that matters. I'm sorry they're not good enough for you, but it's not my fault that Rust, Python, and Go will let me upload to their package manager but Debian/Fedora/whatever you use will not. Your fight is not with the language PMs, it is with your OS vendor who will neither give me upload rights nor send out someone else who does, who neither designs tools for me to use nor sends someone who already knows how to use them. To be clear, I think there are good reasons why OS vendors won't change their ways, but software developers won't change our ways either, so neither language PMs nor OS PMs are going away any time soon.
- ciupicri 11y ago
  No one is stopping you from creating your own yum or apt repository. For Fedora you could use COPR [1]: > Copr is an easy-to-use automatic build system providing a package repository as its output. > Start with making your own repository in these three steps: 1) choose an architecture and system you want to build for; 2) provide Copr with src.rpm packages available online; 3) let Copr do all the work and wait for your new repo. [1]: https://copr.fedorainfracloud.org/ https://copr.fedorainfracloud.org/
  - lbotos 11y ago
    You missed his point. As a software level developer if he uses "his language package manager" that's where his responsibility stops. In your example, he'd make a package for yum or apt but what about brew what about $obscure_other_one? It's an abstraction layer.
- tadfisher 11y ago
  The responsibility to package software at a system level isn't yours, it's the system vendor's. Let them hash out a standard packaging model while you ship your plain tarball or provide `configure && make && make install` instructions.
digi_owl 11y ago
One more reason for me to enjoy Gobolinux i guess. The people in charge there have either implemented or plan to implement support for those language specific managers into the Gobolinux tree. As best i understand it, this by making the Compile tool able to use these managers to download, and then capture the output and place it into the proper place in the /Programs tree.

klodolph 11y ago

What would you say is the difference then, between Maven / Gradle and say, NPM? I haven't used JVM tools, but what you are describing sounds like how NPM works. You specify that your project needs library X, version 4.5, in your package.json file and then NPM handles the rest.

matzipan 11y ago
Used both, difference is non-existant. Not sure what parent means.
dlandis 11y ago
It is similar, but you need to have npm installed first don't you? With gradle you typically interact with it through a shell script (aka gradle wrapper) and that downloads the correct version of gradle if it isn't already present in the project directory and then gradle downloads the dependencies. The person who sets up the project would generate the wrapper script using gradle.
- aikah 11y ago
  > It is similar, but you need to have npm installed first don't you? Same thing for gradle, you need to install it at first place. There is absolutely no difference except the language that is being used. And yes you can use shell scripts with npm too, except your shell scripts aren't going to run on windows.
  - dikaiosune 11y ago
    Does npm have this? https://docs.gradle.org/current/userguide/gradle_wrapper.html https://docs.gradle.org/current/userguide/gradle_wrapper.htm... I think that's what GP was referring to -- it's not a shell script for automating use of the tool, it's a shell script for automating acquisition and initialization of the tool.
    - aikah 11y ago
      When you fetch a package with npm, the package will either be : a library(installed locally) or a command line tool (installed globally). You can also run any number of tasks per project (npm run <task>), described in the npm manifest, or execute tasks before or after installing, and before publishing a library.
      - 11y ago

cgm616 11y ago

These were my thoughts entirely while reading the article. If I want a command line utility, I really don't want to install npm then mess with any problems with that. I want to simply 'brew install' everything like that and be done. Sure, if I need the source of something for compilation or testing, i.e. 'pip install numpy', then having that handled by pip + virtualenv makes sense. Besides that use case, brew should handle everything to reduce conflicts.

_pmf_ 11y ago

> The JVM world has good things going with Maven and Gradle. Gradle internally uses Maven. Leiningen (the Clojure build tool) also uses Maven, and the nice thing here (while being rife with other problems) is that it pulls the correct language version in as a dependency (i.e. it pulls in Clojure in the required version).

lmm 11y ago

The language package manager can have so much richer information than the system one, is the thing. The system package manager only understands a crude, lowest-common-denominator notion of dependencies, and everything goes via the system repo which is never going to be remotely up-to-date with the language package repo. Finally most system package managers won't let you have multiple versions of a library installed at all, which makes using more than 2 programs at the same time virtually impossible (because they usually want different versions of some dependent library or other).

davexunit 11y ago
>The language package manager can have so much richer information than the system one, is the thing. This is simply not true. Language package managers cannot describe the entire dependency graph of a piece of software because they can't handle things that aren't written in that language. For example, there are plenty of Ruby gems that require a C compiler and C shared libraries in order to work, but those dependencies cannot be encoded in that system. System package managers can describe the entire graph, thus providing richer information. >Finally most system package managers won't let you have multiple versions of a library installed at all, which makes using more than 2 programs at the same time virtually impossible (because they usually want different versions of some dependent library or other). Yes, this is a big issue with the status quo system package managers. Fortunately, there's hope on the horizon in the form of functional package management. Functional package managers such as GNU Guix and Nix allow for an arbitrary number of variants of the same software to coexist without conflict. Additional benefits include unprivileged package management, transactional upgrades and roll backs, reproducible builds, and no central point of trust for binaries (anything can be built from source). Language package managers are good for quickly sharing code written purely in that language with other developers, but unacceptable for anything else.
- sdboyer 11y ago
  > Language package managers are good for quickly sharing code written purely in that language with other developers, but unacceptable for anything else. Right, so, what almost all of the FLOSS world is doing, almost all of the time. Miiiight be worth optimizing for that case. Especially if a proper PDM would compose well with larger systems...which it would!

drewcrawford 11y ago

The problem is that, as a software developer, I am not going to learn 3 PMs to distribute my free software (yum, apt, homebrew) nor am I going to navigate the politics of Fedora, Debian, Ubuntu, etc. for getting my free software included in their repositories and/or keeping it current. I have things to do, like write software. Language PMs are "good enough" to solve the problems of the people actually writing the software, and in most cases that is really all that matters. I'm sorry they're not good enough for you, but it's not my fault that Rust, Python, and Go will let me upload to their package manager but Debian/Fedora/whatever you use will not. Your fight is not with the language PMs, it is with your OS vendor who will neither give me upload rights nor send out someone else who does, who neither designs tools for me to use nor sends someone who already knows how to use them. To be clear, I think there are good reasons why OS vendors won't change their ways, but software developers won't change our ways either, so neither language PMs nor OS PMs are going away any time soon.

ciupicri 11y ago
No one is stopping you from creating your own yum or apt repository. For Fedora you could use COPR [1]: > Copr is an easy-to-use automatic build system providing a package repository as its output. > Start with making your own repository in these three steps: 1) choose an architecture and system you want to build for; 2) provide Copr with src.rpm packages available online; 3) let Copr do all the work and wait for your new repo. [1]: https://copr.fedorainfracloud.org/ https://copr.fedorainfracloud.org/
- lbotos 11y ago
  You missed his point. As a software level developer if he uses "his language package manager" that's where his responsibility stops. In your example, he'd make a package for yum or apt but what about brew what about $obscure_other_one? It's an abstraction layer.
tadfisher 11y ago
The responsibility to package software at a system level isn't yours, it's the system vendor's. Let them hash out a standard packaging model while you ship your plain tarball or provide `configure && make && make install` instructions.

digi_owl 11y ago

One more reason for me to enjoy Gobolinux i guess. The people in charge there have either implemented or plan to implement support for those language specific managers into the Gobolinux tree. As best i understand it, this by making the Compile tool able to use these managers to download, and then capture the output and place it into the proper place in the /Programs tree.