5 ms·

Package management is hard. Ruby gems has had its issues and Go has got itself into a holy war just deciding how best to do dependency management. However over

Package management is hard. Ruby gems has had its issues and Go has got itself into a holy war just deciding how best to do dependency management. However over 2018 we seem to have got to a point where I’m worrying about some basic professional competencies and communication systems inside the npm project. I don’t know much about it, but looking at it from a sideways glance, you have to start worrying about what it is that they’re getting wrong, and how to derail their mistakes in my own team and workflows.

pmlnr 8y ago
> Package management is hard. and this is why writing yet another package manager instead of gradually improving existing ones - rpm, deb, etc. - is far from ideal.
- TheCoreh 8y ago
  rpm and deb are tied to specific operating system distributions, and are also meant to install packages globally on a system. npm, on the other hand, is meant to be cross-platform and is primarily focused on installing dependencies on a per-project basis. Its rules for lookup, matching, deduplication etc are mostly defined or informed by Node.JS conventions. I don't think apt or rpm would be willing to cover these use cases, and even if they did, it would probably not be worth it
  - LoSboccacc 8y ago
    the maven model has worked quite well so far.
    - codefined 8y ago
      I'm not sure if it's a model thing or just that software has bugs, but lots of people really dislike Maven. Not sure it's an example of a good package manager.
      - jrs95 8y ago
        The CLI is among the least intuitive I’ve ever used, and pom.xml is pretty verbose and complex. That being said, it works very well and I’ve never had a problem with it (other than it’s aesthetics) after grokking it.
      - peeters 8y ago
        Do they hate the package management or the build tool? Maven introduced a new package format (the Project Object Model, pom.xml) at the same time as introducing a very declarative build tool. The really confusing part is that the two are conflated by design: if building with Maven, your project's pom.xml also contains the declarative Maven configuration used to build your package, instead of containing just the package metadata (name, dependencies, etc). But that's not required, and it need not be part of the *.pom XML file deployed in the published package. Since release, many build tools have been introduced that can consume and produce POM-compatible packages but don't require you to use Maven itself as your builder (Ivy for Ant, gradle for writing build scripts in Groovy, SBT for Scala, etc).
        _asummers 8y ago
        Just a note: you can use Gradle for Java too. I haven't built a Java project without Gradle since like 2010 or so.
        peeters 8y ago
        Yeah sorry if I wasn't clear, in Gradle the build script is written in Groovy. It is used to build any number of project types.
        vorg 8y ago
        Gradle build scripts can also be written in Kotlin.
      - tlarkworthy 8y ago
        I spent yesterday trying to get protobuffers working in maven, see http://vlkan.com/blog/post/2015/11/27/maven-protobuf/ http://vlkan.com/blog/post/2015/11/27/maven-protobuf/ for the pain. Anything counter to maven's way is a PITA. In this case, Maven dislikes platform specific binaries.
    - realusername 8y ago
      Technically yes, on the usability part there's still a long way to go...
  - pmlnr 8y ago
    > globally on a system In the world of containers? Come on.
    - solatic 8y ago
      You understand that you can install a deb/rpm inside a container, right? The conservation of complexity is at play here. Either you have a quasi-complex deb/rpm, and a simple Dockerfile with a RUN command installing it, or you have a quasi-complex Dockerfile which does all the packaging work itself.
      - pmlnr 8y ago
        > You understand that you can install a deb/rpm inside a container, right? That's exactly why I replied. deb/rpm in container is a reasonable choice, and that way, they aren't global for the system but local in the container.
- dEnigma 8y ago
  Unless the existing ones would require radical changes to realize your vision, which would be hard to do in a gradual manner and probably lead to more problems than just writing something from scratch.
- JepZ 8y ago
  Well, package management often starts too simple. Sadly, most people who build new ones seem to ignore the lessons from earlier projects. In my opinion, Paludis/Cave [1] is a decent example of which cases have to be handled, but as it uses the ebuild/exheres system it is slow as hell. The user experience isn't that good either, but at least they know how to do proper dependency management (e.g. keeping the system as stable as possible during updates by using a reasonable order). For anybody who wants to build a package manager I highly recommend to take a look at what Paludis does (not necessarily how they do it). [1]: https://paludis.exherbo.org https://paludis.exherbo.org
  - filmor 8y ago
    I didn't use cave, but I remember using Paludis instead of Portage on Gentoo about 15 years ago, and it was really really fast at the actual package management tasks. The builds themselves take time of course, and especially for smaller packages the "./configure" part is infuriating, but that's nothing a package manager can fix.
    - JepZ 8y ago
      In comparison to Portage it is fast, but I wouldn't consider Portage a benchmark ;-) and if you compare Paludis with Apt/Yum/Pacman and the likes it is still very slow (e.g. comparing installing binary packages). The comparison isn't completely fair, as Paludis has a much better dependency resolution algorithm (in terms of quality), but I think the main difference comes from the package format. Ebuilds/exheres are Bash scripts which need to be evaluated in order to obtain basic information like for example build- or run-time dependencies. So to find out which packages should be updated, Paludis has to evaluate a few thousand bash scripts while other package formats allow much quicker algorithms. Cave is just a newer front-end for Paludis which improves the user experience a bit, but the Performance should be pretty much the same.
  - digi_owl 8y ago
    > Sadly, most people who build new ones seem to ignore the lessons from earlier projects. This applies to FOSS in general, and then some...
hinkley 8y ago
I’m two bug fixes away from strongarming my team into moving to yarn. Npm.org needs an io.js sized attitude adjustment. The number of showstopper bugs in the last five releases is fucking bullshit.
jonny_eh 8y ago
I suspect that the problems NPM face are not qualitatively different from other similar systems but NPM is far larger so gets a lot more press.
- repolfx 8y ago
  No. The Maven ecosystem is also massive and I don't remember the last time I heard about problems with it.

pmlnr 8y ago

> Package management is hard. and this is why writing yet another package manager instead of gradually improving existing ones - rpm, deb, etc. - is far from ideal.

TheCoreh 8y ago
rpm and deb are tied to specific operating system distributions, and are also meant to install packages globally on a system. npm, on the other hand, is meant to be cross-platform and is primarily focused on installing dependencies on a per-project basis. Its rules for lookup, matching, deduplication etc are mostly defined or informed by Node.JS conventions. I don't think apt or rpm would be willing to cover these use cases, and even if they did, it would probably not be worth it
- LoSboccacc 8y ago
  the maven model has worked quite well so far.
  - codefined 8y ago
    I'm not sure if it's a model thing or just that software has bugs, but lots of people really dislike Maven. Not sure it's an example of a good package manager.
    - jrs95 8y ago
      The CLI is among the least intuitive I’ve ever used, and pom.xml is pretty verbose and complex. That being said, it works very well and I’ve never had a problem with it (other than it’s aesthetics) after grokking it.
    - peeters 8y ago
      Do they hate the package management or the build tool? Maven introduced a new package format (the Project Object Model, pom.xml) at the same time as introducing a very declarative build tool. The really confusing part is that the two are conflated by design: if building with Maven, your project's pom.xml also contains the declarative Maven configuration used to build your package, instead of containing just the package metadata (name, dependencies, etc). But that's not required, and it need not be part of the *.pom XML file deployed in the published package. Since release, many build tools have been introduced that can consume and produce POM-compatible packages but don't require you to use Maven itself as your builder (Ivy for Ant, gradle for writing build scripts in Groovy, SBT for Scala, etc).
      - _asummers 8y ago
        Just a note: you can use Gradle for Java too. I haven't built a Java project without Gradle since like 2010 or so.
        peeters 8y ago
        Yeah sorry if I wasn't clear, in Gradle the build script is written in Groovy. It is used to build any number of project types.
        vorg 8y ago
        Gradle build scripts can also be written in Kotlin.
    - tlarkworthy 8y ago
      I spent yesterday trying to get protobuffers working in maven, see http://vlkan.com/blog/post/2015/11/27/maven-protobuf/ http://vlkan.com/blog/post/2015/11/27/maven-protobuf/ for the pain. Anything counter to maven's way is a PITA. In this case, Maven dislikes platform specific binaries.
  - realusername 8y ago
    Technically yes, on the usability part there's still a long way to go...
- pmlnr 8y ago
  > globally on a system In the world of containers? Come on.
  - solatic 8y ago
    You understand that you can install a deb/rpm inside a container, right? The conservation of complexity is at play here. Either you have a quasi-complex deb/rpm, and a simple Dockerfile with a RUN command installing it, or you have a quasi-complex Dockerfile which does all the packaging work itself.
    - pmlnr 8y ago
      > You understand that you can install a deb/rpm inside a container, right? That's exactly why I replied. deb/rpm in container is a reasonable choice, and that way, they aren't global for the system but local in the container.
dEnigma 8y ago
Unless the existing ones would require radical changes to realize your vision, which would be hard to do in a gradual manner and probably lead to more problems than just writing something from scratch.
JepZ 8y ago
Well, package management often starts too simple. Sadly, most people who build new ones seem to ignore the lessons from earlier projects. In my opinion, Paludis/Cave [1] is a decent example of which cases have to be handled, but as it uses the ebuild/exheres system it is slow as hell. The user experience isn't that good either, but at least they know how to do proper dependency management (e.g. keeping the system as stable as possible during updates by using a reasonable order). For anybody who wants to build a package manager I highly recommend to take a look at what Paludis does (not necessarily how they do it). [1]: https://paludis.exherbo.org https://paludis.exherbo.org
- filmor 8y ago
  I didn't use cave, but I remember using Paludis instead of Portage on Gentoo about 15 years ago, and it was really really fast at the actual package management tasks. The builds themselves take time of course, and especially for smaller packages the "./configure" part is infuriating, but that's nothing a package manager can fix.
  - JepZ 8y ago
    In comparison to Portage it is fast, but I wouldn't consider Portage a benchmark ;-) and if you compare Paludis with Apt/Yum/Pacman and the likes it is still very slow (e.g. comparing installing binary packages). The comparison isn't completely fair, as Paludis has a much better dependency resolution algorithm (in terms of quality), but I think the main difference comes from the package format. Ebuilds/exheres are Bash scripts which need to be evaluated in order to obtain basic information like for example build- or run-time dependencies. So to find out which packages should be updated, Paludis has to evaluate a few thousand bash scripts while other package formats allow much quicker algorithms. Cave is just a newer front-end for Paludis which improves the user experience a bit, but the Performance should be pretty much the same.
- digi_owl 8y ago
  > Sadly, most people who build new ones seem to ignore the lessons from earlier projects. This applies to FOSS in general, and then some...

hinkley 8y ago

I’m two bug fixes away from strongarming my team into moving to yarn. Npm.org needs an io.js sized attitude adjustment. The number of showstopper bugs in the last five releases is fucking bullshit.

jonny_eh 8y ago

I suspect that the problems NPM face are not qualitatively different from other similar systems but NPM is far larger so gets a lot more press.

repolfx 8y ago
No. The Maven ecosystem is also massive and I don't remember the last time I heard about problems with it.