6 ms·

> But if you want an existence proof: Maven. The Java library ecosystem has been going strong for 20 years, and during that time not once have we needed a lockf

> But if you want an existence proof: Maven. The Java library ecosystem has been going strong for 20 years, and during that time not once have we needed a lockfile. And we are pulling hundreds of libraries just to log two lines of text, so it is actively used at scale. Maven, by default, does not check your transitive dependencies for version conflicts. To do that, you need a frustrating plugin that produces much worse error messages than NPM does: https://ourcraft.wordpress.com/2016/08/22/how-to-read-maven-enforcer-plugins-requireupperbounddeps-rule-failure-report/ https://ourcraft.wordpress.com/2016/08/22/how-to-read-maven-.... How does Maven resolve dependencies when two libraries pull in different versions? It does something insane. https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html https://maven.apache.org/guides/introduction/introduction-to.... Do not pretend, for even half a second, that dependency resolution is not hell in maven (though I do like that packages are namespaced by creators, npm shoulda stolen that).

potetm 1y ago
The point isn't, "There are zero problems with maven. It solves all problems perfectly." The point is, "You don't need lockfiles." And that much is true. (Miss you on twitter btw. Come back!)
- jeltz 1y ago
  You don't need package management by the same token. C is proof of that. Having worked professionally in C, Java, Rust, Ruby, Perl, PHP I strongly prefer lock files. They make it so much nicer to manage dependencies.
  - potetm 1y ago
    "There is another tool that does exactly the job of a lockfile, but better." vs "You can use make to ape the job of dependency managers" wat?
    - jeltz 1y ago
      I have worked with Maven and dependency management is a pain. Not much nicer than vendoting dependencies like you do for C. When I first was introduced to lock files that was amazing. It solved so many problems I had with vendored dependencies, CPAN and Maven. Just because thousands of programmers manage to suffer through your bad system every day does not make it good.
    - aidenn0 1y ago
      Now you're moving the goalposts; I think lockfiles that are checked-in to version control are superior to Maven's "Let's YOLO it if your transitive dependencies conflict." Version ranges are more expressive than single-versions, and when you add lockfiles you get deterministic builds.
      - deepsun 1y ago
        I don't understand how Maven's YOLO is different from NPM's range. If you force a transitive dependency in Maven, then yes, some other library may get incompatible with it. But in NPM when people declare dependency as, say, ~1.2.3 the also don't know if they will be compatible with a future 1.2.4 version. They just _assume_ the next patch release won't break anything. Yes npm will try to find a version that satisfies all declarations, but library devs couldn't know the new version would be compatible because it wasn't published at that time. And my point is that it's _exactly_ the same probability that the next patch version is incompatible in both Maven and NPM. That's why NPM users are not afraid to depend on ~x.x or even ^x.x, they basically YOLOing.
        eptcyka 1y ago
        Yeah, npm people expect that semantic versioning will be abided by. Obviously, it will not work if a minor version bump introduces a breaking change. Obviously this is better than pinning the same one dependency in literally every package - imagine the churn and the amount of life lost to bumping dependencies in any given ecosystem if every package had to pin a specific version of a dependency. Ultimately, these are imperfect solutions to practical problems, and I know that I much prefer the semantic versioning and lockfile approach to whatever the java people are into.
- hyperpape 1y ago
  I think Maven's approach is functionally lock-files with worse ergonomics. You can only use the dependency from the libraries you use, but you're waiting for those libraries to update. As an escape hatch, you end up doing a lot of exclusions and overrides, basically creating a lockfile smeared over your pom. P.S. Sadly, I think enough people have left Twitter that it's never going to be what it was again.
  - potetm 1y ago
    Of course it's functionally lock files. They do the same thing! There's a very strong argument that manually managing deps > auto updating, regardless of the ergonomics. P.S. You're, right, but also it's where the greatest remnant remains. :(
    - shadowgovt 1y ago
      I fear it says something unfortunate about our entire subculture if the greatest remnant remains at the Nazi bar. :( (To be generous: it might be that we didn't build our own bar the moment someone who is at least Nazi-tolerant started sniffing around for the opportunity to purchas the deed to the bar. The big criticism might be "we, as a subculture, aren't punk-rock enough.")
      - LAC-Tech 1y ago
        If Twitter is a Nazi site then why do I have to block Laura Loomer & Ben Shapiro manually?
    - KerrAvon 1y ago
      JFC, get off Twitter. It's a Nazi propaganda site and you are going to be affected by that even if you think you're somehow immune.
  - Alupis 1y ago
    > P.S. Sadly, I think enough people have left Twitter that it's never going to be what it was again. Majority of those people came back after a while. The alternatives get near-zero engagement, so it's just shouting into the wind. For the ones that left over political reasons, receiving near-zero engagement takes all the fun out of posting... so they're back.
Karrot_Kream 1y ago
When I used to lead a Maven project I'd take dependency-upgrade tickets that would just be me bumping up a package version then whack-a-moling overrides and editing callsites to make dependency resolution not pull up conflicting packages until it worked. Probably lost a few days a quarter that way. I even remember the playlists I used to listen to when I was doing that work (: Lockfiles are great.
- RobRivera 1y ago
  > I even remember the playlists I used to listen to when I was doing that work (: Im a big fan of anything Aphex Twin for these type of sessions.
- yawaramin 1y ago
  How do lockfiles solve this problem? You would still have dependency-upgrade tickets and whack-a-mole, no? Or do you just never upgrade anything?
  - chowells 1y ago
    The difference is that the data is centralized with a single source of truth, and you have tools for working with it automatically. It doesn't mean lockfiles are cheap to update, but it does mean it's a much more streamlined process when it's time.
    - yawaramin 1y ago
      The data is also centralized without lockfiles though...it's in the package spec file itself, where all version can be upgraded together. If you are saying the only difference is some tooling for automation, that's a temporary problem, not a fundamental one.
      - growse 1y ago
        Without a lock file your transitive dependencies are.... by definition not centralised? Or have I misunderstood?
arcbyte 1y ago
I have YEARS of zero problems with maven dependencias. And yet i cant leave up a node project for more than a month without immediately encountering transitive dependency breakage that take days to resolve. Maven is dependency heaven.
- nemothekid 1y ago
  That's a problem with node's culture, not with lockfiles. I've never experienced the level of bitrot Node suffers from in Rust, Ruby, Go, or PHP, which do have lockfiles.
- creesch 1y ago
  You might not have issues, but they definitely happen. Especially once you bring in heavy frameworks like Spring, which for some reason ship with a ton of dependencies, some of which are surprisingly outdated. I had this happen with JUnit after a JDK upgrade. We needed to update to a newer major version of JUnit to match the new JDK, so we updated the test code accordingly. But then things broke. Methods were missing, imports couldn't be resolved, etc. Turned out something else in the dependency tree was still pulling in JUnit 4, and Maven's “nearest-wins” logic just silently went with the older version. No error, no warning. Just weird runtime/classpath issues. This something else turned out to be spring, for some odd reason it was an ancient version of Junit 4 as well. And yeah, you can eventually sort it out, maybe with mvn dependency:tree and a lot of manual overrides, but it is a mess. And Maven still doesn't give you anything like a lockfile to consistently reproduce builds over time. That's fine if your whole org pins versions very strictly, but it's naive to claim it "just works" in all cases. Certainly because versions often don't get pinned that strictly and it is easy to set up things in such a way that you think you have pinned the version while that isn't the case. Really fun stuff..
  - arcbyte 1y ago
    After more than a decade of working on spring projects in fortune 500 companies, I've literally never had this problem with Maven dependencies. I agree its theoretically possible, but I disagree that its difficult to resolve or somehow "hell". It works extremely well.
Bjartr 1y ago
Seems like there's room then in the Maven ecosystem that does what maven-enforcer-plugin does, but which just looks at a lockfile to make its decisions.
adrianmsmith 1y ago
If you use two dependencies, and one requires Foo 1.2.3 and the other Foo 1.2.4 then 99% of the time including either version of Foo will work fine. (I was a Java developer and used Maven for about 10 years.) For those times where that's not the case, you can look at the dependency tree to see which is included and why. You can then add a <dependency> override in your pom.xml file specifying the one you want. It's not an "insane" algorithm. It gives you predictability. If you write something in your pom.xml that overrides whatever dependency your dependency requires, because you can update your pom.xml if you need to. And because pom.xml is hand-written there are very few merge conflicts (as much as you'd normally find in source code), vs. a lock file where huge chunks change each time you change a dependency, and when it comes to a merge conflict you just have to delete the lot and redo it and hope nothing important has been changed.
- ffsm8 1y ago
  Depending on the dependency, you can also use shadow versions right? Essentially including both versions, and providing each dependency with it's own desired version. I believe it's done with the maven shade plug-in Never used it myself though, just read about it but never had an actual usecase
- zdragnar 1y ago
  > You can then add a <dependency> override in your pom.xml file specifying the one you want. Isn't that basically a crappy, hand-rolled equivalent to a lock file?
  - Muromec 1y ago
    Of course it is
  - smrtinsert 1y ago
    A single override does not equate to an entire lockfile of dependencies.
    - zdragnar 1y ago
xg15 1y ago
The irony is that it even has something like lockfiles as well: The <dependencyManagement> section: > Dependency management - this allows project authors to directly specify the versions of artifacts to be used when they are encountered in transitive dependencies or in dependencies where no version has been specified. It's just less convenient because you have to manage it yourself.
oftenwrong 1y ago
To add: if you would like to avoid depending on Maven's dependency mediation behaviour, then a useful tool is Maven Enforcer's dependencyConvergence rule. https://maven.apache.org/enforcer/enforcer-rules/index.html https://maven.apache.org/enforcer/enforcer-rules/index.html
kemitchell 1y ago
npm got around to `@{author}/{package}` and `@{org}/{package}` beyond just global `{package}`, albeit midstream, rather than very early on. The jargon is "scoped packages". I've seen more adoption recently, also with scopes for particular projects, like https://www.npmjs.com/package/@babel/core https://www.npmjs.com/package/@babel/core
- reactordev 1y ago
  The issue is what happens when libX@latest is updated and uses libd@2.0 but your other dependency libA@latest uses libd@1.3.1? In maven, crazy things happen. Sometimes it’s fine but if you have any kind of security, the version mismatch has different signatures and blows up. Ask any spring developer what happens when they have more than 1 slf4j in their classpath.
smrtinsert 1y ago
The java ecosystem never went through the level of pain node ecosystem has. For a while it was simply insanity in node. I've worked heavily in both, and the developer experience in java was always way better.
Perepiska 1y ago
The author of the article asked the same question in his personal blog in Telegram a year and a half ago [1], and received in response exactly the same example with maven. Probably, to build a personal brand and increase visibility, he needs to ask the same thing many times in different places. 1. https://t.me/nikitonsky_pub/597 https://t.me/nikitonsky_pub/597

potetm 1y ago

The point isn't, "There are zero problems with maven. It solves all problems perfectly." The point is, "You don't need lockfiles." And that much is true. (Miss you on twitter btw. Come back!)

jeltz 1y ago
You don't need package management by the same token. C is proof of that. Having worked professionally in C, Java, Rust, Ruby, Perl, PHP I strongly prefer lock files. They make it so much nicer to manage dependencies.
- potetm 1y ago
  "There is another tool that does exactly the job of a lockfile, but better." vs "You can use make to ape the job of dependency managers" wat?
  - jeltz 1y ago
    I have worked with Maven and dependency management is a pain. Not much nicer than vendoting dependencies like you do for C. When I first was introduced to lock files that was amazing. It solved so many problems I had with vendored dependencies, CPAN and Maven. Just because thousands of programmers manage to suffer through your bad system every day does not make it good.
  - aidenn0 1y ago
    Now you're moving the goalposts; I think lockfiles that are checked-in to version control are superior to Maven's "Let's YOLO it if your transitive dependencies conflict." Version ranges are more expressive than single-versions, and when you add lockfiles you get deterministic builds.
    - deepsun 1y ago
      I don't understand how Maven's YOLO is different from NPM's range. If you force a transitive dependency in Maven, then yes, some other library may get incompatible with it. But in NPM when people declare dependency as, say, ~1.2.3 the also don't know if they will be compatible with a future 1.2.4 version. They just _assume_ the next patch release won't break anything. Yes npm will try to find a version that satisfies all declarations, but library devs couldn't know the new version would be compatible because it wasn't published at that time. And my point is that it's _exactly_ the same probability that the next patch version is incompatible in both Maven and NPM. That's why NPM users are not afraid to depend on ~x.x or even ^x.x, they basically YOLOing.
      - eptcyka 1y ago
        Yeah, npm people expect that semantic versioning will be abided by. Obviously, it will not work if a minor version bump introduces a breaking change. Obviously this is better than pinning the same one dependency in literally every package - imagine the churn and the amount of life lost to bumping dependencies in any given ecosystem if every package had to pin a specific version of a dependency. Ultimately, these are imperfect solutions to practical problems, and I know that I much prefer the semantic versioning and lockfile approach to whatever the java people are into.
hyperpape 1y ago
I think Maven's approach is functionally lock-files with worse ergonomics. You can only use the dependency from the libraries you use, but you're waiting for those libraries to update. As an escape hatch, you end up doing a lot of exclusions and overrides, basically creating a lockfile smeared over your pom. P.S. Sadly, I think enough people have left Twitter that it's never going to be what it was again.
- potetm 1y ago
  Of course it's functionally lock files. They do the same thing! There's a very strong argument that manually managing deps > auto updating, regardless of the ergonomics. P.S. You're, right, but also it's where the greatest remnant remains. :(
  - shadowgovt 1y ago
    I fear it says something unfortunate about our entire subculture if the greatest remnant remains at the Nazi bar. :( (To be generous: it might be that we didn't build our own bar the moment someone who is at least Nazi-tolerant started sniffing around for the opportunity to purchas the deed to the bar. The big criticism might be "we, as a subculture, aren't punk-rock enough.")
    - LAC-Tech 1y ago
      If Twitter is a Nazi site then why do I have to block Laura Loomer & Ben Shapiro manually?
  - KerrAvon 1y ago
    JFC, get off Twitter. It's a Nazi propaganda site and you are going to be affected by that even if you think you're somehow immune.
- Alupis 1y ago
  > P.S. Sadly, I think enough people have left Twitter that it's never going to be what it was again. Majority of those people came back after a while. The alternatives get near-zero engagement, so it's just shouting into the wind. For the ones that left over political reasons, receiving near-zero engagement takes all the fun out of posting... so they're back.

Karrot_Kream 1y ago

When I used to lead a Maven project I'd take dependency-upgrade tickets that would just be me bumping up a package version then whack-a-moling overrides and editing callsites to make dependency resolution not pull up conflicting packages until it worked. Probably lost a few days a quarter that way. I even remember the playlists I used to listen to when I was doing that work (: Lockfiles are great.

RobRivera 1y ago
> I even remember the playlists I used to listen to when I was doing that work (: Im a big fan of anything Aphex Twin for these type of sessions.
yawaramin 1y ago
How do lockfiles solve this problem? You would still have dependency-upgrade tickets and whack-a-mole, no? Or do you just never upgrade anything?
- chowells 1y ago
  The difference is that the data is centralized with a single source of truth, and you have tools for working with it automatically. It doesn't mean lockfiles are cheap to update, but it does mean it's a much more streamlined process when it's time.
  - yawaramin 1y ago
    The data is also centralized without lockfiles though...it's in the package spec file itself, where all version can be upgraded together. If you are saying the only difference is some tooling for automation, that's a temporary problem, not a fundamental one.
    - growse 1y ago
      Without a lock file your transitive dependencies are.... by definition not centralised? Or have I misunderstood?

arcbyte 1y ago

I have YEARS of zero problems with maven dependencias. And yet i cant leave up a node project for more than a month without immediately encountering transitive dependency breakage that take days to resolve. Maven is dependency heaven.

nemothekid 1y ago
That's a problem with node's culture, not with lockfiles. I've never experienced the level of bitrot Node suffers from in Rust, Ruby, Go, or PHP, which do have lockfiles.
creesch 1y ago
You might not have issues, but they definitely happen. Especially once you bring in heavy frameworks like Spring, which for some reason ship with a ton of dependencies, some of which are surprisingly outdated. I had this happen with JUnit after a JDK upgrade. We needed to update to a newer major version of JUnit to match the new JDK, so we updated the test code accordingly. But then things broke. Methods were missing, imports couldn't be resolved, etc. Turned out something else in the dependency tree was still pulling in JUnit 4, and Maven's “nearest-wins” logic just silently went with the older version. No error, no warning. Just weird runtime/classpath issues. This something else turned out to be spring, for some odd reason it was an ancient version of Junit 4 as well. And yeah, you can eventually sort it out, maybe with mvn dependency:tree and a lot of manual overrides, but it is a mess. And Maven still doesn't give you anything like a lockfile to consistently reproduce builds over time. That's fine if your whole org pins versions very strictly, but it's naive to claim it "just works" in all cases. Certainly because versions often don't get pinned that strictly and it is easy to set up things in such a way that you think you have pinned the version while that isn't the case. Really fun stuff..
- arcbyte 1y ago
  After more than a decade of working on spring projects in fortune 500 companies, I've literally never had this problem with Maven dependencies. I agree its theoretically possible, but I disagree that its difficult to resolve or somehow "hell". It works extremely well.

Bjartr 1y ago

Seems like there's room then in the Maven ecosystem that does what maven-enforcer-plugin does, but which just looks at a lockfile to make its decisions.

adrianmsmith 1y ago

If you use two dependencies, and one requires Foo 1.2.3 and the other Foo 1.2.4 then 99% of the time including either version of Foo will work fine. (I was a Java developer and used Maven for about 10 years.) For those times where that's not the case, you can look at the dependency tree to see which is included and why. You can then add a <dependency> override in your pom.xml file specifying the one you want. It's not an "insane" algorithm. It gives you predictability. If you write something in your pom.xml that overrides whatever dependency your dependency requires, because you can update your pom.xml if you need to. And because pom.xml is hand-written there are very few merge conflicts (as much as you'd normally find in source code), vs. a lock file where huge chunks change each time you change a dependency, and when it comes to a merge conflict you just have to delete the lot and redo it and hope nothing important has been changed.

ffsm8 1y ago
Depending on the dependency, you can also use shadow versions right? Essentially including both versions, and providing each dependency with it's own desired version. I believe it's done with the maven shade plug-in Never used it myself though, just read about it but never had an actual usecase
zdragnar 1y ago
> You can then add a <dependency> override in your pom.xml file specifying the one you want. Isn't that basically a crappy, hand-rolled equivalent to a lock file?
- Muromec 1y ago
  Of course it is
- smrtinsert 1y ago
  A single override does not equate to an entire lockfile of dependencies.
  - zdragnar 1y ago

xg15 1y ago

The irony is that it even has something like lockfiles as well: The <dependencyManagement> section: > Dependency management - this allows project authors to directly specify the versions of artifacts to be used when they are encountered in transitive dependencies or in dependencies where no version has been specified. It's just less convenient because you have to manage it yourself.

oftenwrong 1y ago

To add: if you would like to avoid depending on Maven's dependency mediation behaviour, then a useful tool is Maven Enforcer's dependencyConvergence rule. https://maven.apache.org/enforcer/enforcer-rules/index.html https://maven.apache.org/enforcer/enforcer-rules/index.html

kemitchell 1y ago

npm got around to `@{author}/{package}` and `@{org}/{package}` beyond just global `{package}`, albeit midstream, rather than very early on. The jargon is "scoped packages". I've seen more adoption recently, also with scopes for particular projects, like https://www.npmjs.com/package/@babel/core https://www.npmjs.com/package/@babel/core

reactordev 1y ago
The issue is what happens when libX@latest is updated and uses libd@2.0 but your other dependency libA@latest uses libd@1.3.1? In maven, crazy things happen. Sometimes it’s fine but if you have any kind of security, the version mismatch has different signatures and blows up. Ask any spring developer what happens when they have more than 1 slf4j in their classpath.

smrtinsert 1y ago

The java ecosystem never went through the level of pain node ecosystem has. For a while it was simply insanity in node. I've worked heavily in both, and the developer experience in java was always way better.

Perepiska 1y ago

The author of the article asked the same question in his personal blog in Telegram a year and a half ago [1], and received in response exactly the same example with maven. Probably, to build a personal brand and increase visibility, he needs to ask the same thing many times in different places. 1. https://t.me/nikitonsky_pub/597 https://t.me/nikitonsky_pub/597