10 ms·

We shouldn't have needed lockfiles

• ratelimitsteve 1y ago
  anyone find a way to get rid of the constantly shifting icons at the bottom of the screen? I'm trying to read and the motion keeps pulling my attention away from the words toward the dancing critters.
  • bencevans 1y ago
    https://times.hntrends.net/story/44813397 https://times.hntrends.net/story/44813397
  • foobarbecue 1y ago
    $("#presence").remove() And yeah, I did that right away. Fun for a moment but extremely distracting.
  • deleted 1y ago
    [deleted]
  • vvillena 1y ago
    Reader mode.
  • karmakurtisaani 1y ago
    Agreed. It's an absolutely useless feature for me to see as well.
  • trinix912 1y ago
    Block ###presence with UBlock.
  • zahlman 1y ago
    I use NoScript, which catches all of these sorts of things by default. I only enable first-party JS when there's a clear good reason why the site should need it, and third-party JS basically never beyond NoScript's default whitelist.
  • yawaramin 1y ago
    https://github.com/t-mart/kill-sticky https://github.com/t-mart/kill-sticky
• deleted 1y ago
  [deleted]
• Joker_vD 1y ago
  NPM has, starting with version 0.5.1, an absolutely lovely feature where it simply ignores the package-lock.json file altogether. Or to be more precise, "npm install" regenerates package-lock.json based on package.json. What's the point of "npm upgrade" then? Eh.
  • thunderfork 1y ago
    You can use `npm ci` for "don't update the lockfile, fail if an exact lockfile match can't be collected"
  • 0cf8612b2e1e 1y ago
    I hate this reality.
  • deleted 1y ago
    [deleted]
• omnicognate 1y ago
  What if your project also uses librupa, which also depends on liblupa? Follow the chain of reasoning from that thought, or maybe spend a couple of decades dealing with the horror created by people who didn't, and you'll get to lockfiles.
• hyperpape 1y ago
  > But if you want an existence proof: Maven. The Java library ecosystem has been going strong for 20 years, and during that time not once have we needed a lockfile. And we are pulling hundreds of libraries just to log two lines of text, so it is actively used at scale. Maven, by default, does not check your transitive dependencies for version conflicts. To do that, you need a frustrating plugin that produces much worse error messages than NPM does: https://ourcraft.wordpress.com/2016/08/22/how-to-read-maven-enforcer-plugins-requireupperbounddeps-rule-failure-report/ https://ourcraft.wordpress.com/2016/08/22/how-to-read-maven-.... How does Maven resolve dependencies when two libraries pull in different versions? It does something insane. https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html https://maven.apache.org/guides/introduction/introduction-to.... Do not pretend, for even half a second, that dependency resolution is not hell in maven (though I do like that packages are namespaced by creators, npm shoulda stolen that).
  • potetm 1y ago
    The point isn't, "There are zero problems with maven. It solves all problems perfectly." The point is, "You don't need lockfiles." And that much is true. (Miss you on twitter btw. Come back!)
    • jeltz 1y ago
      You don't need package management by the same token. C is proof of that. Having worked professionally in C, Java, Rust, Ruby, Perl, PHP I strongly prefer lock files. They make it so much nicer to manage dependencies.
      • potetm 1y ago
        "There is another tool that does exactly the job of a lockfile, but better." vs "You can use make to ape the job of dependency managers" wat?
        • jeltz 1y ago
          I have worked with Maven and dependency management is a pain. Not much nicer than vendoting dependencies like you do for C. When I first was introduced to lock files that was amazing. It solved so many problems I had with vendored dependencies, CPAN and Maven. Just because thousands of programmers manage to suffer through your bad system every day does not make it good.
• simonw 1y ago
  I see lockfiles as something you use for applications you are deploying - if you run something like a web app it's very useful to know exactly what is being deployed to production, make sure it exactly matches staging and development environments, make sure you can audit new upgrades to your dependencies etc. This article appears to be talking about lockfiles for libraries - and I agree, for libraries you shouldn't be locking exact versions because it will inevitably pay havoc with other dependencies. Or maybe I'm missing something about the JavaScript ecosystem here? I mainly understand Python.
  • kaelwd 1y ago
    The lockfile only applies when you run `npm install` in the project directory, other projects using your package will have their own lockfile and resolve your dependencies using only your package.json.
  • aidenn0 1y ago
    I think you missed the point of the article. Consider Application A, that depends on Library L1. Library L1 in turn depends on Library L2: A -> L1 -> L2 They are saying that A should not need a lockfile because it should specify a single version of L1 in its dependencies (i.e. using an == version check in Python), which in turn should specify a single version of L2 (again with an == version check). Obviously if everybody did this, then we wouldn't need lockfiles (which is what TFA says). The main downsides (which many comments here point out) are: 1. Transitive dependency conflicts would abound 2. Security updates are no longer in the hands of the app developers (in my above example, the developer of A1 is dependent on the developer of L1 whenever a security bug happens in L2). 3. When you update a direct dependency, your transitive dependencies may all change, making what you that was a small change into a big change. (FWIW, I put these in order of importance to me; I find #3 to be a nothingburger, since I've hardly ever updated a direct dependency without it increasing the minimum dependency of at least one of its dependencies).
    • hosh 1y ago
      Is the article also suggesting that if there are version conflicts, it goes with the top level library? For example, if we want to use a secure version of L2, it would be specified at A, ignoring the version specified by L1? Or maybe I misread the article and it did not say that.
• lalaithion 1y ago
  What if your program depends on library a1.0 and library b1.0, and library a1.0 depends on c2.1 and library b1.0 depends on c2.3? Which one do you install in your executable? Choosing one randomly might break the other library. Installing both _might_ work, unless you need to pass a struct defined in library c from a1.0 to b1.0, in which case a1.0 and b1.0 may expect different memory layouts (even if the public interface for the struct is the exact same between versions). The reason we have dependency ranges and lockfiles is so that library a1.0 can declare "I need >2.1" and b1.0 can declare "I need >2.3" and when you depend on a1.0 and b1.0, we can do dependency resolution and lock in c2.3 as the dependency for the binary.
  • tonsky 1y ago
    One of the versions will be picked up. If that version doesn’t work, you can try another one. The process is exactly the same
    • Joker_vD 1y ago
      > If that version doesn’t work, you can try another one. And how will this look like, if your app doesn't have library C mentioned in its dependencies, only libraries A and B? You are prohibited from answering "well, just specify all the transitive dependencies manually" because it's precisely what a lockfile is/does.
      • tonsky 1y ago
        Maven's version resolution mechanism determines which version of a dependency to use when multiple versions are specified in a project's dependency tree. Here's how it works: - Nearest Definition Wins: When multiple versions of the same dependency appear in the dependency tree, the version closest to your project in the tree will be used. - First Declaration Wins: If two versions of the same dependency are at the same depth in the tree, the first one declared in the POM will be used.
        • Joker_vD 1y ago
• egh 1y ago
  we've all learned about things, not understood them, and thought "wow, these people must be idiots. why would they have made this complicated thing? makes no sense whatsoever. I can't believe these people, idiots, never thought this through like I have." Most of us, fortunately, don't post these thoughts to the internet for anybody to read.
  • zahlman 1y ago
    > Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. While I share the view that TFA is misguided in some ways, this isn't a productive or insightful way to make the point.
  • tonsky 1y ago
    I worked for 20 years in an ecosystem that didn’t have lockfiles and had reproducible builds before the term was invented, and now you come and tell me that it couldn’t be?
    • spooky_deep 1y ago
      Where did the packages come from? How did you know they hadn’t been changed at the bit-level since last time?
• boscillator 1y ago
  Ok, but what happens when lib-a depends on lib-x:0.1.4 and lib-b depends on lib-x:0.1.5, even though it could have worked with any lib-x:0.1.*? Are these libraries just incompatible now? Lockfiles don't guarantee that new versions are compatible, but it guarantees that if your code works in development, it will work in production (at least in terms of dependencies). I assume java gets around this by bundling libraries into the deployed .jar file. That this is better than a lock file, but doesn't make sense for scripting languages that don't have a build stage. (You won't have trouble convincing me that every language should have a proper build stage, but you might have trouble convincing the millions of lines of code already written in languages that don't.)
  • aidenn0 1y ago
    > I assume java gets around this by bundling libraries into the deployed .jar file. That this is better than a lock file, but doesn't make sense for scripting languages that don't have a build stage. (You won't have trouble convincing me that every language should have a proper build stage, but you might have trouble convincing the millions of lines of code already written in languages that don't.) You are wrong; Maven just picks one of lib-x:0.1.4 or lib-x:0.1.5 depending on the ordering of the dependency tree.
    • yladiz 1y ago
      How do you change the order?
      • adrianmsmith 1y ago
        You go into your pom.xml file (bunch of <dependency>) using a text editor and change the order.
    • Tadpole9181 1y ago
      Maven will also silently choose different minor and major versions, destroying your application. Sometimes at compile time, sometimes at runtime. Java dependency management is unhinged, antiquated garbage to anyone who has used any other ecosystem.
• epage 1y ago
  Let's play this out in a compiled language like Cargo. If every dependency was a `=` and cargo allowed multiple versions of SemVer compatible packages. The first impact will be that your build will fail. Say you are using `regex` and you are interacting with two libraries that take a `regex::Regex`. All of the versions need to align to pass `Regex` between yourself and your dependencies. The second impact will be that your builds will be slow. People are already annoyed when there are multiple SemVer incompatible versions of their dependencies in their dependency tree, now it can happen to any of your dependencies and you are working across your dependency tree to get everything aligned. The third impact is if you, as the application developer, need a security fix in a transitive dependency. You now need to work through the entire bubble up process before it becomes available to you. Ultimately, lockfiles are about giving the top-level application control over their dependency tree balanced with build times and cross-package interoperability. Similarly, SemVer is a tool any library with transitive dependencies [0] [0] https://matklad.github.io/2024/11/23/semver-is-not-about-you.html https://matklad.github.io/2024/11/23/semver-is-not-about-you...
  • hosh 1y ago
    Wasn’t the article suggesting that the top level dependencies override transitive dependencies, and that could be done in the main package file instead of the lock file?
    • epage 1y ago
      That was only in an update, not in the original text.
  • junon 1y ago
    You should not be editing your cargo.lock file manually. Cargo gives you a first-class way of overriding transitive dependencies.
    • richardwhiuk 1y ago
      You can also do cargo update -p
• trjordan 1y ago
  There is absolutely a good reason for version ranges: security updates. When I, the owner of an application, choose a library (libuseful 2.1.1), I think it's fine that the library author uses other libraries (libinsecure 0.2.0). But in 3 months, libinsecure is discovered (surprise!) to be insecure. So they release libinsecure 0.2.1, because they're good at semver. The libuseful library authors, meanwhile, are on vacation because it's August. I would like to update. Turns out libinsecure's vulnerability is kind of a big deal. And with fully hardcoded dependencies, I cannot, without some horrible annoying work like forking/building/repackaging libuseful. I'd much rather libuseful depend on libinsecure 0.2.*, even if libinsecure isn't terribly good at semver. I would love software to be deterministically built. But as long as we have security bugs, the current state is a reasonable compromise.
  • tonsky 1y ago
    It’s totally fine in Maven, no need to rebuild or repackage anything. You just override version of libinsecure in your pom.xml and it uses the version you told it to
    • zahlman 1y ago
      So you... manually re-lock the parts you need to?
      • aidenn0 1y ago
        Don't forget the part where Maven silently picks one version for you when there are transitive dependency conflicts (and no, it's not always the newest one).
      • deredede 1y ago
        Sure, I'm happy with locking the parts I need to lock. Why would I lock the parts I don't need to lock?
        • skywhopper 1y ago
          Because you can’t know which ones you “need” to lock.
• alexandrehtrb 1y ago
  I completely agree. .NET doesn't have lock files either, and its dependency tree runs great. Using fixed versions for dependencies is a best practice, in my opinion.
  • horsawlarway 1y ago
    This is wrong. DotNet uses packages.lock.json explicitly to support the case where you want to be able to lock transitive dependencies that are specified with a range value, or several other edge cases that might warrant explicitly declaring versions that are absent from csproj or sln files. https://devblogs.microsoft.com/dotnet/enable-repeatable-package-restores-using-a-lock-file/ https://devblogs.microsoft.com/dotnet/enable-repeatable-pack... https://learn.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files#enabling-the-lock-file https://learn.microsoft.com/en-us/nuget/consume-packages/pac... Again - there's no free lunch here.
    • alexandrehtrb 1y ago
      Practically no one uses that.
• andix 1y ago
  Lockfiles are essential for somewhat reproducible builds. If a transient dependency (not directly referenced) updates, this might introduce different behavior. if you test a piece of software and fix some bugs, the next build shouldn't contain completely different versions of dependencies. This might introduce new bugs.
  • tonsky 1y ago
    > Lockfiles are essential for somewhat reproducible builds. No they are not. Fully reproducible builds have existed without lockfiles for decades
    • its-summertime 1y ago
      of distros, they usually refer to an upstream by hash https://src.fedoraproject.org/rpms/conky/blob/rawhide/f/sources https://src.fedoraproject.org/rpms/conky/blob/rawhide/f/sour... also of flathub https://github.com/flathub/com.belmoussaoui.ashpd.demo/blob/bf73e97322c7130a023b1d81fe5b8d766bffe492/com.belmoussaoui.ashpd.demo.json#L62 https://github.com/flathub/com.belmoussaoui.ashpd.demo/blob/... "they are not lockfiles!" is a debatable separate topic, but for a wider disconnected ecosystem of sources, you can't really rely on versions being useful for reproducibility
      • andix 1y ago
        > they usually refer to an upstream by hash exactly the same thing as a lockfile
    • andix 1y ago
      Sure, without package managers. It's also not about fully reproducible builds, it's about a tradeoff to get modern package manger (npm, cargo, ...) experience and also somewhat reproducible builds.
      • chriswarbo 1y ago
        > modern package manger (npm, cargo, ...) experience Lol, the word "modern" has truly lost all meaning. Your list of "modern package managers" seems to coincide with a list of legacy tooling I wrote four years ago! https://news.ycombinator.com/item?id=29459209 https://news.ycombinator.com/item?id=29459209
• zokier 1y ago
  umm wat > “But Niki, you can regenerate the lockfile and pull in all the new dependencies!” > Sure. In exactly the same way you can update your top-level dependencies. how does updating top-level deps help with updating leaf packages? Is the author assuming that whenever a leaf package is updated, every other package in the dep chain gets immediately new release? That is fundamentally impossible considering that the releases would need to happen serially.
  • tonsky 1y ago
    I updated the post, see near the bottom
• nine_k 1y ago
  The author seems to miss the point of version ranges. Yes, specific versions of dependencies get frozen in the lock file at the moment of building. But the only way to determine these specific versions is to run version resolution across the whole tree. The process finds out which specific versions within the ranges can be chosen to satisfy all the version constraints. This works with minimal coordination between authors of the dependencies. It becomes a big deal when you have several unrelated dependencies, each transitively requiring that libpupa. The chance they converge on the same exact version is slim. The chance a satisfying version can be found within specified ranges is much higher. Physical things that are built from many parts have the very same limitation: they need to specify tolerances to account for the differences in production, and would be unable to be assembled otherwise.
  • tonsky 1y ago
    Yeah but version ranges are fiction. Some says: we require libpupa 0.2.0+. Sure you can find a version in that range. But what if it doesn’t work? How can you know that your library will work with all the future libpupa releases in advance?
    • nine_k 1y ago
      More often than not things are compatible within a major version. Very often things are compatible within a minor version. Not being able to build because one thing depends on libpupa 1.2.34.pre5 and another, on 1.2.35 would be a worse outcome, on average.
    • mystifyingpoi 1y ago
      It reminds me of the whole mess of Angular 2+ upgrades. It was I believe before lockfiles in npm? Literally every new person joining the team had to get the node_modules handed to them from someone else's machine for the project to work, since `npm install` could never install anything working together.
    • wpollock 1y ago
      Under semver, any dependency version X.Y.* is supposed to be compatible with any software that was built with version X.Z.* when Y > Z. If not, the author of the dependency has broken semver. "Supposed to" being the operative phrase. This is of little comfort when you need version X.Y for a security fix but your build breaks. Note that Maven is more complex than others here have mentioned. In some cases, Maven compares versions lexically (e.g. version 1.2 is considered newer than version 1.10). Dependency management is indeed hell.
• freetonik 1y ago
  In the world of Python-based end-user libraries the pinned (non-ranged) versions result in users being unable to use your library in an environment with other libraries. I’d love to lock my library to numpy 2.3.4, but if the developers of another library pin theirs to 2.3.5 then game over. For server-side or other completely controlled environments the only good reason to have lock files is if they are actually hashed and thus allow to confirm security audits. Lock files without hashes do not guarantee security (depending on the package registry, of course, but at least in Python world (damn it) the maintainer can re-publish a package with an existing version but different content).
  • deleted 1y ago
    [deleted]
  • tonsky 1y ago
    > I’d love to lock my library to numpy 2.3.4, but if the developers of another library pin theirs to 2.3.5 then game over. Why? Can’t you specify which version to use?
• spooky_deep 1y ago
  > The important point of this algorithm is that it’s fully deterministic. The algorithm can be deterministic, but fetching the dependencies of a package is not. It is usually an HTTP call to some endpoint that might flake out or change its mind. Lock files were invented to make it either deterministic or fail. Even with Maven, deterministic builds (such as with Bazel) lock the hashes down. This article is mistaken.
  • Tainnor 1y ago
    Maven artifacts are immutable, so the whole resolution is deterministic (even if hard to understand), unless you're using snapshot versions (which are mutable) or you use version ranges (which is rare in the Maven world).
    • mystifyingpoi 1y ago
      I never understood this. I can delete anything from Nexus and reupload something else in its place. Is this supposed immutability just a convention that's followed?
      • Tainnor 1y ago
        Yes, you can, if you manually interact with it. Most people don't do that because why would you? You generally use it as a proxy for maven central and similar and to upload your own internal artifacts through maven-publish. I'm not sure if it's the standard setting, but Nexus should typically be configured to prevent you from overwriting non-snapshot versions.
    • spooky_deep 1y ago
      If your model is that you trust Maven to never change anything, then sure. However, I think most people in the reproducible build space would consider Maven an external uncontrolled input.
      • Tainnor 1y ago
        > I think most people in the reproducible build space would consider Maven an external uncontrolled input In an academic sense, you're probably right. In practice it turns out that this isn't an issue in 99% of cases. Yes, I have once run into a weird issue where Nexus was corrupted and it took some debugging, so it's not like it can't happen, but assuming you don't do anything weird, the assumption that Maven artifacts are immutable is fairly safe. I'm not saying that lockfiles aren't technically superior or anything, but the failure modes are so rare that people usually don't bother (even in Gradle where lockfiles are technically supported).
• horsawlarway 1y ago
  This is a great example of chesterton's fence. The author of this piece doesn't understand why a top level project might want control of its dependencies dependencies. That's the flaw in this whole article, if you can't articulate why it's important to be able to control those... don't write an article. You don't understand the problem space. Semantic versioning isn't perfect, but it's more than a "hint", and it sure as hell beats having to manually patch (or fork) an entire dependency chain to fix a security problem.
  • aidenn0 1y ago
    Author puts up Maven as an example of no lockfiles. Maven does allow a top-level project to control its transitive dependencies (when there is a version conflict, the shallowest dependency wins; the trivial version of this is if you specify it as a top-level dependency). I think rather that the author doesn't realize that many people in the lockfile world put their lockfiles under version control. Which makes builds reproducible again.
    • horsawlarway 1y ago
      Yes, but Maven doesn't support reproducibility (outside of plugins that basically haul in a lockfile). So his whole point is moot (Gradle now does, as an aside: https://docs.gradle.org/current/userguide/dependency_locking.html https://docs.gradle.org/current/userguide/dependency_locking...) Again - I don't think the author is aware enough of the problem space to be making the sort of claim that he is. He doesn't understand the problem lockfiles are solving, so he doesn't know why they exist and wants them gone... chesterton's fence in action. --- Directly declaring deps is great. It's so great that we'd like to do it for every dependency in many (arguably most) cases. But doing that really sort of sucks when you start getting into even low 10s of deps. Enter... lockfiles and the tooling to auto-resolve them.
      • oftenwrong 1y ago
        >Maven doesn't support reproducibility I don't think that is true: https://maven.apache.org/guides/mini/guide-reproducible-builds.html https://maven.apache.org/guides/mini/guide-reproducible-buil...
• palotasb 1y ago
  The author is perhaps presenting a good argument for languages/runtimes like JavaScript/Node where dependencies may be isolated and conflicting dependencies may coexist in the dependency tree (e.g., "app -> { libpupa 1.2.3 -> liblupa 0.7.8 }, { libxyz 2.0 -> liblupa 2.4.5 }" would be fine), but the proposed dependency resolution algorithm... > Our dependency resolution algorithm thus is like this: > 1. Get the top-level dependency versions > 2. Look up versions of libraries they depend on > 3. Look up versions of libraries they depend on ...would fail in languages like Python where dependencies are shared, and the steps 2, 3, etc. would result in conflicting versions. In these languages, there is good reason to define dependencies in a relaxed way (with constraints that exclude known-bad versions; but without pins to any specific known-to-work version and without constraining only to existing known-good versions) at first. This way dependency resolution always involves some sort of constraint solving (with indeterminate results due to the constraints being open-ended), but then for the sake of reproducibility the result of the constraint solving process may be used as a lockfile. In the Python world this is only done in the final application (the final environment running the code, this may be the test suite in for a pure library) and the pins in the lock aren't published for anyone to reuse. To reiterate, the originally proposed algorithm doesn't work for languages with shared dependencies. Using version constraints and then lockfiles as a two-layer solution is a common and reasonable way of resolving the dependency topic in these languages.
  • hosh 1y ago
    What if the top level can override the transitive dependencies? I have had to do that with Ruby apps, where libraries are also shared.
  • tonsky 1y ago
    > would fail in languages like Python where dependencies are shared And yet Java and Maven exist...
    • deleted 1y ago
      [deleted]
• zahlman 1y ago
  > Imagine you voluntarily made your build non-reproducible by making them depend on time. If I build my app now, I get libpupa 1.2.3 and liblupa 0.7.8. If I repeat the same build in 10 minutes, I’ll get liblupa 0.7.9. Crazy, right? That would be chaos. No; in fact it's perfectly reasonable, and at the core of what the author doesn't seem to get. Developers have motivations other than reproducibility. The entire reason we have version number schemes like this is so that we can improve our code while also advertising reasonable expectations about compatibility. If we have dependents, then hopefully this also improves their UX indirectly — whether by taking advantage of optimizations we made, not encountering bugs that were actually our fault, etc. Similarly, if we have dependencies, we can seek to take advantage of that. Upgrading environments is an opportunity to test new configurations, and see if they're any better than what's in existing lockfiles. > But this is what version ranges essentially are. Instead of saying “libpupa 1.2.3 depends on liblupa 0.7.8”, they are saying “libpupa 1.2.3 depends on whatever the latest liblupa version is at the time of the build.” But also, developers aren't necessarily using the latest versions of their dependencies locally anyway. If I did pin a version in my requirements, it'd be the one that I tested the build with, not necessarily the one that was most recently released at the time of the build. Not everyone runs an industrial-strength CI system, and for the size of lots of useful packages out there, they really shouldn't have to, either. (And in the pathological case, someone else could re-release while I'm building and testing!) > But... why would libpupa’s author write a version range that includes versions that don’t exist yet? How could they know that liblupa 0.7.9, whenever it will be released, will continue to work with libpupa? Surely they can’t see the future? Semantic versioning is a hint, but it has never been a guarantee. The thing about this is that "work with [a dependency]" is not really a binary. New versions also fix things — again, that's the main reason that new versions get released in the first place. Why would I keep writing the software after it's "done" if I don't think there's anything about it that could be fixed? For that matter, software packages break for external reasons. If I pin my dependency, and that dependency is, say, a wrapper for a third-party web API, and the company operating that website makes a breaking change to the API, then I just locked myself out of new versions of the dependency that cope with that change. In practice, there are good reasons to not need a guarantee and accept the kind of risk described. Lockfiles exist for those who do need a guarantee that their local environment will be set in concrete (which has other, implicit risks). I see it as much like personal finance. Yes, investments beyond a HISA may carry some kind of risk. This is worthwhile for most people. And on the flip side, you also can't predict the future inflation rate, and definitely can't predict what will happen to the price of the individual goods and services you care about most. > The funny thing is, these version ranges end up not being used anyway. You lock your dependencies once in a lockfile and they stay there, unchanged. You don’t even get the good part! ??? What ecosystem is this author talking about? Generating a lockfile doesn't cause the underlying dependency metadata to disappear. You "get the good part" as a developer by periodically regenerating a lockfile, testing the resulting environment and shipping the new lock. Or as a user by grabbing a new lockfile, or by just choosing not to use provided lockfiles. > “But Niki, you can regenerate the lockfile and pull in all the new dependencies!” Sure. In exactly the same way you can update your top-level dependencies. Has the author tried both approaches, I wonder? Not to mention: the lockfile-less world the author describes, would require everyone to pin dependency versions. In practice, this would require dropping support for anything else in the metadata format. And (I did have to look it up) this appears to be the world of Maven that gets cited at the end (cf. https://stackoverflow.com/questions/44521542 https://stackoverflow.com/questions/44521542). I like choice and freedom in my software, thank you. > “But Niki, lockfiles help resolve version conflicts!” In what way? Version conflicts don’t happen because of what’s written in dependency files. Perhaps the author hasn't worked in an ecosystem where people routinely attempt to install new packages into existing environments? Or one where users don't want to have multiple point versions of the same dependency downloaded and installed locally if one of them would satisfy the requirements of other software? Or where dependency graphs never end up having "diamonds"? (Yes, there are package managers that work around this, but not all programming languages can sanely support multiple versions of the same dependency in the same environment.)
• jedberg 1y ago
  Tangential, but what is up with all those flashing icons at the bottom of the page? It made it nearly unreadable.
  • jofla_net 1y ago
    it looks like who's reading the page , and from where. just my guess.
• imtringued 1y ago
  I disagree with this blogpost in its entirety. Lockfiles are neither unnecessary, nor are they complicated. The argument presented against lockfiles boils down to a misrepresentation. I also dislike the presentation using the godawful yellow color and the stupid websocket gadget in the footer. The entire point of lockfiles is to let the user decide when the version resolution algorithm should execute and when it shouldn't. That's all they do and they do it exactly as promised.
  • mrspuratic 1y ago
    OMG is it full of yellow. And there I was wondering how one might manage a modem or a mailqueue without a lock file: these are not your father's lockfiles[1] [1] https://linux.die.net/man/1/lockfile https://linux.die.net/man/1/lockfile
• hosh 1y ago
  Many comments talk about how top-level and transitive dependencies can conflict. I think the article is suggesting you can resolve those by specifying them in the top-level packages and overriding any transitive package versions. If we are doing that anyways, it circles back to if lock files are necessary. Given that, I still see some consequences: The burden for testing if a library can use its dependency falls back on the application developer instead of the library developer. A case could be made that, while library developers should test what their libraries are compatible with, the application developer has the ultimate responsibility for making sure everything can work together. I also see that there would need to be tooling to automate resolutions. If ranges are retained, the resolver needs to report every conflict and force the developer to explicitly specify the version they want at the top-level. Many package managers automatically pick one and write it into the lock file. If we don’t have lock files, and we want it to be automatic, then we can have it write to the top level package manager and not the lock file. That creates its own problems. One of those problems comes from humans and tooling writing to the same configuration file. I have seen problems with that idea pop up — most recently, letting letsencrypt modify nginx configs, and now I have to manually edit those. Letsencrypt can no longer manage them. Arguably, we can also say LLMs can work with that, but I am a pessimist when it comes to LLM capabilities. So in conclusion, I think the article writer’s reasoning is sound, but incomplete. Humans don’t need lockfiles, but our tooling need lockfiles until it is capable of working with the chaos of human-managed package files.
  • hosh 1y ago
    If the dependencies are specified as data, such as package.json, or a yaml or xml file, it may be structured enough that tools can still manage it. Npm install has a save flag that lets you do that. Python dep files may be structured enough to do this as well. If the package specification file is code and not data, then this becomes more problematic. Elixir specified dep as data within code. Arguably, we can add code to read and write from a separate file… but at that point, those might as well be lock files.
  • skybrian 1y ago
    Not quite. Library authors are expected to specify the version that they tested with. (Or the oldest version that they tested with that works.) If it's an old release of a library then it will specify old dependencies, but you just have to deal with that yourself, because library authors aren't expected to have a crystal ball that tells them which future versions will be broken or have security holes.
• 10000truths 1y ago
  When discoverability and versioning of libraries is more-or-less standardized (a la Cargo/PyPI/NPM), automated tooling for dependency resolution/freezing follows naturally. The build tools for Java and C historically did not have this luxury, which is why their ecosystems have a reputation for caring a lot about backwards compatibility.
• andy99 1y ago
  In case the author is reading, I can't read your article because of that animation at the bottom. I get it, it's cute, but it makes it too distracting to concentrate on the article, so I ended up just closing it.
  • somehnguy 1y ago
    I read the article but that animation was incredibly distracting. I don't even understand what it's for - clicking it does nothing. Best guess is a representation of how many people active on page.
  • fennecbutt 1y ago
    It also covers a whole 1/4 of the screen on mobile...
    • Aaargh20318 1y ago
      It covers 90% of the screen on iPad
  • fellowniusmonk 1y ago
    I've never seen something so egregious before, it made it impossible to read without covering it with my hand. But I realized something by attempting to read this article several times first. If I ever want to write an article and reduce peoples ability to critically engage with the argument in it I should add a focus pulling animation that thwarts concerted focus. It's like the blog equivalent of public speakers who ramble their audience into a coma.
    • deepsun 1y ago
      https://neal.fun/stimulation-clicker/ https://neal.fun/stimulation-clicker/
    • IrishTechie 1y ago
      > I've never seen something so egregious before Streaming comments on YouTube give it a run for its money, what absolute garbage.
• kaptainscarlet 1y ago
  I somewhat agree because the main package file .eg package.json can act as a lock file if you pin packages to specific versions
  • whilenot-dev 1y ago
    No tag other than latest has any special significance to npm itself. Tags can be republished and that's why integrity checks should be in place. Supply chain attacks are happening in open source communities, sadly.
    • beart 1y ago
      I don't think you can republish to npm. https://docs.npmjs.com/cli/v11/commands/npm-publish https://docs.npmjs.com/cli/v11/commands/npm-publish > The publish will fail if the package name and version combination already exists in the specified registry. > Once a package is published with a given name and version, that specific name and version combination can never be used again, even if it is removed with npm unpublish.
      • whilenot-dev 1y ago
        > if the package name and version combination already exists I was talking about tags above, eg. "npm i react@next", and you can use tags in your package.json. npm allows you to republish them at will, and you can never force your users to use a specific version.
• bunjeejmpr 1y ago
  This metadata should be in the top of your source as documentation. We need the metadata. Not a new container.
• wedn3sday 1y ago
  I absolutely abhor the design of this site. I cannot engage with the content as Im filled with a deep burning hatred of the delivery. Anyone making a personal site: do not do this.
• throwaway984393 1y ago
  [dead]
• ConanRus 1y ago
  [dead]
• shadowgovt 1y ago
  This author's approach would probably work "fine" (1) for something like npm, where individual dependencies also have a subtree of their dependencies (and, by extension, "any situation where dependencies are statically linked"). It doesn't work at all for something like Python. In Python, libpupa 1.2.3 depends on liblupa 0.7.8. But libsupa 4.5.6 depends on liblupa 0.7.9. Since the Python environment can only have one version of each module at a time, I need to decide on a universe in which libpupa and libsupa can both have their dependencies satisfied simultaneously. Version ranges give me multiple possible universes, and then for reproducibility (2) I use a lockfile to define one. (1) npm's dependencies-of-dependencies design introduces its own risks and sharp edges. liblupa has a LupaStuff object in it. It changed very subtly between v0.7.8 and v0.7.9, so subtly that the author didn't think to bump the minor version. And that's okay, because both libpupa and libsupa should be wrapping their dependent objects in an opaque interface anyway; they shouldn't be just barfing liblupa-generated objects directly-accessible into their client code. Oh, you think people actually encapsulate like that? You're hilarious. So eventually, a LupaStuff generated by libpupa is going to get passed to libsupa, which is actually expecting a subtly different object. Will it work? Hahah, who knows! Python actually avoids this failure mode by forcing one coherent environment; since 'pupa and 'supa have to be depending on the same 'lupa (without very fancy module shenanigans), you can have some expectation that their LupaStuff objects will be compatible. (2) I think the author is hitting on something real though, which is that semantic versioning is a convention, not a guarantee; nobody really knows if your code working with 0.7.8 implies it will work with 0.7.9. It should. Will it? "Cut yourself and find out." In an ideal world, every dependency-of-a-dependency pairing has been hand-tested by someone before it gets to you; in practice, individual software authors are responsible for one web of dependencies, and the Lockfile is a candle in the darkness: "Well, it worked on my machine in this configuration."
• broken_broken_ 1y ago
  I agree with the premise, just use a specific version of your dependencies, that’s generally fine. However: You absolutely do need a lock file to store a cryptographic hash of each dependency to ensure that what is fetched has not been tampered with. And users are definitely not typing a hash when adding a new dependency to package.json or Cargo.toml.
  • chriswarbo 1y ago
    > And users are definitely not typing a hash when adding a new dependency to package.json or Cargo.toml I actually much prefer that: specify the git revision to use (i.e. a SHA1 hash). I don't particularly care what "version number" that may or may not have.
• numbsafari 1y ago
  ... no mention of golang and minimal version selection? https://go.dev/ref/mod#minimal-version-selection https://go.dev/ref/mod#minimal-version-selection https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-mvs Instead of a "lock" file, go includes a "sum" file, which basically tells you the checksums of the versions of the modules that were used during a build happened to be, so that you can download them from a central place later and ensure you are working from the same thing (so that any surreptitious changes are identified).
• nc0 1y ago
  So much thought works for not accepting the only real, future-proof, safe, and deterministic solution that is downloading your dependencies' code next to your code forever (a.k.a. vendoring)....
• maxmcd 1y ago
  For what it's worth I think Go's MVS somewhat meets the desire here. It does not require lockfiles, but also doesn't allow use of multiple different minor/patch versions of a library: https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-mvs I believe Zig is also considering adopting it. If there are any dependencies with the same major version the algorithm simply picks the newest one of them all (but not the newest in the package registry), so you don't need a lockfile to track version decisions. Go's go.sum contains checksums to validate content, but is not required for version selection decisions.
  • nycticorax 1y ago
    Strongly endorse. That paper is really wonderful. It seems to me that MVS is the solution to the version selection problem, and now we just have to wait for awareness of this to fully percolate through the developer community.
  • vl 1y ago
    Indirect require section in go.mod file is essentially a lockfile. Once decision is made by tool, it's codified for future builds.
    • maxmcd 1y ago
      The //indirect dependencies I believe are just there to track dependencies that are not in the project, or to help with caching: https://github.com/golang/go/issues/36460 https://github.com/golang/go/issues/36460 In go 1.17 they were added so that project loading did not require downloading the go.mod of every dependency in the graph.
    • arccy 1y ago
      The decision process is deterministic and requires no human input, it's only there for caching / speed, so it's not like a traditional lockfile. You can delete the indirect section and it will be reconstructed exactly as before.
• nemothekid 1y ago
  Most of the issues in this thread and the article, are, IMO, problems with Node, not with lockfiles. >How could they know that liblupa 0.7.9, whenever it will be released, will continue to work with libpupa? Surely they can’t see the future? Semantic versioning is a hint, but it has never been a guarantee. Yes, this is a social contract. Not everything in the universe can be locked into code, and with Semantic versioning, we hope that our fellow humans won't unnecessarily break packages in non-major releases. It happens, and people usually apologize and fix, but it's rare. This has worked successfully if you look at RubyGems which is 6 years older than npm (although Gemfile.lock was introduced in 2010, npm didn't introduce it until 2017). RubyGems doesn't have the same reputation for dysfunction as Node does. Neither does Rust, Go, PHP, and Haskell. Even more that I probably don't use a daily basis. Node is the only language that I will come back and find a docker container that straight up won't build or a package that requires the entire dependency tree to update because one package pushed a minor-version change that ended up requiring a minor version change to Node, then that new version of Node isn't compatible with some hack that another package did in it's C extension. In fact, I expect some Node developer to read this article and deploy yet another tool that will break _everything_ in the build process. In other languages I don't even think I've ever really thought about dependency resolution in years.
  • bravesoul2 1y ago
    Semver is best effort. It may work but spacebar heating to change VIM mode (or whatever the XKCD joke was) may get lost in your minor update. But it is still useful. It's like a bloom filter. Most of the time you can happily pull minor or patch upgrades with no issue. Occasionally it will break. But that's less work than analysing every upgrade.
• untech 1y ago
  Don’t see it mentioned in the comments, but the names liblupa and libpupa are based on a penis joke. The joke is this: Lupa and Pupa received their paycheques, but the accountant messed up, so Lupa received payment belonging to Pupa, and Pupa — belonging to Lupa. “To Lupa” sounds like “dick head” when translated to Russian. The ending reads as if Pupa received a dick head, which means that he didn’t receive anything. I am not sure, but it could that the entire post intent is to get English-speaking folks to discuss “libpupa” and “liblupa”.
• jacksavage 1y ago
  Let's say "A" has a direct dependency on "B". The author of "A" knows how they use "B" and are qualified to state what versions of "B" that "A" is compatible with. Yes, some assumptions are made about "B" respecting semver. It's imperfect but helpful. If I'm writing package/app "C" and I consume "A", I'm not qualified to decide what versions of "B" to use without studying the source code of "A". Some situations necessitate this, but it doesn't scale. As a separate thought, it seems that it would be possible to statically analyze the usage of "B" in the source code of "A" and compare it to the public API for any version of "B" to determine API compatibility. This doesn't account for package incompatibility due to side effects that occur behind the API of "B", but it seems that it would get you pretty far. I assume this would be a solution for purely functional languages.
• jchw 1y ago
  Go MVS ought to be deterministic, but it still benefits from modules having lockfiles as it allows one to guarantee that the resolution of modules is consistent without needing to trust a central authority. Go's system may be worth emulating in future designs. It's not perfect (still requires some centralized elements, module identities for versions ≥2 are confusing, etc.) but it does present a way to both not depend strongly on specific centralized authorities without also making any random VCS server on the Internet a potential SPoF for compiling software. On the other hand, it only really works well for module systems that purely deal with source code and not binary artifacts, and it also is going to be the least hazardous when fetching and compiling modules is defined to not allow arbitrary code execution. Those constraints together make this system pretty much uniquely suited to Go for now, which is a bit of a shame, because it has some cool knock-on effects. (Regarding deterministic MVS resolution: imagine a@1.0 depending on b@1.0, and c@1.0 depending on a@1.1. What if a@1.1 no longer depends on b? You can construct trickier versions of this possibly using loops, but the basic idea is that it might be tricky to give a stable resolution to version constraints when the set of constraints that are applied depends on the set of constraints that are applied. There are possible deterministic ways to resolve this of course, it's just that a lot of these edge cases are pretty hard to reason about and I think Go MVS had a lot of bugs early on.)
  • arccy 1y ago
    I don't think anything about Go requires a central authority. You only need access to the sources of all your dependencies. Go by default uses proxy.golang.org for speed/security, and sum.golang.org for sharing verification, but it works just fine without them. I think trust in binary packages / no remote code execution is orthogonal to dependency selection.
    • jchw 1y ago
      Consider a project like Kubernetes. Find a dependency whose domain name has lapsed, or might soon. Snipe it, replicate the original VCS, add a new malicious version under a new tag. Wait for someone to go get -u it. This is pretty bad even today: If it goes unnoticed, it could lead to compromised binaries being released. However, it underscores design decisions related to the module system that helps to reduce the impact: - Use of MVS means that even without a sum file, you still won't automatically download malware published in new versions. - Use of a centralized proxy and sumdb by most of the ecosystem greatly reduces the likelihood that an attacker will attempt to change an existing version as it likely won't work and might result in detection instead. - Lack of code execution in dependency fetching substantially reduces the odds of being able to compromise a developer machine. It doesn't make the problem a non-issue, but the prognosis is a ton better. In NPM, if you don't have a lockfile, the version constraints will be resolved as you install dependencies. These version constraints are not necessarily deterministic, and installation can execute arbitrary code as the user. This is a pretty bad combination but there is a bit of relief because at least malware can be pulled from NPM when it is discovered... Which doesn't solve everything, but it's a lot better than nothing. Anyway, that's why I don't consider these topics to be fully orthogonal. They do have some interplay.
• zaptheimpaler 1y ago
  Dependency management is a deep problem with a 100 different concerns, and every time someone says "oh here it's easy, you don't need that complexity" it turns out to only apply to a tiny subset of dependency management that they thought about. Maven/Java does absolutely insane things, it will just compile and run programs with incompatible version dependencies and then they crash at some point, and pick some arbitrary first version of a dependency it sees. Then you start shading JARs and writing regex rules to change import paths in dependencies and your program crashes with a mysterious error with 1 google result and you spend 8 hours figuring out WTF happened and doing weird surgery on your dependencies dependencies in an XML file with terrible plugins. This proposed solution is "let's just never use version ranges and hard-code dependency versions". Now a package 5 layers deep is unmaintained and is on an ancient dependency version, other stuff needs a newer version. Now what? Manually dig through dependencies and update versions? It doesn't even understand lockfiles fully. They don't make your build non-reproducible, they give you both reproducible builds (by not updating the lockfile) and an easy way to update dependencies if and when you want to. They were made for the express purpose of making your build reproducible. I wish there was a mega article explaining all the concerns, tradeoffs and approaches to dependency management - there are a lot of them.
  • adrianmsmith 1y ago
    1) "it will just compile and run programs with incompatible version dependencies and then they crash at some point" 2) "Now a package 5 layers deep is unmaintained and is on an ancient dependency version, other stuff needs a newer version. Now what? Manually dig through dependencies and update versions?" You can't solve both of these simultaneously. If you want a library's dependences to be updated to versions other than the original library author wanted to use (e.g. because that library is unmaintained) then you're going to get those incompatibilities and crashes. I think it's reasonable to be able to override dependencies (e.g. if something is unmaintained) but you have to accept there are going to be surprises and be prepared to solve them, which might be a bit painful, but necessary.
    • nixosbestos 1y ago
      Yeah, you have to bump stuff and use packages that are actually compatible. Like Rust. Which does not do the insane things that Maven does, that the post author is presumably advocating for.
• RangerScience 1y ago
  No, no, a thousand times no. The package file (whatever your system) is communication to other humans about what you know about the versions you need. The lockfile is the communication to other computers about the versions you are using. What you shouldn't have needed is fully defined versions in your package files (but you do need it, in case some package or another doesn't do a good enough job following semver) So, this: package1: latest # We're stuck on an old version b/c of X, Y, Z package2: ~1.2 (Related: npm/yarn should use a JSON variant (or YAML, regular or simplified) that allows for comments for precisely this reason)
  • skybrian 1y ago
    With deterministic version control, library authors are supposed to document the exact version that a library was tested with. (Or the oldest version that they tested with and still works.) People who use a library might use newer versions (via diamond dependencies or because they use latest), but it will result in a combination of dependencies that wasn't tested by the library's authors. Often that's okay because libraries try to maintain backward compatibility. Old libraries that haven't had a new release in a while are going to specify older dependencies and you just have to deal with that. The authors aren't expected to guess which future versions will work. They don't know about security bugs or broken versions of dependencies that haven't been released yet. There are other mechanisms for communicating about that.
• furstenheim 1y ago
  > But if you want an existence proof: Maven. The Java library ecosystem has been going strong for 20 years, and during that time not once have we needed a lockfile. And we are pulling hundreds of libraries just to log two lines of text, so it is actively used at scale. Maven and Java is simply broken when dealing with transitive dependencies. I've been hit so many times with running time exception "MethodNotFound" cause two libraries have the same transitive dependency and one version gets picked over the other one.
• TJTorola 1y ago
  I generally agree, pinning versions and then having some script to automatically update to capture security updates makes sense, except that it also assumes that every package is just using standard symver, which in my experience is something like 99% true. But it's also missing the value of hashes, even if every package used symver, then you had a script that could easily update to get recent security updates, we would still gain value from a lockfile hashes to protect against source code changing underneath the same version code.
• jmull 1y ago
  > …why would [an] author write a version range that includes versions that don’t exist yet? … For that, kids, I have no good answer. When you first take a dependency, you typically want the latest compatible version, to have all the available bug fixes (especially security fixes). Once you’ve started building on top of a dependency you need stability and have to choose when to take updates. It’s about validating the dependency… on first use, there’s no question you will be validating its use in your app. Later, you have to control when you take an update so you can ensure you have a chance to validate it. BTW, of course semantics versioning isn’t perfect. It just lowers the risk of taking certain bug fixes, making it feasible to take them more frequently. The lock file just holds the state for this mechanism.
• nailer 1y ago
  You can remove the animations and read the article with: window.presence.remove() In Developer Tools. You don't even need to use querySelector, since IDs are JS globals.
• nixosbestos 1y ago
  Oh the rich irony of using Maven. Maven, apparently has the same basic fundamental issues it had 15 years ago when [redacted] paid me to write a Maven plugin that would detect these version skews. I thought I'd just done it wrong because of the massive, sweeping number of places that it did things that were not just unfortunate, but were serious (you can imagine).
• choeger 1y ago
  Version ranges solve the problem of transitive dependencies if libA needs libZ 1.0 and libB needs libZ 1.1 how am I supposed to use both dependencies at the same time when my language doesn't allow for isolation of transitive deps?
• peterpost2 1y ago
  Ugh, bunch of strawmans and then the author comes to a conclusion. Not a good article.
• _verandaguy 1y ago
  > But... why would libpupa’s author write a version range that includes versions that don’t exist yet? How could they know that liblupa 0.7.9, whenever it will be released, will continue to work with libpupa? Surely they can’t see the future? Semantic versioning is a hint, but it has never been a guarantee. > For that, kids, I have no good answer. Because semantic version is good enough for me, as a package author, to say with a good degree of confidence, "if security or stability patches land within the patch (or sometimes, even minor) fields of a semver version number, I'd like to have those rolled out with all new installs, and I'm willing to shoulder the risk." You actually kind-of answer your own question with this bit. Semver not being a guarantee of anything is true, but I'd extend this (and hopefully it's not a stretch): package authors will republish packages with the same version number, but different package contents or dependency specs. Especially newer authors, or authors new to a language or packaging system, or with packages that are very early in their lifecycle. There are also cases where packages get yanked! While this isn't a universally-available behaviour, many packaging systems acknolwedge that software will ship with unintentional vulnerabilities or serious stability/correctness issues, and give authors the ability to say, "I absolutely have to make sure that nobody can install this specific version again because it could cause problems." In those cases, having flexible subdependency version constraints helps. It might be helpful to think by analogy here. If a structure is _completely rigid,_ it does have some desirable properties, not the least of which being that you don't have to account for the cascading effects of beams compressing and extending, elements of the structure coming under changing loads, and you can forget about accounting for thermal expansion or contraction and other external factors. Which is great, in a vacuum, but structures exist in environments, and they're subject to wear from usage, heat, cold, rain, and (especially for taller structures), high winds. Incorporating a planned amount of mechanical compliance ends up being the easier way to deal with this, and forces the engineers behind it to account for failure modes that'll arise over its lifetime.
• sjrd 1y ago
  Having read the article and read some of the comments here, I think many could learn from dependency management rules in the Scala ecosystem. Scala uses Maven repositories (where the common practice is to use fixed dependency versions) but with different resolution rules: * When there are conflicting transitive versions, the highest number prevails (not the closest to the root). * Artifacts declare the versioning scheme they use (SemVer is common, but there are others) * When resolving a conflict, the resolution checks whether the chosen version is compatible with the evicted version according to the declared version scheme. If incompatible, an error is reported. * You can manually override a transitive resolution and bypass the error if you need to. The above has all the advantages of all the approaches advocated for here: * Deterministic, time-independent resolution. * No need for lock files. * No silent eviction of a version in favor of an incompatible one. * For compatible evictions, everything works out of the box. * Security update in a transitive dependency? No problem, declare a dependency on the new version. (We have bots that even automatically send PRs for this.) * Conflicting dependencies, but you know what you're doing? No problem, force an override.
• pyrale 1y ago
  Semver was always a scam. Or rather, people were glad to scam themselves with semver rather than fixing their dependencies. Languages ecosystems who try to make it sane for developers usually end up with some sort of snapshot/bom system that lists that are compatible together, and that nudges lib developers to stay compatible with each other. I'm not going to pretend this is easy, because this is hard work on the side of lib devs, but it's invaluable for the community. Compared to that, people extolling the virtues of semver always seem to miss the mark.
• pentagrama 1y ago
  The OP is well-versed in UX design, so it's hard for me to understand why they shipped that feature showing user avatars in real time [1]. It's mostly useless, distracting, takes up valuable screen space, and feels creepy. [1] https://imgur.com/a/q1XVDZU https://imgur.com/a/q1XVDZU
• _kst_ 1y ago
  The author seems to have assumed that readers are going to know that he's talking about NPM and JavaScript, and that "lockfiles" are an NPM-specific feature (to me, it means something completely different). Perhaps that's a valid assumption for readers of his blog, but once it appears here there are going to be a lot of readers who don't have the context to know what it's about. Can an "NPM" tag be added to the subject of this post? More generally, I encourage authors to include a bit more context at the top of an article.
  • deathanatos 1y ago
    > that "lockfiles" are an NPM-specific feature … they're not, though. Python & Rust both have lockfiles. I don't know enough Go to say if go.sum counts, but it might also be a lockfile. They're definitely not unique to NPM, because nothing about the problem being solved is unique to NPM.
    • _kst_ 1y ago
      OK, but to me a "lockfile" is a file whose existence signals that some resource is locked. https://en.wikipedia.org/wiki/File_locking#Lock_files https://en.wikipedia.org/wiki/File_locking#Lock_files When I saw the title "We shouldn't have needed lockfiles", I expected something about preferring some other mechanism for resource locking. More generally, I see a lot of articles that talk about an issue in some language or framework that don't mention that context. Just adding "JavaScript" or "NPM" (or whatever) in the title or near the top of the article would be very helpful.
• xp84 1y ago
  This is weird to me. (Note: i'll use ruby terms like 'gem' and 'bundle' but the same basic deal applies everywhere) Generally our practice is to pin everything to major versions, in ruby-speak this means like `gem 'net-sftp', '~> 4.0'` which allows 4.0.0 up to 4.9999.9999 but not 5. Exceptions for non-semver such as `pg` and `rails` which we just pin to exact versions and monitor manually. This little file contains our intentions of which gems to update automatically and for any exceptions, why not. Then we encourage aggressive performances of `bundle update` which pulls in tons of little security patches and minor bugfixes frequently, but intentionally. Without the lockfile though, you would not be able to do our approach. Every bundle install would be a bundle update, so any random build might upgrade a gem without anyone even meaning to or realizing it, so, your builds are no longer reproducible. So we'd fix reproducibility by reverting to pinning everything to X.Y.Z, specifically to make the build deterministic, and then count on someone to go in and update every gem's approved version numbers manually on a weekly or monthly basis. (yeah right, definitely will happen).
• chriswarbo 1y ago
  I would agree with this if the author's examples were using hashes, rather than "version numbers". Specifying a hash lets us check whether any random blob of code is or isn't what we specified; versions can't do this, because any blob of code can claim to have any name, version, etc. it likes. As long as we have a hash, we don't need version numbers (or names, though it's usually helpful to provide them). Using hashes also makes it easier to distribute, fetch, proxy, etc. since there's no need for trust. In contrast, fetching code based only on (name and) version number requires more centralised repositories with a bunch of security hoops to jump through. Also, on that note, I can plug my own post on the topic: http://www.chriswarbo.net/blog/2024-05-17-lock_files_considered_harmful.html http://www.chriswarbo.net/blog/2024-05-17-lock_files_conside...
• stephenlf 1y ago
  [dead]
• bravesoul2 1y ago
  This works to a point. It would work well on a curated ecosystem. You could get to the point where one package with one version has everything you need. They are called SDKs. However for NPM you will hit issues where 2 packages need a different React version and if you want to use them both you need to pick. In addition it is better for security. The lock file is a distributed checksum. Not impervious to supply chain attacks but better equipped than trusting the package author not to retrospectively bump (I guess you could have a hash for this tbat included the downloaded source code and claimed deps).
• pnt12 1y ago
  To me, it's the opposite. In Python, every time I see 'requirements.txt' without lock files, I cry a bit. A: You're handling problem X and then unrelated problem Y suddenly arises because you're not locking package versions thoroughly. It's not fun. B: Now the opposite. You lock all versions of the libs you use. You use renovate or schedule time for updates periodically. You have a thorough test suite that you can automatically exercise when trying the new updates. You can apply the updates and deoy the new version to a test environment to run a final test manually. Things look good. You deploy to production and, quite often, things go smoothly. A is the blue pill, easy to taste but things are out of your control and will bite you eventually. B is the red pill: you're in control, for the better or worst.
  • pnt12 1y ago
    (sorry for the bad Matrix analogy)
• burnt-resistor 1y ago
  Requirements provide absolute maximal bounds on interoperability. (This is why setting bounds to be only very recent versions arbitrarily is unhelpful because barely much of the world exists in bleeding-edge, rolling releases.) A lockfile provide a specific, concrete, minimized, satisfied solution on what an application or library uses to operate. Generally, deployed applications have and save lock files so that nothing changes without testing and interactive approval. Libraries don't usually ship lock files to give the end user more flexibility. What solved system package dependency hell is allowing multiple versions and configurations of side-by-side dependencies rather than demanding a single, one-size-fits-all dependency that forces exclusion and creates unsatisfiable constraints.
• jondwillis 1y ago
  `git push —-force` some malware would like a word. I skimmed, did the article mention this?
• zakius 1y ago
  lockfiles are useful to speed things up, you avoid waterfalling and as some people mentioned, if a dependency of a dependency provides an important security patch, do you want to wait for your dependency to update first? or do you rely on overrides?
• dent9 1y ago
  This article is written by someone who has never worked outside a single ecosystem
• bandunder 1y ago
  IMHO GO modules is one of the best implementations for reproducible builds (and deps resolution). It takes semantic versioning to the next level.
• kiitos 1y ago
  > Write down your top-level dependencies. The computer will figure out transitive ones. They are guaranteed to be the same, since everything is immutable. dep@ver can resolve to totally different outputs, depending on resolver (repository), DNS, time of day, user location, user network configuration, software author overrides, mistakes by anyone in the supply chain, etc. everything is very obviously not immutable, this is one of the main motivations for lock files, checksums, etc. dumb article :(
• quacker 1y ago
  This entire article is a strawman. It fails to even understand the basic important problem lock files try to address. Sure, I can deterministically resolve the same X.Y.Z versions of packages according to whatever algorithm I like. But literally everything is mutable (read: can be compromised). Package managers, bytes flying over the network, build servers - all mutable. A published version of a package can be overwritten. Some package managers allow authors to overwrite existing versions. That means no guarantee of reproducibility. I build again the next day, and I may not download the same code for the same X.Y.Z version of a package. Or my local cached packages may have changed. So I need checksums on packages. Then I can prove I have the exact same code in those packages. And if I have checksums on packages, I have a lock file. > lockfiles are an absolutely unnecessary concept that complicates things without a good reason What is even the complication? The lock file is autogenerated on a successful build and I commit it. It's trivial to use. It's literally all benefit and no downside.
• achikin 1y ago
  What are we trying to achieve by removing the lockfiles?
• kelsolaar 1y ago
  Maybe I suggest to pick a different name for `liblupa`, it is visually close to `libpupa`.