5 ms·
Languages should not have a package management system. They all have a all the world is my language blindspot and fail hard when you have anything else. Sometim
by bluGill 2y ago
Languages should not have a package management system. They all have a all the world is my language blindspot and fail hard when you have anything else. Sometimes you can build plugins in a different language but they still assume the one true language is all you want.
package management belongs to the os - or at least something else.
don't get me wrong, package management is a real problem and needs to be solved. I'm arguing against a language package manager we need a language agnostic package manager.
- poulpy123 2y agoideally yes. In practice if they wnat to provide a reasonnably good experience they have to do it
- kccqzy 2y agoAgreed. At least, languages should not require its own package management system to be used. There should be a way to invoke the compiler or interpreter without involving that language's own package management system, so that something else (like Bazel) can build on top. Fortunately, most common languages are all like that. You can invoke rustc without cargo. You can use python without pip. You can use javac without maven.
- rileymat2 2y agoI think the term "package management system" is a bit over broad a term to talk about. If we are talking about global shared dependencies, sure it may belong in the OS. If we are talking about directly shared code, it may as well belong in the language layer. If we are talking about combining independent opaque libraries, then it might belong in a different "pseudo os" level like NPM.
- the__alchemist 2y agoInteresting point, and I'm included to agree with your main point. I don't think the OS level is preferable, however: Point 1: I do not want my program to only run on only one OS, or to require custom code to make it multi-platform. Point 2: What if there's no OS?
- beeflet 2y ago>Point 1: I do not want my program to only run on only one OS, or to require custom code to make it multi-platform. To run on only one OS at build time? I usually just set up cross-compilers from linux if I am making cross-platform C/C++ code. >Point 2: What if there's no OS? You can use a system like bitbake I think.
- nickelpro 2y agoSpecifications for package interchange are absolutely essential, which is distinct from language endorsed package managers. Python doesn't have a language package manager, you're free to use pip or poetry or uv or whatever, but it does have PEP 517/518, which allow all Python package managers to interact with a common package ecosystem which encompasses polyglot codebases. C++ is only starting to address this problem with efforts like CPS. We have a plethora of packaging formats, Debian, pkg-config, conan, CMake configs, but they cannot speak fluently to one another so the package ecosystem is fractured, presenting an immense obstacle to any integration effort.
- physicsguy 2y agoPython polyglot code bases are not a solved problem at all. There have been difficulties installing TensorFlow and PyTorch with poetry for some time, and the installs still regularly break. This is the reason so many people use Conda. In HPC people are increasingly using Spack and EasyBuild to stop you having 10 versions of BLAS installed with all your Python dependencies. Comparing it to other languages isn’t really fair since they don’t have polyglot code bases in the same way, and where native packages exist in for e.g. Npm, then you run into the same problems anyway.
- howtofly 2y ago> Python doesn't have a language package manager, you're free to use pip or poetry or uv or whatever, but it does have PEP 517/518, which allow all Python package managers to interact with a common package ecosystem which encompasses polyglot codebases. This is a long-standing pain point. LWN has a series of reports covering this, one of which is: https://lwn.net/Articles/920832/ https://lwn.net/Articles/920832/
- diath 2y agoI think C++ is a living proof that not having a standard tooling around the language makes the language a complete pain in the ass to use, with any other language that does standard package managing/tooling out of the box, I can just pin the versions, commit a file to the repository, and on any computer that I'm working on I just issue a single command and everything is handled for me; meanwhile one of the C++ projects I've been working on, it turned out that I cannot build it on my server because one of the libraries I'm using only worked with clang17 which my desktop OS provides but the Debian I'm using on my server is shipping with clang16, and the library was not compatible with the earlier version of some C++ implementation, meanwhile Arch on my desktop updated to clang18, which also broke the library in some fashion, so now I'm sitting here with two systems, one where I want to deploy my software, and one where I want to develop the software, both of which are completely defunct and unable to build my project anymore; now I have to figure out how to build the specific version of clang on both systems and ensure I override a bunch of environment variables when configuring the builds on both of these systems, and then do the same on every new computer I'm developing/deploying on - with a proper tool I could just tell the project file to "use this compiler with this version with this standard" and things would just work. Some people will tell you "yeah bro just use docker with this and that and you will have a reproducible build system everywhere", but the thing is - I do not want to learn a completely unrelated tool and spend hours writing some scripts just to be able to continue working on my project when in any other programming language (like Go, Rust, JS), I can just install the runtime, clone the repo, run a command, and everything is handled for me seamlessly like it should be in 2024.
- beeflet 2y agoThe problem for me is a "political" one, not a matter of convenience: When I choose a linux distro I implicitly trust the distro maintainers to not backdoor the liveCD, so I might as well trust them to maintain packages transparently. If something happens upstream, we expect the distro maintainers to patch out undesirable behavior, integrate changes into the system as a whole or warn us of changes. Most distros are the same in functionality: the choice of a certain distro is mostly a choice of which political institution (such as a business or non-profit) that we trust to maintain the interoperability of the OS. Languages need to be more agnostic than a package manager requires because I should not have to rope another organization into my trust model. Cargo already goes too far in encouraging a single repository (crates.io) for everything through its default behavior. Who maintains crates.io? Where is the transparency? This is the most important information the user should know when deciding to use crates.io, which is whether or not they can trust the maintainers not to backdoor code, and it is rarely discussed or even mentioned! The default cargo crate (template?) encourages people to use permissive licensing for their code. So that is an example where you are already making implicit political decisions on behalf of the ecosystem and developers. That is alarming and should not be for the language maintainers to decide at all. In C/C++ you have a separation of the standard from the implementation. This is really what makes C/C++ code long-lived, because you do not have to worry about the standard being hijacked by a single group. You have a standard and multiple competing implementations, like the WWW. I cannot encourage the use of Rust while there is only a single widely-accepted implementation.
- moomin 2y agoHonestly I don’t know why more languages don’t just adopt e.g. npm, maven or NuGet. They’re largely language independent at the binary level anyway.
- beeflet 2y agonpm, maven, and NuGet have caused me far more problems in trying to reproduce builds than the OS package manager ever will.
- gwervc 2y ago> package management belongs to the os It clearly doesn't except if you're a fan of dll hell and outdated packages.
- beeflet 2y agothe solution to DLL hell is to patch the applications to all use the same version of the library.
- FridgeSeal 2y agoOh but of course! The solution to…a problem created directly by a specific approach is to…do even more work ourselves to try and untangle ourselves? And just cross our fingers and just _hope_ that every app/library is fully amenable to being patched this way? Alternatively, we could realise that this isn’t really feasible at the scale that the ecosystem operates at now, and that instead of taking an approach that requires us to “do extra work to untangle ourselves” we should try and…not have that problem in the first place.
- beeflet 2y agoI don't think it's unreasonable to have a system where every program uses the same version of a library. >And just cross our fingers and just _hope_ that every app/library is fully amenable to being patched this way? It requires some foresight in designing the application, and whether or not you even choose to use that application in the first place. We should strive to decrease the complexity of the system as a whole. The fact that packages are using different versions of the same library in the first place is a canary and the system should disincentivize that use case to some extent. Using static libraries or a chroot or a sandbox for everything is sweeping the problems under the carpet. >taking an approach that requires us to “do extra work to untangle ourselves” we should try and…not have that problem in the first place. I would prefer a system that allows you to link every application to the same library as a default, but also allows for some per-application override, perhaps by using symlinks. That would cover the majority of use cases. But I do not think that dynamic linking is generally in vain. In my own projects, I try to rely on static linking as much as possible, so I understand your perspective as a developer. But as a user I do not want programs to have their own dependencies separate from the rest of the system.
- RcouF1uZ4gsC 2y agoDisagree completely. OS package managers are one of the biggest sources of problems. Basically, once you have an OS level package manager, you have issues of versioning and ABI. You have people writing to the lowest common denominator - see for example being limited to the compiler and libraries available on an old Red Hat version. This need to maintain ABI compatibility has been one of the hugest issues with evolving C++. The OS package manager ends up being a Procrustean bed forcing everything into its mold whether or not it actually fits. Also, this doesn't even have the issue of multiple operating systems and even distros which have different package managers. Rust and Go having their own package managers has helped greatly with real world usage and evolution.
- beeflet 2y agoThis is a weird opinion, but I think that the OS package manager's complexity is largely owing to the unix directory structure which it just dumps all binaries in /bin, all configuration files in /etc, all libraries in /lib. It comes from a time where everything on the OS was developed by the same group of people. By dumping all the same file types in massive top-level directories, you need a separate program (the package manager) to keep track of which files belong to which packages and dealing with their versions and ABI and stuff. Each package represents code developed by a specific group with a certain model of the system's interoperability. GoboLinux has an interesting play on the problem by changing the directory structure so that the filesystem does most of the heavy lifting.
- deleted 2y ago[deleted]
- FridgeSeal 2y agoBut like…why? Let’s say we make a “thing” which contains packages for all participating languages. 98% of the time, aren’t users just going to go “filter down to my language” and just continue what they’re doing, except with a somewhat worse overall experience, depending on whatever the “lowest common denominator” API + semantics we use for this shared package management solution. Multi-language build systems already exist, which happily serve the needs to those projects which find themselves needing cross-language (+distributed) builds. Could there be some easier versions of these? Sure, but I don’t feel like “throw everyone in the same big box” is the solution here.
- bluGill 2y ago> I don’t feel like “throw everyone in the same big box” is the solution It has to be - while nobody needs more than a subset of that big box, the intersection of what everyone needs turns out to be throw everyone in the same big box. If you have anything less than that one big box you end up many standards and then everyone chooses which standard and in turn something important you need choose the other standard and you can't use it (ie the situation we are in now) Of course making that "one standard to rule them all" easy enough to use is a hard problem. It may be itself impossible and thus everyone drops back to the current mess.
- FridgeSeal 2y agoI don’t entirely buy the argument, but I am intrigued. > the intersection of what everyone needs turns out to be throw everyone in the same big box I don’t follow: what’s the intersection of JS/Python/Go/Rust package management? What are they all needing that isn’t “download and store packages”? It can’t be OS level configuration, because that’s going to vary by OS and language.
- mrkeen 2y agoThis sets up an untenable N*M explosion: Will the GhostBSD maintainers pin the right version of Haskell's aeson package? Will the Fedora Asahi devs stay on top of the latest Ocaml TLS developments? Will MS package PureScript's code for DOM manipulation?
- exDM69 2y ago> package management belongs to the os Os package managers do a fundamentally different task than dependency management tools used in development. They ship a bunch of applications and the libraries you need to run the applications. If you need different version of libfoo than e.g. Firefox does, you're out of luck. Need to support a customer with an older release which needs a different version of libfoo? Not gonna happen. Unless you're talking about Nix or Guix, your OS package manager is not a substitute for a dependency management tools.