5 ms·
Package managers are a failed experiment. We have libraries like SQLite, which is a single .c file that you drag into your project and it immediately does a to
by nananana9 7mo ago
Package managers are a failed experiment.
We have libraries like SQLite, which is a single .c file that you drag into your project and it immediately does a ton of incredibly useful, non-trivial work for you, while barely increasing your executable's size.
The issue is not dependencies themselves, it's transitive ones. Nobody installs left-pad or is-even-number directly, and "libraries" like these are the vast majority of the attack surface. If you get rid of transitive dependencies, you get rid of the need of a package manager, as installing a package becomes unzipping a few files into a vendor/ folder.
There's so many C libraries like this. Off the top of my head, SQLite, FreeType, OpenSSL, libcurl, libpng/jpeg, stb everything, zlib, lua, SDL, GLFW... I do game development so I'm most familiar with the ones commonly used in game engines, but I'm sure other fields have similarly high quality C libraries.
They also bindings for every language under the sun. Rust libraries are very rarely used outside of Rust, and C#/Java/JS/Python libraries are never used outside their respective language (aside form Java ones in other JVM langs).
- hvb2 7mo agoIf you're developing for the web your attack surface is quite a bit bigger. Your proposed solution of copying a few files might work but how do you keep track of updates? You might be vulnerable to a published exploit fixed a few months ago. A package manager might tell you a new version is available. I don't know how that would work in your scenario.
- deleted 7mo ago[deleted]
- voidfunc 7mo agoI'd really like to see package managers organized around rings where a very small core of incredibly important stuff is kept in ring 0, ring 1 gets a slightly wider amount of stuff and can only depend on ring 0 dependencies and then ring 2+ is the crapware libraries that infect most ecosystems. But maybe that's not the right fit either. The world where package managers are just open to whatever needs to die. It's no longer a safe model.
- anakaine 7mo agoMalicious actor KPI: affect a Ring 0 package.
- swiftcoder 7mo agoIn practice, "ring 0" is whatever gets merged into your language's standard library. Node and python both have pretty expansive standard libraries at this point, stepping outside of those is a choice
- regularfry 7mo agoThe OS distro model is actually the right one here. Upstream authors hate it, but having a layer that's responsible for picking versions out of the ecosystem and compiling an internally consistent grouping of known mutually-compatible versions that you can subscribe to means that a lot of the random churn just falls away. Once you've got that layer, you only need to be aware of security problems in the specific versions you care about, you can specifically patch only them, and you've got a distribution channel for the fixes where it's far more feasible to say "just auto-apply anything that comes via this route". That model effectively becomes your ring 1. Ring 0 is the stdlib and the package manager itself, and - because you would always need to be able to step outside the distribution for either freshness or "that's not been picked up by the distro yet" reasons - the ecosystem package repositories are the wild west ring 2. In the language ecosystems I'm only aware of Quicklisp/Ultralisp and Haskell's Stackage that work like this. Everything else is effectively a rolling distro that hasn't realised that's what it is yet.
- pjc50 7mo agoPackage managers are now basically a requirement for language adoption. Doing it manually is not a solution, in an automated world. What is a problem is library quality. Which is downstream of nobody getting paid for it, combined with an optimistic but unrealistic "all packages are equal" philosophy. > High quality C libraries > OpenSSL OpenSSL is one of the ones where there's a ground up rewrite happening because the code quality is so terrible while being security critical. On the other end, javascript is uniquely bad because of the deployment model and difficulty of adding things to the standard library, so everything is littered with polyfills.
- hresvelgr 7mo ago> Package managers are now basically a requirement for language adoption. Doing it manually is not a solution, in an automated world. Absolute nonsense. What does automated world even mean? Even if one could infer reasonably, it's no justification. Appealing to "the real world" in lieu of any further consideration is exactly the kind of mindlessness that has led to the present state of affairs. Automation of dependency versions was never something we needed it was always a convenience, and even that's a stretch given that dependency hell is abundant in all of these systems, and now we have supply chain attacks. While everyone is welcome to do as they please, I'm going to stick to vendoring my dependencies, statically compiling, and not blindly trusting code I haven't seen before.
- nailer 7mo ago> Automation of dependency versions was never something we needed How do you handle updating dependencies then?
- PedroBatista 7mo agoRelax, while mentioning the real world without any criticism for the soundness of the solution is absolute nonsense, some would say idiotic, thinking only in the absolute best solution given your narrow world view is not any better.
- 7mo ago
- layer8 7mo agoFor some reason, NPM is the only ecosystem with substantial issues with supply-chain attacks.
- techterrier 7mo agoapart from that python one the other day
- SoKamil 7mo agoPopularity
- fsflover 7mo agoThe number of issues is disproportionately larger than the one for Debian.
- pyrolistical 7mo agoDebian is slower so npm is more attractive
- indy 7mo agoThe culture within the npm/js community has mainly been one of using the package manager rather than "re-inventing the wheel", as such the blast radius of a compromised package is much greater
- christophilus 7mo agoIt’s not just NPM, though. Every Rails project and every Rust project I’ve seen ended up with massive numbers of dependencies vs what an equivalent project in Go or C# would have needed.
- deleted 7mo ago[deleted]
- 7mo ago
- vincnetas 7mo agono no, please we don't want to get back to dragging files to your project to make them work.
- Tarraq 7mo agoAnd manual FTP uploads, while we're at it.
- pyrolistical 7mo agoYou don’t need to. Llm will do it for you and include the security bug
- pie_flavor 7mo agoRust libraries are infrequently used outside of Rust because if you have the option, you'd just use Rust, not the ancient featureless language intrinsically responsible for 70% of all security issues. C libraries are infrequently used in Rust outside of system libc, for the same reason; I go and toggle the reqwest switch to use rustls every time, because OpenSSL is horrendous. This is also why you say 'rarely' instead of 'never', when a few years ago it was 'never'; a few years from now you'll say 'uncommonly', and so on. The reason C libraries are used is because you don't feel like reimplementing it yourself, and they are there; but that doesn't apply more to C libraries than Rust libraries, and the vast majority of crates.io wouldn't be usefully represented in C anyway, or would take longer to bind to than to rewrite. (No, nobody uses libcurl.) Finally, this only happens in NPM, and the Rust libraries you pull in are all high-quality. So this sounds like a bunch of handwaving about nonsense.
- physicsguy 7mo agoRust is terrible for pulling in hundreds of dependencies though. Add tokio as a dependency and you'll get well over 100 packages added to your project.
- pie_flavor 7mo agopin-project-lite is the only base dependency, which itself has no dependencies. If you enable the "full" feature, ie all optional doodads turned on (which you likely don't need), it's 17: bytes, cfg-if, errno, libc, mio, parking_lot+parking_lot_core+lock_api, pin-project-lite, proc_macro2+quote+syn+unicode-ident, scopeguard, signal-hook-registry, smallvec, and socket2. You let me know which ones you think are bloat that it should reimplement or bind to a C library about, and without the blatant fabrication this time.
- estebank 7mo agoEven side stepping that tokio no longer pulls multiple packages, it used to be split into multiple packages, in the same way that KDE in Rust would be hundreds of packages. Rust projects tend to take their project and split it into many smaller packages, for ease of development, faster compiles through parallelization, ensuring proper splitting of concerns, and allowing code reuse by others. But the packages are equivalent to a single big package. The people that write it are the same. They get developed in tandem and published at the same time. You can take a look at the del tree for ripgrep, and the split of different parts of that app allows me to reuse the regex engine without dealing with APIs that only make sense in the context of a CLI app or pulling in code I won't ever use (which might be hiding an exploit too). Counting 100 100 line long crates all by the same authors as inherently more dangerous than 1 10000 line long crate makes no sense to me.
- allreduce 7mo agoI don't think this community of professionals is going to come around to a solution which requires marginally more effort. If no one checks their dependencies, the solution is to centralize this responsibility at the package repository. Something like left-pad should simply not be admitted to npm. Enforce a set of stricter rules which only allow non-trivial packages maintained by someone who is clearly accountable. Another change one could make is develop bigger standard libraries with all the utilities which are useful. For example in Rust there are a few de facto standard packages one needs very often, which then also force you to pull in a bunch of transitive dependencies. Those could also be part of the standard library. This all amounts to increasing the minimal scope of useful functionality a package has to have to be admitted and increasing accountability of the people maintaining them. This obviously comes with more effort on the maintainers part, but hey maybe we could even pay them for their labor.
- jonkoops 7mo ago> We have libraries like SQLite, which is a single .c file that you drag into your project You are just swapping a package manager with security by obscurity by copy pasting code into your project. It is arguably a much worse way of handling supply chain security, as now there is no way to audit your dependencies. > If you get rid of transitive dependencies, you get rid of the need of a package manager This argument makes no sense. Obviously reducing the amount of transitive dependencies is almost always a good thing, but it doesn't change the fundamental benefits of a package manager. > There's so many C libraries like this The language with the most fundamental and dangerous ways of handling memory, the language that is constantly in the news for numerous security problems even in massively popular libraries such as OpenSSL? Yes, definitely copy-paste that code in, surely nothing can go wrong. > They also bindings for every language under the sun. Rust libraries are very rarely used outside of Rust This is a WILD assumption, doing C-style bindings is actually quite common. YOu will of course then also be exposing a memory unsafe interface, as that is what you get with C. What exactly is your argument here? It feels like what you are trying to say is that we should just stop doing JS and instead all make C programs that copy paste massive libraries because that is somhow 'high quality'. This seems like a massively uninformed, one-sided and frankly ridiculous take.
- nananana9 7mo ago> You are just swapping a package manager with security by obscurity by copy pasting code into your project You should try writing code, and not relying on libraries for everything, it may change how you look at programming and actually ground your opinions in reality. I'm staring at company's vendor/ folder. It has ~15 libraries, all but one of which operate on trusted input (game assets). > fundamental benefits of a package manager. I literally told you why they don't matter if you write code in a sane way. > doing C-style bindings is actually quite common I know bindings for Rust libraries exist. Read the literal words you quoted. "Rust libraries are very rarely used outside of Rust". Got some counterexamples?
- vablings 7mo agohttps://github.com/memflow/memflow https://github.com/memflow/memflow https://github.com/PyO3/pyo3 https://github.com/PyO3/pyo3 https://slint.dev/ https://slint.dev/ https://github.com/dora-rs/dora https://github.com/dora-rs/dora It is VERY common in existing codebases that are migrating from C++/C to make heave use of FFI/ existing C
- victorbjorklund 7mo agoI think you can do copy paste in most languages. But it will be a pain to update when there are improvements / security fixes. You got a project with 1-2 depencies? Sure. But if you need to bring in 100 different libs (because you bring in 10 libs which in turn brings in 10 libs) good luck.
- skydhash 7mo ago> But if you need to bring in 100 different libs (because you bring in 10 libs which in turn brings in 10 libs So don’t? With manual deps management, everyone soon gravitates to a core set of deps. And libraries developer tends to reduce their deps needs, That’s why you see most C libraries deals with file formats, protocols, and broad concerns. Smaller algorithms can be shared with gists and blog articles.
- habinero 7mo ago> Smaller algorithms can be shared with gists and blog articles You just invented a worse Stack Overflow. Using libraries is good, actually.
- skydhash 7mo agoLike is-even or leftpad?
- habinero 7mo agoLike requests and pytest and ruff and so on, yes. Rewriting the world to protect against a specific kind of threat is insane.
- skydhash 7mo agoIt's not there isn't good libraries everywhere. It just the practice around NPM that people are appalled with. It's all about lowering the barrier for developers, even in spite of security and quality.
- doginasuit 7mo ago> We have libraries like SQLite, which is a single .c file that you drag into your project and it immediately does a ton of incredibly useful, non-trivial work for you, while barely increasing your executable's size. I'm not sure why you believe this is more secure than a package manager. At least with a package manager there is an opportunity for vetting. It's also trivial that it did not increase your executable's size. If your executable depends on it, it increases its effective size.
- anthk 7mo agoPackage managers are older than some users here. From CPAN/CTAN to ports under BSD's. Some pm's are badly maintained (Pip/NPM), while others are curated enough. Again, if you have GNU/Linux installed, install Guix, read the Info manual on 'guix import' and just create a shell/container with 'guix shell --container' (and a manifest package created from guix import) and use any crap you need for NPM in a reproducible and isolated way. You $HOME will be safe, for sure.
- staticassertion 7mo agoThey're not a failed experiment. No one has ever "experimented" by making a safe package manager for their new language. And it is not that insane to do so. Very basic things will get you very far: 1. Packages should carry a manifest that declares what they do at build time, just like Chrome extensions do. This manifest would then be used to configure its build environment. 2. Publishers to official registries should be forced to use 2FA. I proposed this a decade ago for crates.io and people lost their minds, like I was suggesting we drag developers to a shed to be shot. 3. Every package registry should produce a detailed audit log that contains a "who, what, when". Every build/ command should be producing audit logs that can be collected by endpoint agents too. 4. Every package registry should support TUF. 5. Typosquatting defenses should be standard. etc etc etc. Some of this is hard, some of this is not hard. All of this is possible. No one has done it, so it's way too early to say "package managers can't be made safe" when no one has tried.
- bbkane 7mo agoLove these ideas!
- philipwhiuk 7mo ago> Publishers to official registries should be forced to use 2FA. I proposed this a decade ago for crates.io and people lost their minds, like I was suggesting we drag developers to a shed to be shot. How is this enforced when it's pushed via a pipeline?
- staticassertion 7mo agoYour account is separate from your publishing. That is, in order to go to my account to change configuration values, 2FA must be required. Publishing should be handled via something like Trusted Publishing, which would leverage short lived tokens and can integrate with cryptographic logs for publish information (ie: "Published from the main branch of this repo at this time").
- mladen5 7mo agoI don't understand commercial aspect of large OSS like package managers but i was wondering for years why this was missing from npm. I think typosquatting was handled by npm last year but only after some popular miss typed packages started stealing developer creds.
- TacticalCoder 7mo agoThen you've got ecosystems like Clojure where many projects are just considered done and used by many. You can pin these (and be warned if a new version still comes out, say for an actual security fix). There are Clojure projects so stable, without any know exploit (we're certainly not talking about daily npm exploits here), that haven't been updated in years because they are... Done. Simply done. Perfection. Something to reflect upon too.
- squeaky-clean 7mo agoSQLite had 6 CVE's last year. FreeType had an RCE bug published last year. libcurl's last CVE reported was 2 weeks ago. Libpng had 4 vulnerabilities published this year....
- Andrex 7mo agoThought experiment: a language that enforces dependencies to be 2 levels deep or less. I wonder if dependencies would flatten out with this limitation in mind.