8 ms·

> People really need to start thinking twice when adding a new dependency. So many supply chain attacks this year. I was really nervous when "language package

> People really need to start thinking twice when adding a new dependency. So many supply chain attacks this year. I was really nervous when "language package managers" started to catch on. I work in the systems programming world, not the web world, so for the past decade, I looked from a distance at stuff like pip and npm and whatever with kind of a questionable side-eye. But when I did a Rust project and saw how trivially easy it was to pull in dozens of completely un-reviewed dependencies from the Internet with Cargo via a single line in a config file, I knew we were in for a bad time. Sure enough. This is a bad direction, and we need to turn back now. (We won't. There is no such thing as computer security.)

rootnod3 1y ago
Fully agree. That is why I vendor all my dependencies. On the common lisp side a new tool emerged a while ago for that[1]. On top of that, I try to keep the dependencies to an absolute minimum. In my current project it's 15 dependencies, including the sub-dependencies. [1]: https://github.com/fosskers/vend https://github.com/fosskers/vend
- skydhash 1y ago
  Vendoring is nice. Using the system version is nicer. If you can’t run on $current_debian, that’s very much a you problem. If postgres and nginx can do it, you can too.
  - rootnod3 1y ago
    But that would lock me in to say whatever $debian provides. And some dependencies only exist as source because they are not packaged for $distribution. Of course, if possible, just saying "hey, I need these dependencies from the system" is nicer, but also not error-free. If a system suddenly uses an older or newer version of a dependency, you might also run into trouble. In either case, you run into either an a) trust problem or b) a maintenance problem. And in that scenario I tend to prefer option b), at least I know exactly whom to blame and who is in charge of fixing it: me. Also comes down to the language I guess. Common Lisp has a tendency to use source packages anyway.
    - skydhash 1y ago
      > If a system suddenly uses an older or newer version of a dependency, you might also run into trouble. You won't. The user may. On his system.
      - rootnod3 1y ago
        Aware of that. So how is that different from any other Debian package? If you rely on a certain set of packages, you are always at the end at fault. You either trust a certain base or you vet it.
  - coldpie 1y ago
    > If you can’t run on $current_debian, that’s very much a you problem. This is a reasonable position for most software, but definitely not all, especially when you fix a bug or add a feature in your dependent library and your Debian users (reasonably!) don't want to wait months or years for Debian to update their packages to get the benefits. This probably happens rarely for stable system software like postgres and nginx, but for less well-established usecases like running modern video games on Linux, it definitely comes up fairly often.
    - teddyh 1y ago
      Something I have seen that recently have become much more common is the software upstream authors providing a Debian repository for the latest versions of their software, including backports for old Debian releases.
      - rcxdude 1y ago
        Yes, mainly because such repositories don't have to follow debian's policies, and so it's a lot easier to package a version that vendors in dependencies in a version/configuration you're willing to support (and it's better to point users there than at an official debian version because if debian breaks something you'll be getting the bug reports no matter how much people try to tell users to report to the distribution first)
  - exDM69 1y ago
    The system package manager and the language package/dependency managers do a very different task. The distro package manager delivers applications (like Firefox) and a coherent set of libraries needed to run those applications. Most distro package managers (except Nix and its kin) don't allow you to install multiple versions of a library, have libs with different compile time options enabled (or they need separate packages for that). Once you need a different version of some library than, say, Firefox does, you're out of luck. A language package manager by contrast delivers your dependency graph, pinned to certain versions you control, to build your application. It can install many different versions of a lib, possibly even link them in the same application.
    - skydhash 1y ago
      But I don’t really want your version of the application, I want the one that is aligned to my system. If some feature is really critical to the application, you can detect them at runtime and bailout (in C at least). Most developers are too aggressive on version pinning. > Most distro package managers (except Nix and its kin) don't allow you to install multiple versions of a library They do, but most distro only supports one or two versions in the official repos.
      - rcxdude 1y ago
        Maybe you want that, but I generally want the version of the application that the devs have tested the most. I've dealt with many issues due to slight differences between dependency versions, and I'd rather not provoke them. (That said, I do like debian for boring infrastructure, because they can keep things patched without changing things, but for complex desktop apps, nah, give me the upstream versions please. And for things I'm developing myself the distro is but a vehicle for a static binary or self-contained folder)
  - imiric 1y ago
    That is an impossible task in practice for most developers. Many distros, and Debian in particular, apply extensive patches to upstream packages. Asking a developer to depend on every possible variation of such packages, across many distros, is a tall order. Postgres and Nginx might be able to do it, but those are established projects with large teams behind them and plenty of leverage. They might even be able to influence distro maintainers to their will, since no distro will want to miss out on carrying such popular packages. So vendoring is in practice the only sane choice for smaller teams and projects. Besides, distro package managers carrying libraries for all programming languages is an insane practice that is impossible to scale and maintain. It exists in this weird unspecified state that can technically be useful for end users, but is completely useless for developers. Are they supposed to develop on a specific distro for some reason? Should it carry sources or only binaries? Is the dependency resolution the same for all languages? Should language tooling support them? It's an entirely ridiculous practice that should be abandoned altogether. Yes, it's also silly that every language has to reinvent the wheel for managing dependencies, and that it can introduce novel supply chain attack vectors, but the alternative is a far more ludicrous proposition.
    - skydhash 1y ago
      You do not depends on a package, you depends on its API. Implementation details shouldn't matter if behavior stays the same. Why do you care if the distro reimplemented ffmpeg or libcurl, or use an alternative version built with musl? Either the library is there or it's not. Or the minimum version you want is there or it's not. You've already provided the code and the requirement list, it's up to the distro maintainer or the user to meet them. If the latter patch the code, why do you care that much? And if a library have a feature flags, check them before using the part that is gated.
      - deleted 1y ago
        [deleted]
      - imiric 1y ago
        There's no guarantee that software/library vX.Y.Z packaged by distro A will be identical in behavior to one packaged by distro B. Sure, distro maintainers have all sorts of guidelines, but in reality, mistakes happen, and there can be incompatibilities between the version a developer has been testing against, and one the end user is using. Relying on feature flags is a pie in the sky solution, and realistically developers shouldn't have to be concerned with such environmental issues. Dependency declarations should be relied on to work 100% of the time, whether they're specified as version numbers or checksums. Since they're not reliable in practice, vendoring build and runtime dependencies is the only failproof method. This isn't to say that larger teams shouldn't support specific distros directly, but my point is that smaller teams simply don't have the resources to do so.
- coldpie 1y ago
  I didn't vendor them, but I did do an eyeball scan of every package in the full tree for my project, primarily to gather their license requirements[1]. (This was surprisingly difficult for something that every project in theory must do to meet licensing requirements!) It amounted to approximately 50 dependencies pulled into the build, to create a single gstreamer plugin. Not a fan. [1] https://github.com/ValveSoftware/Proton/commit/f21922d970888f09cb6fb84a275f0d68db9674d6#diff-e3c9359371c39fa222f39237aeaa90cd2a724c67a2af20894977d274b081bf63 https://github.com/ValveSoftware/Proton/commit/f21922d970888...
cedws 1y ago
Rust makes me especially nervous due to the possibility of compile-time code execution. So a cargo build invocation is all it could take to own you. In Go there is no such possibility by design.
- exDM69 1y ago
  The same applies to any Makefile, the Python script invoked by CMake or pretty much any other scriptable build system. They are all untrusted scripts you download from the internet and run on your computer. Rust build.rs is not really special in that regard. Maybe go build doesn't allow this but most other language ecosystems share the same weakness.
  - cedws 1y ago
    Yes but it's the fact that cargo can pull a massive unreviewed dependency tree and then immediately execute code from those dependencies that's the problem. If you have a repo with a Makefile you have the opportunity to review it first at least.
    - pharrington 1y ago
      You are allowed to read Cargo.toml.
      - cedws 1y ago
        Cargo.toml does not contain the source code of dependencies nor transient dependencies.
        magackame 1y ago
        Welp, `cargo tree`, 100 nights and 100 coffees then it is
        marshray 1y ago
skydhash 1y ago
The thing is, system based package managers require discipline, especially from library authors. Even in the web world, it’s really distressing when you see a minor library is already on its 15 iteration in less that 5 years. I was trying to build just (the task runner) on Debian 12 and it was impossible. It kept complaining about rust version, then some libraries shenanigans. It is way easier to build Emacs and ffmpeg.
- ajross 1y ago
  Indeed, it seems insane that we're pining for the days of autotools, configure scripts and the cleanly inspectable dependency structure. But... We absolutely are.
  - kalaksi 1y ago
    Disagree. And why wouldn't dep structure be cleanly inspectable.
    - ajross 1y ago
      Decades ago, you'd type "configure" and be told you need to install libfoobar version 9, and you would, and it would work. Now you get npm whining at you about an unsatisfiable dependency cycle because some three-level-removed transitive dependency you've never heard of put a hard lock file in that reference a version that got pulled for a security flaw.
BobbyTables2 1y ago
Fully agree. So many people are so drunk on the kool aid, I often wonder if I’m the weirdo for not wanting dozens of third party libraries just to build a simple HTTP client for a simple internal REST api. (No I don’t want tokio, Unicode, multipart forms, SSL, web sockets, …). At least Rust has “features”. With pip and such, avoiding the kitchen sink is not an option. I also find anything not extensively used has bugs or missing features I need. It’s easier to fork/replace a lot of simple dependencies than hope the maintainer merges my PR on a timeline convenient for my work.
- bethekidyouwant 1y ago
  Just use your fork until they merge your MR?
- 3036e4 1y ago
  There is only one Rust application (server) I use enough that I try to keep up and rebuild it from the latest release every now and then. Most of the time new releases mostly bump versions of some of the 200 or so dependencies. I have no idea how I, or the server code's maintainers, can have any clue what exactly is brought in with each release. How many upgrades times 200 projects before there is a near 100% chance of something bad being included? The ideal number of both dependencies and releases are zero. That is the only way to know nothing bad was added. Sadly much software seems to push for MORE, not fewer, of both. Languages and libraries keep changing their APIs , forcing cascades of unnecessary changes to everything. It's like we want supply chain attacks to hurt as much as possible.
  - carols10cents 1y ago
    So why are you upgrading?
- WD-42 1y ago
  If you don’t want Tokio I have bad news for you. Rust doesn’t ship an asynchronous runtime. So you’ll need something if you want to run async.
- chasd00 1y ago
Sleaker 1y ago
This isn't as new as you make it out, ant + ivy / maven / gradle had already started this in the 00s. Definitely turned into a mess, but I think the java/cross platform nature pushed this style of development along pretty heavily. Before this wasn't CPAN already big?
sheerun 1y ago
Back as in using less dependencies or throwing bunch of "certifying" services at all of them?
smohare 1y ago
[dead]
rom1v 1y ago
I feel that Rust increases security by avoiding a whole class of bugs (thanks to memory safety), but decreases security by making supply chain attacks easier (due to the large number of transitive dependencies required even for simple projects).
- carols10cents 1y ago
  Who is requiring you to use large numbers of transitive dependencies? You can always write all the code yourself instead.
thayne 1y ago
> This is a bad direction, and we need to turn back now. I don't deny there are some problems with package managers, but I also don't want to go back to a world where it is a huge pain to add any dependency, which leads to projects wasting effort on implementing things themselves, often in a buggy and/or inefficient way, and/or using huge libraries that try to do everything, but do nothing well.
- username223 1y ago
  It's a tradeoff. When package users had to manually install dependencies, package developers had to reckon with that friction. Now we're living in a world where developers don't care about another 10^X dependencies, because the package manager will just run the scripts and install the files, and the users will accept it.
jacobsenscott 1y ago
Remember the pre package manager days was ossified, archaic, insecure installations because self managing dependencies is hard, and people didn't keep them up to date. You need to get your deps from somewhere, so in the pre-package manager days you still just downloaded it from somewhere - a vendor's web site, or sourceforge, or whatever, and probably didn't audit it, and hoped it was secure. It's still work to keep things up to date and audited, but less work at least.
- rixed 1y ago
  If most of your deps are coming from the distro, they are audited already. Typically, I never had to add more than a handful of extra deps in any projects I ever worked on. That's a no brainer to manage.
sieabahlpark 1y ago
[dead]
rkagerer 1y ago
I'm actually really frustrated how hard it's become to manually add, review and understand dependencies to my code. Libraries used to come with decent documentation, now it's just a couple lines of "npm install blah", as if that tells me anything.

rootnod3 1y ago

Fully agree. That is why I vendor all my dependencies. On the common lisp side a new tool emerged a while ago for that[1]. On top of that, I try to keep the dependencies to an absolute minimum. In my current project it's 15 dependencies, including the sub-dependencies. [1]: https://github.com/fosskers/vend https://github.com/fosskers/vend

skydhash 1y ago
Vendoring is nice. Using the system version is nicer. If you can’t run on $current_debian, that’s very much a you problem. If postgres and nginx can do it, you can too.
- rootnod3 1y ago
  But that would lock me in to say whatever $debian provides. And some dependencies only exist as source because they are not packaged for $distribution. Of course, if possible, just saying "hey, I need these dependencies from the system" is nicer, but also not error-free. If a system suddenly uses an older or newer version of a dependency, you might also run into trouble. In either case, you run into either an a) trust problem or b) a maintenance problem. And in that scenario I tend to prefer option b), at least I know exactly whom to blame and who is in charge of fixing it: me. Also comes down to the language I guess. Common Lisp has a tendency to use source packages anyway.
  - skydhash 1y ago
    > If a system suddenly uses an older or newer version of a dependency, you might also run into trouble. You won't. The user may. On his system.
    - rootnod3 1y ago
      Aware of that. So how is that different from any other Debian package? If you rely on a certain set of packages, you are always at the end at fault. You either trust a certain base or you vet it.
- coldpie 1y ago
  > If you can’t run on $current_debian, that’s very much a you problem. This is a reasonable position for most software, but definitely not all, especially when you fix a bug or add a feature in your dependent library and your Debian users (reasonably!) don't want to wait months or years for Debian to update their packages to get the benefits. This probably happens rarely for stable system software like postgres and nginx, but for less well-established usecases like running modern video games on Linux, it definitely comes up fairly often.
  - teddyh 1y ago
    Something I have seen that recently have become much more common is the software upstream authors providing a Debian repository for the latest versions of their software, including backports for old Debian releases.
    - rcxdude 1y ago
      Yes, mainly because such repositories don't have to follow debian's policies, and so it's a lot easier to package a version that vendors in dependencies in a version/configuration you're willing to support (and it's better to point users there than at an official debian version because if debian breaks something you'll be getting the bug reports no matter how much people try to tell users to report to the distribution first)
- exDM69 1y ago
  The system package manager and the language package/dependency managers do a very different task. The distro package manager delivers applications (like Firefox) and a coherent set of libraries needed to run those applications. Most distro package managers (except Nix and its kin) don't allow you to install multiple versions of a library, have libs with different compile time options enabled (or they need separate packages for that). Once you need a different version of some library than, say, Firefox does, you're out of luck. A language package manager by contrast delivers your dependency graph, pinned to certain versions you control, to build your application. It can install many different versions of a lib, possibly even link them in the same application.
  - skydhash 1y ago
    But I don’t really want your version of the application, I want the one that is aligned to my system. If some feature is really critical to the application, you can detect them at runtime and bailout (in C at least). Most developers are too aggressive on version pinning. > Most distro package managers (except Nix and its kin) don't allow you to install multiple versions of a library They do, but most distro only supports one or two versions in the official repos.
    - rcxdude 1y ago
      Maybe you want that, but I generally want the version of the application that the devs have tested the most. I've dealt with many issues due to slight differences between dependency versions, and I'd rather not provoke them. (That said, I do like debian for boring infrastructure, because they can keep things patched without changing things, but for complex desktop apps, nah, give me the upstream versions please. And for things I'm developing myself the distro is but a vehicle for a static binary or self-contained folder)
- imiric 1y ago
  That is an impossible task in practice for most developers. Many distros, and Debian in particular, apply extensive patches to upstream packages. Asking a developer to depend on every possible variation of such packages, across many distros, is a tall order. Postgres and Nginx might be able to do it, but those are established projects with large teams behind them and plenty of leverage. They might even be able to influence distro maintainers to their will, since no distro will want to miss out on carrying such popular packages. So vendoring is in practice the only sane choice for smaller teams and projects. Besides, distro package managers carrying libraries for all programming languages is an insane practice that is impossible to scale and maintain. It exists in this weird unspecified state that can technically be useful for end users, but is completely useless for developers. Are they supposed to develop on a specific distro for some reason? Should it carry sources or only binaries? Is the dependency resolution the same for all languages? Should language tooling support them? It's an entirely ridiculous practice that should be abandoned altogether. Yes, it's also silly that every language has to reinvent the wheel for managing dependencies, and that it can introduce novel supply chain attack vectors, but the alternative is a far more ludicrous proposition.
  - skydhash 1y ago
    You do not depends on a package, you depends on its API. Implementation details shouldn't matter if behavior stays the same. Why do you care if the distro reimplemented ffmpeg or libcurl, or use an alternative version built with musl? Either the library is there or it's not. Or the minimum version you want is there or it's not. You've already provided the code and the requirement list, it's up to the distro maintainer or the user to meet them. If the latter patch the code, why do you care that much? And if a library have a feature flags, check them before using the part that is gated.
    - deleted 1y ago
      [deleted]
    - imiric 1y ago
      There's no guarantee that software/library vX.Y.Z packaged by distro A will be identical in behavior to one packaged by distro B. Sure, distro maintainers have all sorts of guidelines, but in reality, mistakes happen, and there can be incompatibilities between the version a developer has been testing against, and one the end user is using. Relying on feature flags is a pie in the sky solution, and realistically developers shouldn't have to be concerned with such environmental issues. Dependency declarations should be relied on to work 100% of the time, whether they're specified as version numbers or checksums. Since they're not reliable in practice, vendoring build and runtime dependencies is the only failproof method. This isn't to say that larger teams shouldn't support specific distros directly, but my point is that smaller teams simply don't have the resources to do so.
coldpie 1y ago
I didn't vendor them, but I did do an eyeball scan of every package in the full tree for my project, primarily to gather their license requirements[1]. (This was surprisingly difficult for something that every project in theory must do to meet licensing requirements!) It amounted to approximately 50 dependencies pulled into the build, to create a single gstreamer plugin. Not a fan. [1] https://github.com/ValveSoftware/Proton/commit/f21922d970888f09cb6fb84a275f0d68db9674d6#diff-e3c9359371c39fa222f39237aeaa90cd2a724c67a2af20894977d274b081bf63 https://github.com/ValveSoftware/Proton/commit/f21922d970888...

cedws 1y ago

Rust makes me especially nervous due to the possibility of compile-time code execution. So a cargo build invocation is all it could take to own you. In Go there is no such possibility by design.

exDM69 1y ago
The same applies to any Makefile, the Python script invoked by CMake or pretty much any other scriptable build system. They are all untrusted scripts you download from the internet and run on your computer. Rust build.rs is not really special in that regard. Maybe go build doesn't allow this but most other language ecosystems share the same weakness.
- cedws 1y ago
  Yes but it's the fact that cargo can pull a massive unreviewed dependency tree and then immediately execute code from those dependencies that's the problem. If you have a repo with a Makefile you have the opportunity to review it first at least.
  - pharrington 1y ago
    You are allowed to read Cargo.toml.
    - cedws 1y ago
      Cargo.toml does not contain the source code of dependencies nor transient dependencies.
      - magackame 1y ago
        Welp, `cargo tree`, 100 nights and 100 coffees then it is
        marshray 1y ago

skydhash 1y ago

The thing is, system based package managers require discipline, especially from library authors. Even in the web world, it’s really distressing when you see a minor library is already on its 15 iteration in less that 5 years. I was trying to build just (the task runner) on Debian 12 and it was impossible. It kept complaining about rust version, then some libraries shenanigans. It is way easier to build Emacs and ffmpeg.

ajross 1y ago
Indeed, it seems insane that we're pining for the days of autotools, configure scripts and the cleanly inspectable dependency structure. But... We absolutely are.
- kalaksi 1y ago
  Disagree. And why wouldn't dep structure be cleanly inspectable.
  - ajross 1y ago
    Decades ago, you'd type "configure" and be told you need to install libfoobar version 9, and you would, and it would work. Now you get npm whining at you about an unsatisfiable dependency cycle because some three-level-removed transitive dependency you've never heard of put a hard lock file in that reference a version that got pulled for a security flaw.

BobbyTables2 1y ago

Fully agree. So many people are so drunk on the kool aid, I often wonder if I’m the weirdo for not wanting dozens of third party libraries just to build a simple HTTP client for a simple internal REST api. (No I don’t want tokio, Unicode, multipart forms, SSL, web sockets, …). At least Rust has “features”. With pip and such, avoiding the kitchen sink is not an option. I also find anything not extensively used has bugs or missing features I need. It’s easier to fork/replace a lot of simple dependencies than hope the maintainer merges my PR on a timeline convenient for my work.

bethekidyouwant 1y ago
Just use your fork until they merge your MR?
3036e4 1y ago
There is only one Rust application (server) I use enough that I try to keep up and rebuild it from the latest release every now and then. Most of the time new releases mostly bump versions of some of the 200 or so dependencies. I have no idea how I, or the server code's maintainers, can have any clue what exactly is brought in with each release. How many upgrades times 200 projects before there is a near 100% chance of something bad being included? The ideal number of both dependencies and releases are zero. That is the only way to know nothing bad was added. Sadly much software seems to push for MORE, not fewer, of both. Languages and libraries keep changing their APIs , forcing cascades of unnecessary changes to everything. It's like we want supply chain attacks to hurt as much as possible.
- carols10cents 1y ago
  So why are you upgrading?
WD-42 1y ago
If you don’t want Tokio I have bad news for you. Rust doesn’t ship an asynchronous runtime. So you’ll need something if you want to run async.
chasd00 1y ago

Sleaker 1y ago

This isn't as new as you make it out, ant + ivy / maven / gradle had already started this in the 00s. Definitely turned into a mess, but I think the java/cross platform nature pushed this style of development along pretty heavily. Before this wasn't CPAN already big?

sheerun 1y ago

Back as in using less dependencies or throwing bunch of "certifying" services at all of them?

smohare 1y ago

[dead]

rom1v 1y ago

I feel that Rust increases security by avoiding a whole class of bugs (thanks to memory safety), but decreases security by making supply chain attacks easier (due to the large number of transitive dependencies required even for simple projects).

carols10cents 1y ago
Who is requiring you to use large numbers of transitive dependencies? You can always write all the code yourself instead.

thayne 1y ago

> This is a bad direction, and we need to turn back now. I don't deny there are some problems with package managers, but I also don't want to go back to a world where it is a huge pain to add any dependency, which leads to projects wasting effort on implementing things themselves, often in a buggy and/or inefficient way, and/or using huge libraries that try to do everything, but do nothing well.

username223 1y ago
It's a tradeoff. When package users had to manually install dependencies, package developers had to reckon with that friction. Now we're living in a world where developers don't care about another 10^X dependencies, because the package manager will just run the scripts and install the files, and the users will accept it.

jacobsenscott 1y ago

Remember the pre package manager days was ossified, archaic, insecure installations because self managing dependencies is hard, and people didn't keep them up to date. You need to get your deps from somewhere, so in the pre-package manager days you still just downloaded it from somewhere - a vendor's web site, or sourceforge, or whatever, and probably didn't audit it, and hoped it was secure. It's still work to keep things up to date and audited, but less work at least.

rixed 1y ago
If most of your deps are coming from the distro, they are audited already. Typically, I never had to add more than a handful of extra deps in any projects I ever worked on. That's a no brainer to manage.

sieabahlpark 1y ago

[dead]

rkagerer 1y ago

I'm actually really frustrated how hard it's become to manually add, review and understand dependencies to my code. Libraries used to come with decent documentation, now it's just a couple lines of "npm install blah", as if that tells me anything.