3 ms·

The languages that have a poor standard library support have this issue and other languages encourage you to import tons of libraries to fix the problem. This

by rvz 2mo ago

The languages that have a poor standard library support have this issue and other languages encourage you to import tons of libraries to fix the problem. This is why Javascript and Typescript suffer from this the most and has little to nothing to do with "popularity" and likely 9/10 of these npm packages import an external library. Golang on the other-hand is just as popular and has a stronger standard library which people build against and it is encouraged to use its standard library rather than rolling your own or importing another package to solve the problem.

tsimionescu 2mo ago
I don't think it's just that, though I agree they are clearly correlated. The reason I don't think it's a sufficient explanation is that there is a clear history of large, 3rd party libraries being created exactly to supplement poor standard libraries. C++ has Boost, Java has Apache Commons (though Java also has a pretty huge standard library), arguably we could even say C has Posix/Win32/Cocoa. I believe there is some deeper cultural reason why certain language ecosystems coalesce large utility libraries, while others prefer myriad tiny dependencies.
- DanielHB 2mo ago
  The reason is simple: the package manager works well. Helps if the package manager is standardized and there is a "default" registry for open source projects. I remember the days where I had to manually put the Spring .jar files into my project. No way I am doing that for 100s of dependencies.
- whstl 2mo ago
  The important difference IMO is in the small vs big libraries culture. Lots of languages have a bad stdlib but don’t fall into the trap of having thousands of micro libraries. The reason people do it is because it brings clout and money. Just look for articles defending micro libs: the popular ones are by people who make a living on donations, due to maintaining 1000+ packages. And collaborating in larger libs/stdlib is hard. Plus: Rust, Node, all have a lot of visibility. You need a good stdlib culture to avoid it (like Go did).
  - porridgeraisin 2mo ago
    Yep. It comes down to the way funding flows in the ecosystem.
weinzierl 2mo ago
Rich official standard library vs "import tons of libraries" are not the only two options. Java's standard library had arguably also been poor for a very long time and "import tons of libraries" just had not been practical for most of that time because the tooling and ecosystem for that did not exist yet. The solution was apache-commons and guava. Two large libraries with everything the developers heart desired and well maintained by large organizations. For Rust be probably will never have anything exactly like that because requirements from no-std development to fully fledged backend service are too diverse, but there is still room for a small number of well maintained backed by reputable developers convenience libraries in my opinion.
- wahern 2mo ago
  Namespacing. Mandatory namespacing. It's an essential prerequisite for any kind of reputation building and managememt, before you even get into the gritty technical or security details. And it has to be mandatory. Top-level package names will always have more cachet. Developers are suckers for good package names, literal or imaginative. Plus it helps address, but by no means completely solves, name and typo squatting. I understand people and groups can run their own crates.io-like repository, but that's a tangential aspect. Even if this were ubiquitous, you'd still want mandatory namespacing. You want provenance, or at least intended/nominal provenance, to be as transparent as possible, not implicit or buried. By no means a complete solution, but an important foundation for better technical and culture patterns.
  - kibwen 2mo ago
    I'm not sure what namespacing has to do with anything here. Namespacing would not even remotely have prevented anything like the OP.
    - wahern 2mo ago
      It would make it easier to, as a matter of policy, to stick to components from teams like boost or Apache or Google, i.e. groups with known reputations, reputations for having long-term, active involvement and maintenance than crates put out by single individuals or startups. And if and when a project was compromised, it's more likely to be discovered in a timely manner. To build reputations (good or bad), an ecosystem needs branding--a reliable way to connect products to an identifiable group with a history. Reputations are crucial if you're just downloading blobs of code in a fast-paced environment, because you're not directly analyzing the code, certainly not every update, and usually not even initially. It's why you don't download software from random websites to run. Linux and BSD distributions that packaged code were trusted (to varying degrees) to vet code through their package maintainers. People apt install packages with m less trepidation and risk than if they pulled random code off Github. The distro reputations tell you something, at least something much more credible that what availability on crates.io and github.com tell you. Technical security processes, which are also lacking, are no substitute for social processes. They're complimentary; you need both. But the most rudimentary and important prerequisite for building more secure social processes is completely absent in the Rust crates ecosystem.
bryanlarsen 2mo ago
I've never seen a non trivial Python or Go project without external dependencies. The dependency tree of comparable Go and Rust projects seem comparable, IMO.
- QuadmasterXLII 2mo ago
  Pytorch has 9 transitive dependencies for cpu-only execution, or 29 to bring in cuda. candle, the most popular rust ml library I found in a cursory search, has 119 for cpu only, and 150 to bring in CUDA. I guess it's taste whether that's comparable this would have been a way more satisfying dunk if nvidia hadn't split the cuda functionality needed by pytorch into 19 (!) packages on pypi but such is life.
  - bryanlarsen 2mo ago
    I expected somebody to pull out an example "disproving" my point. It wouldn't be hard. I can point out rust projects that have a ton of external dependencies, and then point at similar projects with very few. But a Python example doesn't really count, in my mind -- Python is pre-GitHub so tends to have small numbers of large external dependencies, like C++ and other older languages.
    - QuadmasterXLII 2mo ago
      If a python example doesn't count, why were your two example languages python and go? But yeah, small number of large dependencies is the way to go.
      - bryanlarsen 2mo ago
        Replying to the sentence "The dependency tree of comparable Go and Rust projects seem comparable, IMO." with a Python example seems pretty odd.
- QuadmasterXLII 2mo ago
  Ok! I had parsed your comment wrong. I see now that you claimed that python and go projects always had at least one dependency, and that go and rust projects had similar dependency counts, but you were not claiming that python and rust had similar dependency counts. Thanks for clarifying, with that I agree my comment was not a good counterexample.
wasmperson 2mo ago
I don't think this is true. People like to scapegoat the JS standard library but in reality most JS implementations have pretty featureful standard libraries, especially browsers. I've never felt the need to use any third-party libraries in the javascript projects I've worked on (except for maybe `ws`).
fukaiall 2mo ago
Can’t agree more. If you take security seriously, you should consider using Go.
sfdsfef3f3f 2mo ago
Go has a stronger standard library for certain use cases like basic CRUD web applications, but a lot of the Go standard library is also extremely low quality (flag, container, image, json, log, math, path, regexp, sync, time). Many of these aren't usable outside of toy use cases and have weird edge cases all over the place. Over time many will probably get new incompatible versions just like json. The container package in Go is the worst collection library in any mainstream programming language by a huge margin. It's actually astounding how bad it is.
- Mawr 2mo ago
  FYI, json/v2 just dropped: https://go.dev/doc/go1.27#jsonv2 https://go.dev/doc/go1.27#jsonv2 and log/slog has been a thing since 1.21: https://go.dev/doc/go1.21#slog https://go.dev/doc/go1.21#slog > a lot of the Go standard library is also extremely low quality Now adjust your definition of "a lot" to include all other languages instead of apparently arbitrarily deciding that "a lot" means at best 10 and pretending that "weird edge cases all over the place" isn't normal. I also wonder what your issue is with most of those, especially since e.g. regexp is excellent in the context of this thread - it runs in linear time, so it's not possible to craft a malicious input that will make it slow down to a crawl. > Over time many will probably get new incompatible versions just like json. What a bizarre statement to make. FYI: "The encoding/json package is now backed by the v2 implementation.". You get all the benefits possible from the v2 implementation for free while maintating backwards compat and if you want to upgrade to something better, v2 is right there. Do you have some magical suggestion how this could have possibly been handled better? And no, not having json in the stdlib isn't a viable path, that's just a cop out. But I guess since e.g. Rust and Java don't have json in their stdlibs at all, you can't say their json packages are bad, how clever and smart! Another frequently made suggestion straight from la-la land is just writing perfect code from the get-go. Brilliant, can't believe nobody's ever thought of that. Meanwhile in the real world, you've been able to reap the benefits of the solid encoding/json for over a decade now, and with v2 you get some nice free backwards compatible improvements and have a clear path to upgrading to v2. Perfectly handled IMO.
  - sfdsfef3f3f 2mo ago
trembolram 2mo ago
I don't want to see a large standard library for Rust. If something is added to the standard library, then it is very difficult to change it afterwards because backwards compatibility. It would be better to have blessed crates in crates.io. The Rust core team would release or audit them. If the blessed crates need breaking changes, it can be done by increasing their major semantic version number. That can't be done to the standard library. Actually, there could be a "trust" level for crates: 1. blessed crates by the Rust core team, 2. trusted developers, 3. untrusted developers. Or something like that..
insanitybit 2mo ago
Total nonsense. Python has a massive stdlib and there are malicious packages.
numbsafari 2mo ago
Maybe it wouldn't matter either way if people stopped putting untrusted executable code into basic build processes. You could get rid of a huge portion of the go standard library and have a massive ecosystem of leftpads, but the fact that `go build` can easily be run in off-line mode and doesn't execute any code other than the go toolchain itself, means you have a fundamentally more trustworthy ecosystem overall. Rust, on the other hand, might keep you from a subset of buffer overruns, but it will gladly run obfuscated, untrusted code in your name when you wouldn't otherwise expect it to. As an industry, we need to somehow stop making this mistake.
- kibwen 2mo ago
  > the fact that `go build` can easily be run in off-line mode and doesn't execute any code other than the go toolchain itself You can trivially run Cargo in offline mode, via the --offline flag. Nothing about this capability, in either Go or Rust, results in a more particularly trustworthy ecosystem.
- uecker 2mo ago
  I do not think this is a "mistake". This is the result of different priorities. Rust has done nothing to make me safer (although I agree that memory safety is desirable) and the supply chain issues make me less safe. Still parts of the industry want it. If your business is putting locked down app stores and media-consuming devices in everybody pockets, than memory safety is much more important than for the rest of us.
sarahsoup 2mo ago
C has a piss poor standard library and doesn't have this problem. Make dependencies annoying to use and it forces people to be more disciplined about them and if you do insist on a package manager you should not support transitive dependencies at all, packages should be self contained.
- imtringued 2mo ago
  Transitive dependencies aren't the problem. C projects often have transitive dependencies twelve packages deep. The problem is that these languages have repositories are both free for all and have no consumer side vetting out of the box. Linux distributions have maintainers that vet the packages and that's why you can blindly download transitive dependencies with your OS package manager. But you can't do the same with the AUR. The AUR workflow requires you to actually read the PKGBUILD.

by rvz 2mo ago

tsimionescu 2mo ago

I don't think it's just that, though I agree they are clearly correlated. The reason I don't think it's a sufficient explanation is that there is a clear history of large, 3rd party libraries being created exactly to supplement poor standard libraries. C++ has Boost, Java has Apache Commons (though Java also has a pretty huge standard library), arguably we could even say C has Posix/Win32/Cocoa. I believe there is some deeper cultural reason why certain language ecosystems coalesce large utility libraries, while others prefer myriad tiny dependencies.

DanielHB 2mo ago
The reason is simple: the package manager works well. Helps if the package manager is standardized and there is a "default" registry for open source projects. I remember the days where I had to manually put the Spring .jar files into my project. No way I am doing that for 100s of dependencies.
whstl 2mo ago
The important difference IMO is in the small vs big libraries culture. Lots of languages have a bad stdlib but don’t fall into the trap of having thousands of micro libraries. The reason people do it is because it brings clout and money. Just look for articles defending micro libs: the popular ones are by people who make a living on donations, due to maintaining 1000+ packages. And collaborating in larger libs/stdlib is hard. Plus: Rust, Node, all have a lot of visibility. You need a good stdlib culture to avoid it (like Go did).
- porridgeraisin 2mo ago
  Yep. It comes down to the way funding flows in the ecosystem.

weinzierl 2mo ago

Rich official standard library vs "import tons of libraries" are not the only two options. Java's standard library had arguably also been poor for a very long time and "import tons of libraries" just had not been practical for most of that time because the tooling and ecosystem for that did not exist yet. The solution was apache-commons and guava. Two large libraries with everything the developers heart desired and well maintained by large organizations. For Rust be probably will never have anything exactly like that because requirements from no-std development to fully fledged backend service are too diverse, but there is still room for a small number of well maintained backed by reputable developers convenience libraries in my opinion.

wahern 2mo ago
Namespacing. Mandatory namespacing. It's an essential prerequisite for any kind of reputation building and managememt, before you even get into the gritty technical or security details. And it has to be mandatory. Top-level package names will always have more cachet. Developers are suckers for good package names, literal or imaginative. Plus it helps address, but by no means completely solves, name and typo squatting. I understand people and groups can run their own crates.io-like repository, but that's a tangential aspect. Even if this were ubiquitous, you'd still want mandatory namespacing. You want provenance, or at least intended/nominal provenance, to be as transparent as possible, not implicit or buried. By no means a complete solution, but an important foundation for better technical and culture patterns.
- kibwen 2mo ago
  I'm not sure what namespacing has to do with anything here. Namespacing would not even remotely have prevented anything like the OP.
  - wahern 2mo ago
    It would make it easier to, as a matter of policy, to stick to components from teams like boost or Apache or Google, i.e. groups with known reputations, reputations for having long-term, active involvement and maintenance than crates put out by single individuals or startups. And if and when a project was compromised, it's more likely to be discovered in a timely manner. To build reputations (good or bad), an ecosystem needs branding--a reliable way to connect products to an identifiable group with a history. Reputations are crucial if you're just downloading blobs of code in a fast-paced environment, because you're not directly analyzing the code, certainly not every update, and usually not even initially. It's why you don't download software from random websites to run. Linux and BSD distributions that packaged code were trusted (to varying degrees) to vet code through their package maintainers. People apt install packages with m less trepidation and risk than if they pulled random code off Github. The distro reputations tell you something, at least something much more credible that what availability on crates.io and github.com tell you. Technical security processes, which are also lacking, are no substitute for social processes. They're complimentary; you need both. But the most rudimentary and important prerequisite for building more secure social processes is completely absent in the Rust crates ecosystem.

bryanlarsen 2mo ago

I've never seen a non trivial Python or Go project without external dependencies. The dependency tree of comparable Go and Rust projects seem comparable, IMO.

QuadmasterXLII 2mo ago
Pytorch has 9 transitive dependencies for cpu-only execution, or 29 to bring in cuda. candle, the most popular rust ml library I found in a cursory search, has 119 for cpu only, and 150 to bring in CUDA. I guess it's taste whether that's comparable this would have been a way more satisfying dunk if nvidia hadn't split the cuda functionality needed by pytorch into 19 (!) packages on pypi but such is life.
- bryanlarsen 2mo ago
  I expected somebody to pull out an example "disproving" my point. It wouldn't be hard. I can point out rust projects that have a ton of external dependencies, and then point at similar projects with very few. But a Python example doesn't really count, in my mind -- Python is pre-GitHub so tends to have small numbers of large external dependencies, like C++ and other older languages.
  - QuadmasterXLII 2mo ago
    If a python example doesn't count, why were your two example languages python and go? But yeah, small number of large dependencies is the way to go.
    - bryanlarsen 2mo ago
      Replying to the sentence "The dependency tree of comparable Go and Rust projects seem comparable, IMO." with a Python example seems pretty odd.
QuadmasterXLII 2mo ago
Ok! I had parsed your comment wrong. I see now that you claimed that python and go projects always had at least one dependency, and that go and rust projects had similar dependency counts, but you were not claiming that python and rust had similar dependency counts. Thanks for clarifying, with that I agree my comment was not a good counterexample.

wasmperson 2mo ago

I don't think this is true. People like to scapegoat the JS standard library but in reality most JS implementations have pretty featureful standard libraries, especially browsers. I've never felt the need to use any third-party libraries in the javascript projects I've worked on (except for maybe `ws`).

fukaiall 2mo ago

Can’t agree more. If you take security seriously, you should consider using Go.

sfdsfef3f3f 2mo ago

Go has a stronger standard library for certain use cases like basic CRUD web applications, but a lot of the Go standard library is also extremely low quality (flag, container, image, json, log, math, path, regexp, sync, time). Many of these aren't usable outside of toy use cases and have weird edge cases all over the place. Over time many will probably get new incompatible versions just like json. The container package in Go is the worst collection library in any mainstream programming language by a huge margin. It's actually astounding how bad it is.

Mawr 2mo ago
FYI, json/v2 just dropped: https://go.dev/doc/go1.27#jsonv2 https://go.dev/doc/go1.27#jsonv2 and log/slog has been a thing since 1.21: https://go.dev/doc/go1.21#slog https://go.dev/doc/go1.21#slog > a lot of the Go standard library is also extremely low quality Now adjust your definition of "a lot" to include all other languages instead of apparently arbitrarily deciding that "a lot" means at best 10 and pretending that "weird edge cases all over the place" isn't normal. I also wonder what your issue is with most of those, especially since e.g. regexp is excellent in the context of this thread - it runs in linear time, so it's not possible to craft a malicious input that will make it slow down to a crawl. > Over time many will probably get new incompatible versions just like json. What a bizarre statement to make. FYI: "The encoding/json package is now backed by the v2 implementation.". You get all the benefits possible from the v2 implementation for free while maintating backwards compat and if you want to upgrade to something better, v2 is right there. Do you have some magical suggestion how this could have possibly been handled better? And no, not having json in the stdlib isn't a viable path, that's just a cop out. But I guess since e.g. Rust and Java don't have json in their stdlibs at all, you can't say their json packages are bad, how clever and smart! Another frequently made suggestion straight from la-la land is just writing perfect code from the get-go. Brilliant, can't believe nobody's ever thought of that. Meanwhile in the real world, you've been able to reap the benefits of the solid encoding/json for over a decade now, and with v2 you get some nice free backwards compatible improvements and have a clear path to upgrading to v2. Perfectly handled IMO.
- sfdsfef3f3f 2mo ago

trembolram 2mo ago

I don't want to see a large standard library for Rust. If something is added to the standard library, then it is very difficult to change it afterwards because backwards compatibility. It would be better to have blessed crates in crates.io. The Rust core team would release or audit them. If the blessed crates need breaking changes, it can be done by increasing their major semantic version number. That can't be done to the standard library. Actually, there could be a "trust" level for crates: 1. blessed crates by the Rust core team, 2. trusted developers, 3. untrusted developers. Or something like that..

insanitybit 2mo ago

Total nonsense. Python has a massive stdlib and there are malicious packages.

numbsafari 2mo ago

Maybe it wouldn't matter either way if people stopped putting untrusted executable code into basic build processes. You could get rid of a huge portion of the go standard library and have a massive ecosystem of leftpads, but the fact that `go build` can easily be run in off-line mode and doesn't execute any code other than the go toolchain itself, means you have a fundamentally more trustworthy ecosystem overall. Rust, on the other hand, might keep you from a subset of buffer overruns, but it will gladly run obfuscated, untrusted code in your name when you wouldn't otherwise expect it to. As an industry, we need to somehow stop making this mistake.

kibwen 2mo ago
> the fact that `go build` can easily be run in off-line mode and doesn't execute any code other than the go toolchain itself You can trivially run Cargo in offline mode, via the --offline flag. Nothing about this capability, in either Go or Rust, results in a more particularly trustworthy ecosystem.
uecker 2mo ago
I do not think this is a "mistake". This is the result of different priorities. Rust has done nothing to make me safer (although I agree that memory safety is desirable) and the supply chain issues make me less safe. Still parts of the industry want it. If your business is putting locked down app stores and media-consuming devices in everybody pockets, than memory safety is much more important than for the rest of us.

sarahsoup 2mo ago

C has a piss poor standard library and doesn't have this problem. Make dependencies annoying to use and it forces people to be more disciplined about them and if you do insist on a package manager you should not support transitive dependencies at all, packages should be self contained.

imtringued 2mo ago
Transitive dependencies aren't the problem. C projects often have transitive dependencies twelve packages deep. The problem is that these languages have repositories are both free for all and have no consumer side vetting out of the box. Linux distributions have maintainers that vet the packages and that's why you can blindly download transitive dependencies with your OS package manager. But you can't do the same with the AUR. The AUR workflow requires you to actually read the PKGBUILD.