6 ms·

Rust has a npm package problem. For reference: https://twitter.com/bascule/status/1166583003021283333 https://twitter.com/bascule/status/1166583003021283333

by floor_ 7y ago

dpc_pw 7y ago
But also has a solution: https://github.com/crev-dev/cargo-crev https://github.com/crev-dev/cargo-crev
- floor_ 7y ago
  This is a solution to a problem that shouldn't have happened in the first place.
  - swsieber 7y ago
    It's a naturally occurring phenomena when publishing packages is easy. There are various ways to address it: * Make it harder to publish * Make it easier to find quality packages I'd prefer the latter.
    - deleted 7y ago
      [deleted]
  - dpc_pw 7y ago
    The alternatives are: * Have every single common thing live in a standard library which makes them stagnate, slows down language development and so on. * Have people roll their own half-baked, half-broken code every time. I stand firmly that big organic ecosystem of packages and thin library is the right way, and all we need is just better tools to manage trust and quality. https://twitter.com/dpc_pw/status/1166754923381280768 https://twitter.com/dpc_pw/status/1166754923381280768
    - johnisgood 7y ago
      1. If it is really common, then yes. Why not? It does not necessarily stagnate or slow down anything. People working on current official crates can do that were it in the standard library. 2. I am not so sure about that. There are many crates that are just a few lines of code. They are not that difficult to get right, but the mentality dictates that you import a crate instead of writing those "easy" one to ten lines. I am not sure it is the right way of doing it because it suffers from the same problems npm does. Making (and maintaining) more tools to patch up these issues is not exactly a great solution to me. What if these tools have issues, too? Do we create tools to patch up the issues of these other tools, ad infinitum? Few lines of code you write yourself, or: https://pbs.twimg.com/media/EDCMkVMXkAIwdI1.png https://pbs.twimg.com/media/EDCMkVMXkAIwdI1.png
      - steveklabnik 7y ago
        > Why not? It does not necessarily stagnate or slow down anything. People working on current official crates can do that were it in the standard library. It is much, much, much harder to work on the standard library than an external package. If you think Rust compile times are poor, you should try building the compiler (which is needed to work on the standard library.) This alone slows down development a bunch. You also have to deal with the PR queue, RFCs for new functionality, etc. It's more heavy weight for good reason, but it's also heavy weight. It also means that the API must be set in stone forever.
        pjmlp 7y ago
        No, it means that a process must be put in place to deprecate them, and remove them after a couple of major releases. Even Java has started to actually remove deprecated APIs.
        steveklabnik 7y ago
        Sure, If the rules were different, things would be different. We’re not doing that, nor do we foresee doing that any time soon, if ever.
- swsieber 7y ago
  As great as that tool is (I do think it's good) I wouldn't call that a solution, I'd call that a mitigation. If the ecosystem is terrible (dependency-bloat wise), then a tool like that doesn't make a big impact.
  - burntsushi 7y ago
    I think the biggest problem crev has is adoption. But if it solves that, then I disagree that it doesn't help with the dependency bloat problem. It would serve as pressure against adding dependencies, because with crev, adding dependencies becomes a lot more work because of the code review system. That in and of itself may be enough pressure to reduce the dependency bloat problem. It's a big if---especially adoption---but it makes sense to me.
    - feanaro 7y ago
      Something that requires a lot of work sounds like something people will be hesitant to use.
      - burntsushi 7y ago
        Hence my acknowledgment that adoption is the biggest problem to overcome for crev. :-)
  - sriram_sun 7y ago
    I hope this takes off. In this really good talk https://www.hillelwayne.com/talks/what-we-know-we-dont-know/ https://www.hillelwayne.com/talks/what-we-know-we-dont-know/ Hillel mentions that (I'm paraphrasing) one metric that positively affects code quality is code review.
swsieber 7y ago
So, a single example enough to state there's a problem? I admit - Rust is on the easier side of packaging, which tends towards proliferation of dependencies, but the general feeling I get right now from rust communities is a deliberate restraint when it comes to dependencies.
jacoblambda 7y ago
If you read into it a bit more, those are optional dependencies and if you drop support for redox OS while on windows you lose all but five of the dependencies and those are a config dependency, libc, and winapi and winapi architecture packages (32bit support + 64 bit support). On Unix/Linux, it only depends on the config dep and libc.
the_duke 7y ago
Interestingly the situation is somewhat similar to Javascript: a very lean standard library, paired with a great package manager that makes sharing code simple. I agree that the dependency problem is getting continuously worse though. An important factor is people splitting up crates into multiple sub-crates, proc macro crates (which have to live in a separate crate), and generally having a much too cavalier attitude about adding dependencies. The last year has been particularly bad, but on the flip side there is also increased awareness and motivation to counter this. Rust will never have a stdlib like Go, with http server/client, compression and crypto algorithms, ... But I do think that it needs to be conservatively extended with some core primitives that are missing. A problem is the still heavily evolving language. A few upcoming features could change idiomatic API design, so putting anything in std now could be a big mistake in the long term.
nindalf 7y ago
This is FUD. Almost all of those dependencies exist only to support Redox, an OS you likely don't use. If you actually compiled it for Linux or Windows, it would have 3 dependencies. One of the people replying to that tweet actually attached a screenshot of this. Since you commented "I don't actually care about Rust, I'm only farming karma to downvote posts on this trash news aggregator", I hope your bad faith attempt to spread FUD fails.
- est31 7y ago
  Those dependencies still get downloaded and shipped by tools like cargo vendor, because cargo vendor has no target specific vendoring support [1]. The dependencies still bloat up the Cargo.lock files and thus probably have a negative impact on Cargo resolution graph creation (which is an NP complete problem). And to top this, the code isn't even used on Redox [2]. [1]: https://github.com/rust-lang/cargo/issues/7058 https://github.com/rust-lang/cargo/issues/7058 [2]: https://gitlab.redox-os.org/redox-os/users/issues/26 https://gitlab.redox-os.org/redox-os/users/issues/26
cryptonector 7y ago
Is there any language / distro that doesn't? In the end you need gatekeeping. That's a security-critical function, and it's a people function, not a function of language.

Rust has a npm package problem. For reference: https://twitter.com/bascule/status/1166583003021283333 https://twitter.com/bascule/status/1166583003021283333

by floor_ 7y ago

Rust has a npm package problem. For reference: https://twitter.com/bascule/status/1166583003021283333 https://twitter.com/bascule/status/1166583003021283333

dpc_pw 7y ago

But also has a solution: https://github.com/crev-dev/cargo-crev https://github.com/crev-dev/cargo-crev

floor_ 7y ago
This is a solution to a problem that shouldn't have happened in the first place.
- swsieber 7y ago
  It's a naturally occurring phenomena when publishing packages is easy. There are various ways to address it: * Make it harder to publish * Make it easier to find quality packages I'd prefer the latter.
  - deleted 7y ago
    [deleted]
- dpc_pw 7y ago
  The alternatives are: * Have every single common thing live in a standard library which makes them stagnate, slows down language development and so on. * Have people roll their own half-baked, half-broken code every time. I stand firmly that big organic ecosystem of packages and thin library is the right way, and all we need is just better tools to manage trust and quality. https://twitter.com/dpc_pw/status/1166754923381280768 https://twitter.com/dpc_pw/status/1166754923381280768
  - johnisgood 7y ago
    1. If it is really common, then yes. Why not? It does not necessarily stagnate or slow down anything. People working on current official crates can do that were it in the standard library. 2. I am not so sure about that. There are many crates that are just a few lines of code. They are not that difficult to get right, but the mentality dictates that you import a crate instead of writing those "easy" one to ten lines. I am not sure it is the right way of doing it because it suffers from the same problems npm does. Making (and maintaining) more tools to patch up these issues is not exactly a great solution to me. What if these tools have issues, too? Do we create tools to patch up the issues of these other tools, ad infinitum? Few lines of code you write yourself, or: https://pbs.twimg.com/media/EDCMkVMXkAIwdI1.png https://pbs.twimg.com/media/EDCMkVMXkAIwdI1.png
    - steveklabnik 7y ago
      > Why not? It does not necessarily stagnate or slow down anything. People working on current official crates can do that were it in the standard library. It is much, much, much harder to work on the standard library than an external package. If you think Rust compile times are poor, you should try building the compiler (which is needed to work on the standard library.) This alone slows down development a bunch. You also have to deal with the PR queue, RFCs for new functionality, etc. It's more heavy weight for good reason, but it's also heavy weight. It also means that the API must be set in stone forever.
      - pjmlp 7y ago
        No, it means that a process must be put in place to deprecate them, and remove them after a couple of major releases. Even Java has started to actually remove deprecated APIs.
        steveklabnik 7y ago
        Sure, If the rules were different, things would be different. We’re not doing that, nor do we foresee doing that any time soon, if ever.
swsieber 7y ago
As great as that tool is (I do think it's good) I wouldn't call that a solution, I'd call that a mitigation. If the ecosystem is terrible (dependency-bloat wise), then a tool like that doesn't make a big impact.
- burntsushi 7y ago
  I think the biggest problem crev has is adoption. But if it solves that, then I disagree that it doesn't help with the dependency bloat problem. It would serve as pressure against adding dependencies, because with crev, adding dependencies becomes a lot more work because of the code review system. That in and of itself may be enough pressure to reduce the dependency bloat problem. It's a big if---especially adoption---but it makes sense to me.
  - feanaro 7y ago
    Something that requires a lot of work sounds like something people will be hesitant to use.
    - burntsushi 7y ago
      Hence my acknowledgment that adoption is the biggest problem to overcome for crev. :-)
- sriram_sun 7y ago
  I hope this takes off. In this really good talk https://www.hillelwayne.com/talks/what-we-know-we-dont-know/ https://www.hillelwayne.com/talks/what-we-know-we-dont-know/ Hillel mentions that (I'm paraphrasing) one metric that positively affects code quality is code review.

swsieber 7y ago

So, a single example enough to state there's a problem? I admit - Rust is on the easier side of packaging, which tends towards proliferation of dependencies, but the general feeling I get right now from rust communities is a deliberate restraint when it comes to dependencies.

jacoblambda 7y ago

If you read into it a bit more, those are optional dependencies and if you drop support for redox OS while on windows you lose all but five of the dependencies and those are a config dependency, libc, and winapi and winapi architecture packages (32bit support + 64 bit support). On Unix/Linux, it only depends on the config dep and libc.

the_duke 7y ago

Interestingly the situation is somewhat similar to Javascript: a very lean standard library, paired with a great package manager that makes sharing code simple. I agree that the dependency problem is getting continuously worse though. An important factor is people splitting up crates into multiple sub-crates, proc macro crates (which have to live in a separate crate), and generally having a much too cavalier attitude about adding dependencies. The last year has been particularly bad, but on the flip side there is also increased awareness and motivation to counter this. Rust will never have a stdlib like Go, with http server/client, compression and crypto algorithms, ... But I do think that it needs to be conservatively extended with some core primitives that are missing. A problem is the still heavily evolving language. A few upcoming features could change idiomatic API design, so putting anything in std now could be a big mistake in the long term.

nindalf 7y ago

This is FUD. Almost all of those dependencies exist only to support Redox, an OS you likely don't use. If you actually compiled it for Linux or Windows, it would have 3 dependencies. One of the people replying to that tweet actually attached a screenshot of this. Since you commented "I don't actually care about Rust, I'm only farming karma to downvote posts on this trash news aggregator", I hope your bad faith attempt to spread FUD fails.

est31 7y ago
Those dependencies still get downloaded and shipped by tools like cargo vendor, because cargo vendor has no target specific vendoring support [1]. The dependencies still bloat up the Cargo.lock files and thus probably have a negative impact on Cargo resolution graph creation (which is an NP complete problem). And to top this, the code isn't even used on Redox [2]. [1]: https://github.com/rust-lang/cargo/issues/7058 https://github.com/rust-lang/cargo/issues/7058 [2]: https://gitlab.redox-os.org/redox-os/users/issues/26 https://gitlab.redox-os.org/redox-os/users/issues/26

cryptonector 7y ago

Is there any language / distro that doesn't? In the end you need gatekeeping. That's a security-critical function, and it's a people function, not a function of language.