3 ms·

In my eyes, the problem isn't precisely whether some components are downloaded separately. What I find problematic is the idea of giving up responsibility by de

In my eyes, the problem isn't precisely whether some components are downloaded separately. What I find problematic is the idea of giving up responsibility by dependending on separately developed components. Components written with more than just one particular project in mind are tough to change. Integrating them into a particular project often requires workarounds and some imperfect abstractions to make things fit, and there tends to be a proliferation of blind spots about "global optimization potential". Not to drag on Rust, but I can see a "global optimization" problem when I try to cargo build a moderate Rust project and have to download hundreds of transitive dependencies, and have a good chance of ending up not building a thing (for reasons that probably include my own incompetence, but still). There are of course tons of C projects that have similar problems in a way - the number of dependencies might be lower but I have to hunt them manually. But well-engineered projects are at most a handful of clearly identifiable dependencies, and everything will generally build without fuss. This is how it is with the Linux kernel today. The main dependency of the Linux kernel is gcc, and as far as I perceive, communication between Linux and gcc projects is alive. I think there is a lesson that can be learned in particular from C development: There is value in growing a system vs mainly integrating existing stuff (junk or not). The larger a project gets, the more sense it makes for it to bring its own tools and implementations so it can continue to build and be maintained as a whole without a lot of complications. This might be the difference between a huge project and an ecosystem, and the question is where are people aiming with Rust support? If it's just about drivers I can see some limited value in the ecosystem approach but it will be a tough sell for any Rust driver that wants its 'M' in the kernel config.

kibwen 5y ago
> Components written with more than just one particular project in mind are tough to change. Even in the very worst-case scenario, you can fork the code. That would still leave you far ahead of having to write everything you need from scratch. And my experience is that plenty of libraries are happy to support general use cases when at all possible. > and have to download hundreds of transitive dependencies, and have a good chance of ending up not building a thing Can you give an example of what you mean by this? I'm unclear what the concern is regarding the idea of "not building a thing". > as far as I perceive, communication between Linux and gcc projects is alive The Rust developers also have open and active lines of communication to the Linux developers. > The larger a project gets, the more sense it makes for it to bring its own tools and implementations so it can continue to build and be maintained as a whole without a lot of complications. Certainly, but even very large projects benefit from sharing code in foundational areas.
- jstimpfle 5y ago
  >> Even in the very worst-case scenario, you can fork the code. You can fork the code that you never knew well enough to write yourself. >> and have to download hundreds of transitive dependencies, and have a good chance of ending up not building a thing > Can you give an example of what you mean by this? I'm unclear what the concern is. My concern is of course that as someone who wants, or is supposed to, mess with a project, I can easily get depressed if I have to invest significant to infinite amounts of energy to bring that project to build. And if I get it to build, have a hard time figuring out how it works because what it does is all over the place, hidden in hundreds of libraries. > Even very large projects benefit from sharing code in foundational areas. "Other crates that I'd like: anyhow, bincode, byteorder, log, once_cell, pin-project, rand, serde, slab, static_assertions, uuid plus some more esoteric ones."
  - kibwen 5y ago
    > You can fork the code that you never knew well enough to write yourself. Certainly. It is hard to envision any circumstance where one is worse off for having the opportunity to stand on the shoulders of giants rather than having to invent the universe from scratch. > I can easily get depressed if I have to invest significant to infinite amounts of energy to bring that project to build Can you give an example of a Rust project where you have had this experience? Nearly every Rust project is as simple to build as `git clone && cargo build`. The exceptions are those with C dependencies, where you will need to turn to your C package manager (apt, etc.) to first install the C dependencies, and it's hard to claim this as a weakness of Rust relative to C when it's no worse than what you'd get in a pure C project. > Other crates that I'd like: Can you be more specific about which of these crates provides functionality which you do not think qualifies as generally foundational?
    - jstimpfle 5y ago
      > Can you give an example of a Rust project where you have had this experience? Trying to make a better experience I just randomly picked "Weylus" from Github's rust/trending list and wasted more than 1 hour building it on Windows 10 and Debian Bullseye, ultimately quitting both. Yes, it was a lot of C library related issues, but also other executables missing, like tsc, or in case of Windows, make. On both systems, the number of transitive dependencies was about 300, and so I guess it was inevitable that there were going to be some problems. I've had similar problems before that trying to build other, less ambitious programs than Weylus using cargo. > it's hard to claim this as a weakness of Rust relative to C when it's no worse than what you'd get in a pure C project. It seems to me that making it more reliable to depend on other modules (like cargo does) does not lead to easier to use projects, since it doesn't change the amount of bullshit that most developers and users are willing to endure. This is an instance of Parkinson's law and I think it was already a learning from the node.js ecosystem. As a consequence, with a system like cargo programs end up having far more dependencies (which is empirically true), but the programs aren't easier to build by a random user. For developers, the situation is changed compared to e.g. C, in that it is easier to add more dependencies before the software becomes unmaintainable. This might shift the development situation to a place where the average developer in this ecosystem is more competent in plumbing things than understanding the problems and coming up with reliable solutions that are easy to maintain without a system like cargo. I'm not sure if this is good or bad - probably it's an evolution that can't be stopped, and which creates new types of developers. What I'm saying is I'm not positive that this attitude is compatible with a central piece of infrastructure, like the Linux kernel is. > Can you be more specific about which of these crates provides functionality which you do not think qualifies as generally foundational? I'm not a Rust developer and I don't know any of these, but looking at those names I'm wondering: which is not some arbitrary helper thing, where it would be easy to maintain an alternative in-tree? At least that's better than each developer bringing their own slightly different preference, resulting in more bloat and maintainability problems.

kibwen 5y ago

> Components written with more than just one particular project in mind are tough to change. Even in the very worst-case scenario, you can fork the code. That would still leave you far ahead of having to write everything you need from scratch. And my experience is that plenty of libraries are happy to support general use cases when at all possible. > and have to download hundreds of transitive dependencies, and have a good chance of ending up not building a thing Can you give an example of what you mean by this? I'm unclear what the concern is regarding the idea of "not building a thing". > as far as I perceive, communication between Linux and gcc projects is alive The Rust developers also have open and active lines of communication to the Linux developers. > The larger a project gets, the more sense it makes for it to bring its own tools and implementations so it can continue to build and be maintained as a whole without a lot of complications. Certainly, but even very large projects benefit from sharing code in foundational areas.

jstimpfle 5y ago
>> Even in the very worst-case scenario, you can fork the code. You can fork the code that you never knew well enough to write yourself. >> and have to download hundreds of transitive dependencies, and have a good chance of ending up not building a thing > Can you give an example of what you mean by this? I'm unclear what the concern is. My concern is of course that as someone who wants, or is supposed to, mess with a project, I can easily get depressed if I have to invest significant to infinite amounts of energy to bring that project to build. And if I get it to build, have a hard time figuring out how it works because what it does is all over the place, hidden in hundreds of libraries. > Even very large projects benefit from sharing code in foundational areas. "Other crates that I'd like: anyhow, bincode, byteorder, log, once_cell, pin-project, rand, serde, slab, static_assertions, uuid plus some more esoteric ones."
- kibwen 5y ago
  > You can fork the code that you never knew well enough to write yourself. Certainly. It is hard to envision any circumstance where one is worse off for having the opportunity to stand on the shoulders of giants rather than having to invent the universe from scratch. > I can easily get depressed if I have to invest significant to infinite amounts of energy to bring that project to build Can you give an example of a Rust project where you have had this experience? Nearly every Rust project is as simple to build as `git clone && cargo build`. The exceptions are those with C dependencies, where you will need to turn to your C package manager (apt, etc.) to first install the C dependencies, and it's hard to claim this as a weakness of Rust relative to C when it's no worse than what you'd get in a pure C project. > Other crates that I'd like: Can you be more specific about which of these crates provides functionality which you do not think qualifies as generally foundational?
  - jstimpfle 5y ago
    > Can you give an example of a Rust project where you have had this experience? Trying to make a better experience I just randomly picked "Weylus" from Github's rust/trending list and wasted more than 1 hour building it on Windows 10 and Debian Bullseye, ultimately quitting both. Yes, it was a lot of C library related issues, but also other executables missing, like tsc, or in case of Windows, make. On both systems, the number of transitive dependencies was about 300, and so I guess it was inevitable that there were going to be some problems. I've had similar problems before that trying to build other, less ambitious programs than Weylus using cargo. > it's hard to claim this as a weakness of Rust relative to C when it's no worse than what you'd get in a pure C project. It seems to me that making it more reliable to depend on other modules (like cargo does) does not lead to easier to use projects, since it doesn't change the amount of bullshit that most developers and users are willing to endure. This is an instance of Parkinson's law and I think it was already a learning from the node.js ecosystem. As a consequence, with a system like cargo programs end up having far more dependencies (which is empirically true), but the programs aren't easier to build by a random user. For developers, the situation is changed compared to e.g. C, in that it is easier to add more dependencies before the software becomes unmaintainable. This might shift the development situation to a place where the average developer in this ecosystem is more competent in plumbing things than understanding the problems and coming up with reliable solutions that are easy to maintain without a system like cargo. I'm not sure if this is good or bad - probably it's an evolution that can't be stopped, and which creates new types of developers. What I'm saying is I'm not positive that this attitude is compatible with a central piece of infrastructure, like the Linux kernel is. > Can you be more specific about which of these crates provides functionality which you do not think qualifies as generally foundational? I'm not a Rust developer and I don't know any of these, but looking at those names I'm wondering: which is not some arbitrary helper thing, where it would be easy to maintain an alternative in-tree? At least that's better than each developer bringing their own slightly different preference, resulting in more bloat and maintainability problems.