9 ms·

Rust module system encourages bad practices

94 points by flurker 3y ago

noobermin 3y ago
I get that cyclic dependencies can appear[0] to encourage bad design but that this automatically yields larger crates does not seem evident or proven here... [0] this is probably a bad opinion from someone who's honestly never designed large systems, but it just seems to me that sometimes cycles are inevitable, you just have to be careful when they arise and minimize the times they happen.
Dowwie 3y ago
This article considers design problems introduced by humans, not the language. Cycles aren't a natural side effect of using Rust but rather a side effect of modularization. Move your shared dependencies into another crate.
_nalply 3y ago
One of the difficulties of breaking up a large crate is the Orphan Rule. I got bitten by it once. I had some traits and when I broke up the crate suddenly some types but also the traits were in different crates each. I wanted to separate the type, the trait and the implementation of the trait in different crates. I was forced to use newtypes but newtypes need a lot of boiler plate. It was painful. In my opinion this is a larger barrier than having separate Cargo.toml files.
- rnestler 3y ago
  > I wanted to separate the type, the trait and the implementation of the trait in different crates. I usually put the traits in a separate crate, but keep concrete types and implementations for them in the same crate. What is the use-case for splitting a type and it's implementation into separate crates? (except for cases where the trait or the type are out of your control, then one indeed needs to use the new-type escape hatch)
  - _nalply 3y ago
    I had three different goals, one for the traits, one for something else where the types fit best, and the third which I achieved with the implementations of the traits. The problem was that the trait implementations of the types was something like not a core task of the types themselves.
sandybandy 3y ago
Excellent Article !! Infact this article helped me make a choice. To learn Go or Rust. Clearly, unless the project is very lahttps://news.ycombinator.com/item?id=36233862rge https://news.ycombinator.com/item?id=36233862rge or demands rewrite of entire stack, then GO is the choice All this rust to learn Rust seems more like a gold rush to me. Go seems to be the python equivalent of systems programming
sandybandy 3y ago
Excellent Article !! Infact this article helped me make a choice. To learn Go or Rust. Clearly, unless the project is very large or demands rewrite of the entire stack, then GO is the choice All this rust to learn Rust seems more like a gold rush to me. Go seems to be the python equivalent of systems programming
physPop 3y ago
I feel like the author really overstates how hard it is to make a new crate. Making a cargo.toml is simple and frankly you don’t do it so often as to be a burden. Encouraging cyclic dependencies on the other hand “because it is easy” seems like a lazy cop out.
- virtualritz 3y ago
  > I feel like the author really overstates how hard it is to make a new crate. I read it more as the maintenance overhead when publishing. You need to bump version, check dependency versions etc. I'm speaking from personal experience here. The more I split my crates into sub-crates, the longer the release/publish process takes. There are some parts you can automate with e.g. cargo-release but I now think a lot harder before cutting a release to crates.io. I also have the gut feeling this is happening throughout the ecosystem. I have a lot more overlays in my Cargo.tomls now than I used to, two years ago. I.e. there are PRs and fixes in the repo. that you need but the author hasn't cut a release. This may be coincidence ofc but I can't help but think there may be a connection.
- jchw 3y ago
  Well, they seem to be cognizant of that, but they have a point: compared to simply creating a folder, it is QUITE a lot more effort. It's like the difference between zero and one. That having been said, I think my bigger issue with this is that this actually gets to be annoying after you do it, not just while doing it. Go manages external dependencies on the module level, which is a level higher than packages. For Rust, it's crates. So a repo with many crates becomes pretty cumbersome. (And a bit moreso if you're trying to maintain a Nix package for said repo, but that's partly the fault of Nix. Only partly, because I actually think Rust's toolchain makes the job of Nix rather difficult...)
  - mikepurvis 3y ago
    As a moderate Nix user/packager, I feel like the ideal model for ecosystem packaging in Nix is poetry2nix, where the existing lockfile hashes are able to be used directly to create pure Nix builds, with no need to generate code, pre-download anything, or have opaque "deps" blobs (like with Bazel). As far as I can tell, all or most of these requirements are met by Cargo lockfiles as well, so what is the gap that makes Cargo/Rust difficult to do well in Nix?
davidhyde 3y ago
The author desires a simple folder based module system and argues how it would improve compilation times. I do agree although it would be nice to know what flexibility would be lost so the argument could have some weight on the other side. On a tangent I recall struggling with cargo features in the beginning. I found them to be hidden and mysterious. Even searching for “rust features” sucks.
- rascul 3y ago
  You probably discovered this by now, but if you can find the Cargo.toml then the features are listed there. Or if you use 'cargo add' the features will be listed. Took me a bit too at first.
- deleted 3y ago
  [deleted]
jeremyjh 3y ago
What the author mentions but doesn’t seem to allocate a lot of significance to is that in Go, modules are a level of organization higher up the hierarchy from packages. Rust has no such a higher level organizing unit. It’s only unit for both sharing dependencies and compiling files is a crate. So crates have to take on additional duties, such as specifying build instructions and dependencies. Been years since I’ve done much Rust. Does this mean bumping dependency versions thus becomes a larger chore in a project?
- drewtato 3y ago
  Rust has workspaces, which is a collection of packages (a Cargo.toml and related source files). Workspaces are still pretty underpowered, but they do have shared dependencies, compilation artifacts, version numbers, and CLI management. A lot of this is new (1 year old).
  - beefsack 3y ago
    Cargo workspaces have existed for a number of years now - the feature was released in Cargo alongside Rust 1.12 in September 2016: https://blog.rust-lang.org/2016/09/29/Rust-1.12.html https://blog.rust-lang.org/2016/09/29/Rust-1.12.html
    - drewtato 3y ago
      The version numbers are from a little more than a year ago. Originally, it was pretty much only compilation artifacts and some CLI. https://github.com/rust-lang/cargo/pull/10563 https://github.com/rust-lang/cargo/pull/10563
Georgelemental 3y ago
Rust's trait coherence rules can make it impossible to split certain crates up. On the flip-side, cargo's workspace features make multi-crate projects easier to manage. I would love it if it were possible to loosen coherence within a workspace.
- duped 3y ago
  "private" trait impls (1) would be a much better solution. You probably never want the impl of a trait for a foreign type to be exposed to other crates, it's so unlikely that forbidding it in the name of coherence would probably work out. But there's apparently many gotchas and no one has clearly written a way around them (2). (1) https://github.com/rust-lang/rfcs/issues/493 https://github.com/rust-lang/rfcs/issues/493 (although the comment section is kind of unhinged and goes off on distant tangents to modular typeclasses and first class modules) (2) http://smallcultfollowing.com/babysteps/blog/2015/01/14/little-orphan-impls/ http://smallcultfollowing.com/babysteps/blog/2015/01/14/litt...
thurn 3y ago
I've never used Go, but do you really not need to create anything similar to a Cargo.toml file specifying dependencies to create a package? That doesn't seem like it could be true within Google since they would use blaze to compile their Go code, which mandates a new BUILD file for each package... The other stuff here is mostly just a matter of convention. Every large Rust codebase I'm familiar with absolutely does avoid creating huge crates, so I think the author is just mistaken about what constitutes standard Rust practice.
- pclmulqdq 3y ago
  There is no required cargo.toml equivalent for go. You put all dependencies at the top of your source code in an import statement. There are other things you can do if you want to lock a specific version of an API.
  - TheDong 3y ago
    Okay, let me try that: $ echo -e 'package main\n\nimport _ "golang.org/x/image/draw"' > main.go $ go run main.go main.go:3:8: no required module provides package golang.org/x/image/draw: go.mod file not found in current directory or any parent directory; see 'go help modules' $ go build main.go <same error> go.mod seems pretty required. With rust, you can also technically just call 'rustc' and never use 'cargo', but just like it's a pain to do that in go, it's a pain to do that in rust.
    - silisili 3y ago
      With Go at least, just prepend GO111MODULE=off and it works. A bit esoteric, but I like that option, especially when working on an application and its dependencies in parallel at times.
- 2h 3y ago
  sibling comment is wrong. this is the typical flow on a new package: 1. go mod init something # create go.mod 2. go mod tidy # create go.sum 3. have fun
curtis3389 3y ago
I was expecting something about how you can't move code into a new file without creating a new module, which means you'll need to modify calling code if you want to reorganize stuff, so you end up with massive source files.
- drewtato 3y ago
  You can fix this by doing `pub use module::*`.
  - 2h 3y ago
    wildcard import, definitely no drawbacks to those!
- eftychis 3y ago
  You mean that if you split a file you define another module when you declare it? That is only visible as a module at the level of declaration not beyond. Also note that modules are not per se compile units. Cf. See https://doc.rust-lang.org/rust-by-example/mod/split.html https://doc.rust-lang.org/rust-by-example/mod/split.html Edit: typo.
phkahler 3y ago
1) he never says what bad practices are encouraged. A crate can have many source files, so what's wrong with big crates? It's better than npm with tons of stupid small dependencies. 2) what if 2 modules in go share a common dependency? It's not clear to me how that's handled in a file system hierarchy. But thats beside the point anyway.
armchairhacker 3y ago
Powerful features encourage bad practices. This is another "Rust empowers the developer, Go protects the developer from themselves" (and empowerment isn't always a good thing and protection-from-self a bad thing, e.g. goto, type systems)
- dimonomid 3y ago
  First of all, what are other examples of "Rust empowers the developer, Go protects the developer from themselves"? As article mentions, Rust as a language is actually must stricter than Go, C++ and many others, and Rust community likes to brag that by restricting what we can do in safe Rust, we end up with better software. And I agree with that. But when it comes to dependency management, Rust turns out to be surprisingly lax. Also, in this case, Go doesn't exactly protect the developer from anything. One could create a single package with tons of files in them, and it'll keep working. But the way Go package system is designed, it just subtly suggests to the developer that it's not the right thing to do: having a package (i.e. directory) with 200+ files in it just feels wrong by default. One trait of a well-designed system is that it's difficult to abuse.
epage 3y ago
> So while Go encourages having reasonably sized packages, Rust unfortunately encourages huge crates. I'm a bit confused by this as I'm used to seeing the opposite complaint, that Rust packages are too small. Maybe if there were examples it would help. And while I do think we should explore ways to allow lower effort packages ("cargo script" is one component of this [0]), I see allowing submodules as a feature though maybe thats my Python/C++ background showing through. I do like that it allows both low overhead modules in your API (just a file) or allows you to separate a module out into multiple files for clearer organization (where cycles become important). While I don't know if there is a feasible design for this, but I do wish we could specify what modules are cyclic vs acyclic. [0] https://github.com/rust-lang/rfcs/pull/3424 https://github.com/rust-lang/rfcs/pull/3424
- dimonomid 3y ago
  > 'm a bit confused by this as I'm used to seeing the opposite complaint, that Rust packages are too small. By "packages" here you're probably referring to the public crates on crates.io, right? Indeed, those public crates are not big in general, but that's not the point of the article. The article is rather talking about large, multi-crate projects, organized as a Cargo workspace. I agree it could have been clearer in the article.
duped 3y ago
There is one giant issue with Rust crates vs modules, and it's not "crates are harder to make." The real problem is that you can't use crates to improve incrementalism when you have a type in one logical part of the module hierarchy implement a trait from another logical part. Due to orphan rules and coherence, it is not possible to implement a foreign trait for a foreign type. And that said, just because the compilation unit is a crate doesn't mean it can't be compiled incrementally or with parallelism. That's a separate problem.
- rnestler 3y ago
  > The real problem is that you can't use crates to improve incrementalism when you have a type in one logical part of the module hierarchy implement a trait from another logical part. Due to orphan rules and coherence, it is not possible to implement a foreign trait for a foreign type. Well you can implement a trait for a type from another logical part either in the crate of the trait or the crate of the type. What is the use-case for implementing a foreign trait for a foreign type if you have both under control?
solidsnack9000 3y ago
It may be that what Rust allows here is not perfect, but it is not exactly bad. Certainly, compared to most languages, there is nothing abnormal about allowing many files in a tree of directories in a crate, gem, egg, rock, package, module, &c.
insanitybit 3y ago
> So while Go encourages having reasonably sized packages, Rust unfortunately encourages huge crates. Interesting since I think Rust very much encourages the opposite, which is why so many crates like hyper, regex, and others, are actually composed of many crates. I don't really get the main thrust of the article. Go forces directories to map to packages. That sounds... awful. I guess it "encourages small packages" because you can't even organize them into folders? IDK, I feel like I'm missing something - particularly, I'm missing why big or small is good or bad, or why creating a crate is onerous.
pie_flavor 3y ago
What goes into another crate vs another module is an API architecture question; you can shake your fist at (overblown) compilation times but it doesn't make it bad practice to make stuff that's semantically a library, a library. As an example of good practice, Go is pointed to, which is an ecosystem full of people splitting lots of stuff into separate libraries that make no sense to the end user as a separate library and in practice serve the role of submodules. This I would call bad practice, if not for the fact that Go disallows you from doing it any other way. If Go had a central package index instead of just using GitHub, it would be a very confusing place, full of 'private submodules'.
- ithkuil 3y ago
  Indeed Go's "internal" packages serve the need to create packages that are not libraries
- mobiuscog 3y ago
  leftpad
- foldr 3y ago
  https://dave.cheney.net/2019/10/06/use-internal-packages-to-reduce-your-public-api-surface https://dave.cheney.net/2019/10/06/use-internal-packages-to-...
eftychis 3y ago
I think the author is unaware of a) how to create crates easily b) that workspaces exist to organize crates I wouldn't pick Go as an example of how to build a module system. Far from it. ML languages yes. Anyways, adding a single line command per compile unit package is not a "cost." Edit: P.S. Also the module logical system in Rust is way much more expressive.
vpastore 3y ago
[dead]
pjmlp 3y ago
Rust 's flexibility exists for a reason, and it isn't as if the whole Go modules aren't without issues. Not to mention the whole bonkers idea of having package names bound to Github repos and source only.
- thumbuddy 3y ago
  The people cheering on Go here surprise me a bit. I don't hate Go, but I have spent way more time reading stack overflow posts trying to get a go project to be glued together, fiddling with environment variables, copy pasting code around then I ever have in rust. Again not claiming superiority of one tool over another... But the user experience for rust has been a lot better for project organization for me then most languages. My biggest gripe is, the rust docs somehow make this incredibly difficult language easy to learn, but the sections around code organization are really really hard for beginners or anyone who hasn't sunk time into it and completely omit the 10 meter view of how to organize rust code.
xpe 3y ago
> You might have noticed that when I'm talking about modules being organized as a tree, I say that it seems to be neatly organized. Because the dependencies are acyclic there, modules aren't actually organized, this tree structure merely provides an illusion, while underneath there is likely a dependency mess, and there can be hundreds of files in that mess, and Rust seemingly encourages that; otherwise why would we need modules to be organized in a tree in the first place. Personally, this strikes me as bizarre. It is only an "illusion" if one expects a module tree structure to map one-to-one with dependencies. No one should expect this, as I will explain with three uncontroversial and widely known points: A. Are there any languages where parent-child relationships between modules necessarily imply dependency semantics? I can't think of any, nor can I envision any sane language doing so. [Note 1] B. In Rust, like with so many languages, when a module is nested in another module, that means the inner module is scoped inside the outer module. It is about namespacing, not dependencies. C. Dependencies are created when the contents of something in a module refer to another module. [Note 1]: How would such a language specify more than one dependency? A node in a tree can only have one parent, after all. This would rule out a one-to-one mapping.
smilekzs 3y ago
To me (a C++ daily driver and Rust hobbyist) this "DAG of strongly-connected" tiered graph system is sane and maps well to practice --- tightly couple your impl details however you want, then expose a clean interface at the crate level. It encourages you to split your code to separate files as you write _without immediately forcing you to untangle the dependency_, and then as "islands" of code organically form, which pieces belong to a separate crate becomes naturally evident. Contrast this with C++ where the DAG relationship is enforced at much lower levels (e.g. Bazel cc_library, CMake target, which in normal cases correspond to a cc/h pair), shifting the friction (of having to deal with deps) much earlier in the dev cycle, which then discourages you from (incrementally) splitting a file at all. I've seen too many times this resulting in 5000+ LoC cc file.
- pjmlp 3y ago
  Modules are similar high level. Naturally you can only enjoy them today on VC++, but the rest will follow.

94 points by flurker 3y ago

I get that cyclic dependencies can appear[0] to encourage bad design but that this automatically yields larger crates does not seem evident or proven here... [0] this is probably a bad opinion from someone who's honestly never designed large systems, but it just seems to me that sometimes cycles are inevitable, you just have to be careful when they arise and minimize the times they happen.

Dowwie 3y ago

This article considers design problems introduced by humans, not the language. Cycles aren't a natural side effect of using Rust but rather a side effect of modularization. Move your shared dependencies into another crate.

_nalply 3y ago

One of the difficulties of breaking up a large crate is the Orphan Rule. I got bitten by it once. I had some traits and when I broke up the crate suddenly some types but also the traits were in different crates each. I wanted to separate the type, the trait and the implementation of the trait in different crates. I was forced to use newtypes but newtypes need a lot of boiler plate. It was painful. In my opinion this is a larger barrier than having separate Cargo.toml files.

rnestler 3y ago
> I wanted to separate the type, the trait and the implementation of the trait in different crates. I usually put the traits in a separate crate, but keep concrete types and implementations for them in the same crate. What is the use-case for splitting a type and it's implementation into separate crates? (except for cases where the trait or the type are out of your control, then one indeed needs to use the new-type escape hatch)
- _nalply 3y ago
  I had three different goals, one for the traits, one for something else where the types fit best, and the third which I achieved with the implementations of the traits. The problem was that the trait implementations of the types was something like not a core task of the types themselves.

sandybandy 3y ago

Excellent Article !! Infact this article helped me make a choice. To learn Go or Rust. Clearly, unless the project is very lahttps://news.ycombinator.com/item?id=36233862rge https://news.ycombinator.com/item?id=36233862rge or demands rewrite of entire stack, then GO is the choice All this rust to learn Rust seems more like a gold rush to me. Go seems to be the python equivalent of systems programming

sandybandy 3y ago

Excellent Article !! Infact this article helped me make a choice. To learn Go or Rust. Clearly, unless the project is very large or demands rewrite of the entire stack, then GO is the choice All this rust to learn Rust seems more like a gold rush to me. Go seems to be the python equivalent of systems programming

physPop 3y ago

I feel like the author really overstates how hard it is to make a new crate. Making a cargo.toml is simple and frankly you don’t do it so often as to be a burden. Encouraging cyclic dependencies on the other hand “because it is easy” seems like a lazy cop out.

virtualritz 3y ago
> I feel like the author really overstates how hard it is to make a new crate. I read it more as the maintenance overhead when publishing. You need to bump version, check dependency versions etc. I'm speaking from personal experience here. The more I split my crates into sub-crates, the longer the release/publish process takes. There are some parts you can automate with e.g. cargo-release but I now think a lot harder before cutting a release to crates.io. I also have the gut feeling this is happening throughout the ecosystem. I have a lot more overlays in my Cargo.tomls now than I used to, two years ago. I.e. there are PRs and fixes in the repo. that you need but the author hasn't cut a release. This may be coincidence ofc but I can't help but think there may be a connection.
jchw 3y ago
Well, they seem to be cognizant of that, but they have a point: compared to simply creating a folder, it is QUITE a lot more effort. It's like the difference between zero and one. That having been said, I think my bigger issue with this is that this actually gets to be annoying after you do it, not just while doing it. Go manages external dependencies on the module level, which is a level higher than packages. For Rust, it's crates. So a repo with many crates becomes pretty cumbersome. (And a bit moreso if you're trying to maintain a Nix package for said repo, but that's partly the fault of Nix. Only partly, because I actually think Rust's toolchain makes the job of Nix rather difficult...)
- mikepurvis 3y ago
  As a moderate Nix user/packager, I feel like the ideal model for ecosystem packaging in Nix is poetry2nix, where the existing lockfile hashes are able to be used directly to create pure Nix builds, with no need to generate code, pre-download anything, or have opaque "deps" blobs (like with Bazel). As far as I can tell, all or most of these requirements are met by Cargo lockfiles as well, so what is the gap that makes Cargo/Rust difficult to do well in Nix?

davidhyde 3y ago

The author desires a simple folder based module system and argues how it would improve compilation times. I do agree although it would be nice to know what flexibility would be lost so the argument could have some weight on the other side. On a tangent I recall struggling with cargo features in the beginning. I found them to be hidden and mysterious. Even searching for “rust features” sucks.

rascul 3y ago
You probably discovered this by now, but if you can find the Cargo.toml then the features are listed there. Or if you use 'cargo add' the features will be listed. Took me a bit too at first.
deleted 3y ago
[deleted]

jeremyjh 3y ago

What the author mentions but doesn’t seem to allocate a lot of significance to is that in Go, modules are a level of organization higher up the hierarchy from packages. Rust has no such a higher level organizing unit. It’s only unit for both sharing dependencies and compiling files is a crate. So crates have to take on additional duties, such as specifying build instructions and dependencies. Been years since I’ve done much Rust. Does this mean bumping dependency versions thus becomes a larger chore in a project?

drewtato 3y ago
Rust has workspaces, which is a collection of packages (a Cargo.toml and related source files). Workspaces are still pretty underpowered, but they do have shared dependencies, compilation artifacts, version numbers, and CLI management. A lot of this is new (1 year old).
- beefsack 3y ago
  Cargo workspaces have existed for a number of years now - the feature was released in Cargo alongside Rust 1.12 in September 2016: https://blog.rust-lang.org/2016/09/29/Rust-1.12.html https://blog.rust-lang.org/2016/09/29/Rust-1.12.html
  - drewtato 3y ago
    The version numbers are from a little more than a year ago. Originally, it was pretty much only compilation artifacts and some CLI. https://github.com/rust-lang/cargo/pull/10563 https://github.com/rust-lang/cargo/pull/10563

Georgelemental 3y ago

Rust's trait coherence rules can make it impossible to split certain crates up. On the flip-side, cargo's workspace features make multi-crate projects easier to manage. I would love it if it were possible to loosen coherence within a workspace.

duped 3y ago
"private" trait impls (1) would be a much better solution. You probably never want the impl of a trait for a foreign type to be exposed to other crates, it's so unlikely that forbidding it in the name of coherence would probably work out. But there's apparently many gotchas and no one has clearly written a way around them (2). (1) https://github.com/rust-lang/rfcs/issues/493 https://github.com/rust-lang/rfcs/issues/493 (although the comment section is kind of unhinged and goes off on distant tangents to modular typeclasses and first class modules) (2) http://smallcultfollowing.com/babysteps/blog/2015/01/14/little-orphan-impls/ http://smallcultfollowing.com/babysteps/blog/2015/01/14/litt...

thurn 3y ago

I've never used Go, but do you really not need to create anything similar to a Cargo.toml file specifying dependencies to create a package? That doesn't seem like it could be true within Google since they would use blaze to compile their Go code, which mandates a new BUILD file for each package... The other stuff here is mostly just a matter of convention. Every large Rust codebase I'm familiar with absolutely does avoid creating huge crates, so I think the author is just mistaken about what constitutes standard Rust practice.

pclmulqdq 3y ago
There is no required cargo.toml equivalent for go. You put all dependencies at the top of your source code in an import statement. There are other things you can do if you want to lock a specific version of an API.
- TheDong 3y ago
  Okay, let me try that: $ echo -e 'package main\n\nimport _ "golang.org/x/image/draw"' > main.go $ go run main.go main.go:3:8: no required module provides package golang.org/x/image/draw: go.mod file not found in current directory or any parent directory; see 'go help modules' $ go build main.go <same error> go.mod seems pretty required. With rust, you can also technically just call 'rustc' and never use 'cargo', but just like it's a pain to do that in go, it's a pain to do that in rust.
  - silisili 3y ago
    With Go at least, just prepend GO111MODULE=off and it works. A bit esoteric, but I like that option, especially when working on an application and its dependencies in parallel at times.
2h 3y ago
sibling comment is wrong. this is the typical flow on a new package: 1. go mod init something # create go.mod 2. go mod tidy # create go.sum 3. have fun

curtis3389 3y ago

I was expecting something about how you can't move code into a new file without creating a new module, which means you'll need to modify calling code if you want to reorganize stuff, so you end up with massive source files.

drewtato 3y ago
You can fix this by doing `pub use module::*`.
- 2h 3y ago
  wildcard import, definitely no drawbacks to those!
eftychis 3y ago
You mean that if you split a file you define another module when you declare it? That is only visible as a module at the level of declaration not beyond. Also note that modules are not per se compile units. Cf. See https://doc.rust-lang.org/rust-by-example/mod/split.html https://doc.rust-lang.org/rust-by-example/mod/split.html Edit: typo.

phkahler 3y ago

1) he never says what bad practices are encouraged. A crate can have many source files, so what's wrong with big crates? It's better than npm with tons of stupid small dependencies. 2) what if 2 modules in go share a common dependency? It's not clear to me how that's handled in a file system hierarchy. But thats beside the point anyway.

armchairhacker 3y ago

Powerful features encourage bad practices. This is another "Rust empowers the developer, Go protects the developer from themselves" (and empowerment isn't always a good thing and protection-from-self a bad thing, e.g. goto, type systems)

dimonomid 3y ago
First of all, what are other examples of "Rust empowers the developer, Go protects the developer from themselves"? As article mentions, Rust as a language is actually must stricter than Go, C++ and many others, and Rust community likes to brag that by restricting what we can do in safe Rust, we end up with better software. And I agree with that. But when it comes to dependency management, Rust turns out to be surprisingly lax. Also, in this case, Go doesn't exactly protect the developer from anything. One could create a single package with tons of files in them, and it'll keep working. But the way Go package system is designed, it just subtly suggests to the developer that it's not the right thing to do: having a package (i.e. directory) with 200+ files in it just feels wrong by default. One trait of a well-designed system is that it's difficult to abuse.

epage 3y ago

> So while Go encourages having reasonably sized packages, Rust unfortunately encourages huge crates. I'm a bit confused by this as I'm used to seeing the opposite complaint, that Rust packages are too small. Maybe if there were examples it would help. And while I do think we should explore ways to allow lower effort packages ("cargo script" is one component of this [0]), I see allowing submodules as a feature though maybe thats my Python/C++ background showing through. I do like that it allows both low overhead modules in your API (just a file) or allows you to separate a module out into multiple files for clearer organization (where cycles become important). While I don't know if there is a feasible design for this, but I do wish we could specify what modules are cyclic vs acyclic. [0] https://github.com/rust-lang/rfcs/pull/3424 https://github.com/rust-lang/rfcs/pull/3424

dimonomid 3y ago
> 'm a bit confused by this as I'm used to seeing the opposite complaint, that Rust packages are too small. By "packages" here you're probably referring to the public crates on crates.io, right? Indeed, those public crates are not big in general, but that's not the point of the article. The article is rather talking about large, multi-crate projects, organized as a Cargo workspace. I agree it could have been clearer in the article.

duped 3y ago

There is one giant issue with Rust crates vs modules, and it's not "crates are harder to make." The real problem is that you can't use crates to improve incrementalism when you have a type in one logical part of the module hierarchy implement a trait from another logical part. Due to orphan rules and coherence, it is not possible to implement a foreign trait for a foreign type. And that said, just because the compilation unit is a crate doesn't mean it can't be compiled incrementally or with parallelism. That's a separate problem.

rnestler 3y ago
> The real problem is that you can't use crates to improve incrementalism when you have a type in one logical part of the module hierarchy implement a trait from another logical part. Due to orphan rules and coherence, it is not possible to implement a foreign trait for a foreign type. Well you can implement a trait for a type from another logical part either in the crate of the trait or the crate of the type. What is the use-case for implementing a foreign trait for a foreign type if you have both under control?

solidsnack9000 3y ago

It may be that what Rust allows here is not perfect, but it is not exactly bad. Certainly, compared to most languages, there is nothing abnormal about allowing many files in a tree of directories in a crate, gem, egg, rock, package, module, &c.

insanitybit 3y ago

> So while Go encourages having reasonably sized packages, Rust unfortunately encourages huge crates. Interesting since I think Rust very much encourages the opposite, which is why so many crates like hyper, regex, and others, are actually composed of many crates. I don't really get the main thrust of the article. Go forces directories to map to packages. That sounds... awful. I guess it "encourages small packages" because you can't even organize them into folders? IDK, I feel like I'm missing something - particularly, I'm missing why big or small is good or bad, or why creating a crate is onerous.

pie_flavor 3y ago

What goes into another crate vs another module is an API architecture question; you can shake your fist at (overblown) compilation times but it doesn't make it bad practice to make stuff that's semantically a library, a library. As an example of good practice, Go is pointed to, which is an ecosystem full of people splitting lots of stuff into separate libraries that make no sense to the end user as a separate library and in practice serve the role of submodules. This I would call bad practice, if not for the fact that Go disallows you from doing it any other way. If Go had a central package index instead of just using GitHub, it would be a very confusing place, full of 'private submodules'.

ithkuil 3y ago
Indeed Go's "internal" packages serve the need to create packages that are not libraries
mobiuscog 3y ago
leftpad
foldr 3y ago
https://dave.cheney.net/2019/10/06/use-internal-packages-to-reduce-your-public-api-surface https://dave.cheney.net/2019/10/06/use-internal-packages-to-...

eftychis 3y ago

I think the author is unaware of a) how to create crates easily b) that workspaces exist to organize crates I wouldn't pick Go as an example of how to build a module system. Far from it. ML languages yes. Anyways, adding a single line command per compile unit package is not a "cost." Edit: P.S. Also the module logical system in Rust is way much more expressive.

vpastore 3y ago

[dead]

pjmlp 3y ago

Rust 's flexibility exists for a reason, and it isn't as if the whole Go modules aren't without issues. Not to mention the whole bonkers idea of having package names bound to Github repos and source only.

thumbuddy 3y ago
The people cheering on Go here surprise me a bit. I don't hate Go, but I have spent way more time reading stack overflow posts trying to get a go project to be glued together, fiddling with environment variables, copy pasting code around then I ever have in rust. Again not claiming superiority of one tool over another... But the user experience for rust has been a lot better for project organization for me then most languages. My biggest gripe is, the rust docs somehow make this incredibly difficult language easy to learn, but the sections around code organization are really really hard for beginners or anyone who hasn't sunk time into it and completely omit the 10 meter view of how to organize rust code.

xpe 3y ago

> You might have noticed that when I'm talking about modules being organized as a tree, I say that it seems to be neatly organized. Because the dependencies are acyclic there, modules aren't actually organized, this tree structure merely provides an illusion, while underneath there is likely a dependency mess, and there can be hundreds of files in that mess, and Rust seemingly encourages that; otherwise why would we need modules to be organized in a tree in the first place. Personally, this strikes me as bizarre. It is only an "illusion" if one expects a module tree structure to map one-to-one with dependencies. No one should expect this, as I will explain with three uncontroversial and widely known points: A. Are there any languages where parent-child relationships between modules necessarily imply dependency semantics? I can't think of any, nor can I envision any sane language doing so. [Note 1] B. In Rust, like with so many languages, when a module is nested in another module, that means the inner module is scoped inside the outer module. It is about namespacing, not dependencies. C. Dependencies are created when the contents of something in a module refer to another module. [Note 1]: How would such a language specify more than one dependency? A node in a tree can only have one parent, after all. This would rule out a one-to-one mapping.

smilekzs 3y ago

To me (a C++ daily driver and Rust hobbyist) this "DAG of strongly-connected" tiered graph system is sane and maps well to practice --- tightly couple your impl details however you want, then expose a clean interface at the crate level. It encourages you to split your code to separate files as you write _without immediately forcing you to untangle the dependency_, and then as "islands" of code organically form, which pieces belong to a separate crate becomes naturally evident. Contrast this with C++ where the DAG relationship is enforced at much lower levels (e.g. Bazel cc_library, CMake target, which in normal cases correspond to a cc/h pair), shifting the friction (of having to deal with deps) much earlier in the dev cycle, which then discourages you from (incrementally) splitting a file at all. I've seen too many times this resulting in 5000+ LoC cc file.

pjmlp 3y ago
Modules are similar high level. Naturally you can only enjoy them today on VC++, but the rest will follow.