11 ms·

Zig, Rust, and Other Languages

139 points by tim_sw 3y ago

alchemio 3y ago
Just a correction: most std C functions don’t allocate. strdup does but it was only recently adopted into the standard, it was previously an extension. Similarly zig’s stdlib shouldn’t allocate behind your back, except for thread spawn where it does: https://github.com/ziglang/zig/blob/5cd7fef17faa2a40c8da23f0ef2485df0af39ed4/lib/std/Thread.zig#L666 https://github.com/ziglang/zig/blob/5cd7fef17faa2a40c8da23f0... Generally speaking, it’s as mentioned just a convention. A zig library might not allow its users to pass allocators for example. In C++, stl containers can take an allocator as a template parameter. Recent C++ versions also provide several polymorphic allocators in the stdlib. You can also override the global allocator or a specific class’ allocator (override placement new).
- AndyKelley 3y ago
  For spawning a thread you're literally asking for the stack of the thread to be memory mapped. I fail to see how this is "behind your back". You also linked specifically to the POSIX threads implementation of thread spawning, which is by definition supposed to play nicely with the libc posix threads API, which expects you to use the libc allocator in combination with POSIX threads API, so that's what it does. You might as well accuse the mmap() function in the zig standard library of allocating behind your back.
  - MindSpunk 3y ago
    The behind your back part is probably referring to the Args payload bouncing through a heap allocation. It isn't explicit on the signature it's making an allocation. The function has no choice though unless you leave it up to the user to keep the payload allocation live until the thread terminates.
    - alchemio 3y ago
      Exactly
  - alchemio 3y ago
    It’s something Zig touts when compared to other languages(1). The idea is that in the end it’s a convention that an allocator needs to be passed to indicate that the function allocates, which not even the stdlib adheres religiously to. I’m fine with it since I do believe a library writer should know best what works with their library. 1. https://ziglang.org/learn/why_zig_rust_d_cpp/#no-hidden-allocations https://ziglang.org/learn/why_zig_rust_d_cpp/#no-hidden-allo...
  - fsckboy 3y ago
    he was pointing out that it does, and he was not applying to it the label "behind your back, that's why he said "except for". the wording makes perfect sense.
    - rootlocus 3y ago
      > Similarly zig’s stdlib shouldn’t allocate behind your back, except for thread spawn where it does shouldn't X, exceptor for Y where it does [X] It makes perfect sense that GP was saying "thread spawn allocates behind your back".
- jcelerier 3y ago
  > strdup does but it was only recently adopted into the standard, it was previously an extension. Does it really matter when people were already writing code with strdup when the zig and rust creators were in middle school ? It was already there in BSD 4.3 (1986) apparently.
  - alchemio 3y ago
    It really doesn’t matter and that’s my point.
  - uecker 3y ago
    GNU obstacks also already exist for as long as I can remember. Everything old is new again...
khaledh 3y ago
Nim is also a strong player as a systems programming language. In terms of memory management, it's configurable, and by default you get ARC (no GC). I've written a hobby kernel (if you can call it that) in Nim[1] as well as Zig[2], and I found Nim to be much more ergonomic and approachable. The fact that Zig requires weaving an allocator through most calls that may allocate gets in the way of what I'm trying to do. I'd rather focus on core logic and have ref counting take care of dropping memory when appropriate. One thing I wish Nim had though is true sum types with payloads. I think there's an RFC for that, but it's a shame it's not in the language yet. [1] https://github.com/khaledh/axiom https://github.com/khaledh/axiom [2] https://github.com/khaledh/axiom-zig https://github.com/khaledh/axiom-zig
- loctal 3y ago
  Gleam is another strong candidate too
  - lokelow 3y ago
    Gleam runs on the Erlang VM, and I believe it also compiles to WASM. That's pretty different than the other languages here. Cool language though, I am a big fan of the BEAM
- throwawaymaths 3y ago
  > The fact that Zig requires weaving an allocator through most calls It does no such thing. Why didn't you just make a global allocator? Alternatively, you can have your allocator be attached to your objects.
slily 3y ago
> People have been making jokes about node_modules for a decade now, but this problem is just as bad in Rust codebases I've seen I agree... but I think this is more indicative of a cultural problem than a language design/standard library scoping problem. A compact standard library is more resistant to scope creep. I don't want every language to end up like C++, and I think keeping the scope of the standard library small helps avoid that. On the other hand, popular third-party libraries that provide common functions have too many third-(fourth-?)party dependencies. Keeping dependency trees small should be a priority for such tools, but convenience trumps all right now so it isn't valued as it should be.
- no_wizard 3y ago
  one way to approach this with compromise is to keep the shipped standard library small, but have officially maintained packages from the language foundation for exceedingly common functionality. That gives an avenue for exploration, and tight controls and quality control on packages, without bloating the standard library.
  - MBCook 3y ago
    What’s so bad about “bloating the standard library”? Java has an incredible amount of stuff and it’s incredibly useful. The compiler should be able to eliminate the unused parts during compilation, it’s not like adding JSON to the standard library is going to make a “hello world” program bigger.
    - shit_game 3y ago
      >The compiler should be able to eliminate the unused parts during compilation, it’s not like adding JSON to the standard library is going to make a “hello world” program bigger. I agree. I would argue against keeping standard libraries terse as an appropriate goal for a language because terse (or alternatively, inadequate) standard libraries are exactly what necessitate third party libraries for common functionality, which in turn lead to this dependency hell that so many people dislike so much. Not to say that a standard library should do absolutely everything, but the more capable a language is out of the box, the fewer third party libraries a language ecosystem will come to rely on, meaning that methodologies will be more consistent in more code and dependency management will be simpler.
Thaxll 3y ago
Zig does not have a good package manager, does it?
- bsder 3y ago
  I don't believe Zig wants to have "package manager" at all. It has the ability to pull in a package/module. However, I believe that having a "package manager" is explicitly an anti-goal. I'm in agreement. It's become pretty clear that having any form of "package repository" without also allocating the necessary manpower to curate that repository causes disasters.
  - wavemode 3y ago
    > I don't believe Zig wants to have "package manager" at all. ? it's literally one of the project's main milestones: https://github.com/ziglang/zig/projects?type=classic https://github.com/ziglang/zig/projects?type=classic
    - bsder 3y ago
      We're probably talking past one another due to my imprecision: Zig can manage packages. It can pull in a Zig package using a reference to a Git repository on GitHub, for example. It can pin that to a hash and cache it. It can specify dependencies. This is, strictly speaking, a "package manager" but is not what people normally think of when you mention that. Zig does not have blessed mechanisms that pull from a central repository of packages. Having a central repository is normally what people think of when they talk about a "package manager" (node, cargo, deno, etc.). Yes, those managers could conceivably not use the central repository and specify everything explicitly, but nobody ever uses them like that.
      - papichulo2023 3y ago
        So same as Golang? Still package manager though.
  - baby 3y ago
coldtea 3y ago
>Zig is not a mature language. But it has made enough useful choices for a number of companies to invest in it and run it in production. The useful choices make Zig worth talking about.Go and Rust are mature languages. But they have both made questionable choices that seem worth talking about. Well, Zig's native string type support, or lack thereof, is also a questionable choice. It's not like Zig did all the right choices and only Go and Rust made questionable ones.
- bsder 3y ago
  Everybody says they want a built-in String type. And then everybody proceeds to write their own String library anyway. "The only winning move is not to play."
  - pornel 3y ago
    Rust has a nice solution of having a &str slice (equivalent of C++ std::string_view) as the lowest common denominator and a deref operator that can easily coerce various string types to the slice. This allows Rust to have all kinds of string flavors (fixed-len or NUL-terminated or growable, on stack or heap or in ROM, with SSO, with CoW, interned or refcounted, atomically or not, and so on), but they all coerce to the basic &str, so they have the same basic methods, and are compatible with most functions that just want a string without caring how it's allocated. You can use your own weird string type if you want, but usually you're not forced to convert or copy it to use it with standard library functions or 3rd party dependencies.
    - josephg 3y ago
      Yeah - although the cost of this is that it makes rust harder to learn. (Try answering the common beginner question: “How is &String different from &str?”) In rust Strings are usually passed in to functions as a &str, and returned from functions as String. It makes sense once you’re used to it, but I think philosophically Rust is much closer to C++ than C. Rust is missing the elegant minimalism of C or Zig.
      - 3y ago
Arnavion 3y ago
>Zig is practically alone in that if you write the next() method and and don't pass an allocator to any method in the next() body, nothing in that next() method will allocate. It's not quite as simple as that. This can allocate just fine: fn next(self: *Self) { self.array_list.append(5); } ... where `array_list` is of type `std.ArrayList(u32)`. `std.ArrayList(T)` takes an allocator at construction time (`array_list = std.ArrayList(u32).init(allocator);`) and stores it as a field, so its methods don't need to take an allocator parameter. (And yes, there is a version of `std.ArrayList(T)` called `std.ArrayListUnmanaged(T)` that does *not* take an allocator in its constructor and does take an allocator in all its methods that need to allocate.)
- AndyKelley 3y ago
  That would not compile because the append operation can fail, and errors cannot be ignored.
  - tialaramex 3y ago
    The exact code written won't compile but it gets the idea across - the original claim from the blog post was just false as far as I can tell. We can't in fact be sure that no allocation took place just because we didn't provide an allocator. It's a convention at most, probably for code that's written by Zig programmers who prefer this style, it's true but you could say the same about any convention for any language.
    - Arnavion 3y ago
      Right. The "will this or won't this allocate" analysis also needs to take into account all values being used by the code because they might be using allocators internally. I haven't written zig code in a few years so I don't know if people have developed a convention for it. That said, there is a reason types like `ArrayListUnmanaged` exist. If you have some complex type with a bunch of different collections as fields, you probably want to use the same allocator for all of those collections, so there's no reason to have the collections themselves store redundant copies of the allocator. Instead you can have the allocator held by the top-level type and use the `Unmanaged` collections as fields. This also means that you do end up needing to reference `self.allocator` explicitly when calling `.append()` etc, satisfying the original claim of TFA.
rwbt 3y ago
There's also Odin[0] too. I experimented them all and Odin is pretty nice. Nim is also good too but a lot more features. But - I concluded that language matters a lot less compared to APIs. Yes, the language should have enough good features to let the programmers express themselves, but overall well designed APIs matter a lot more than language. For example -tossing most of the C stdlib and following a consistent coding style (similar to one described here -[1]), with using Arenas for memory allocation, I can be just as productive in C. [0] - https://odin-lang.org https://odin-lang.org [1] - https://nullprogram.com/blog/2023/10/08/ https://nullprogram.com/blog/2023/10/08/
duped 3y ago
> People have been making jokes about node_modules for a decade now, but this problem is just as bad in Rust codebases I've seen. Just w.r.t the issue of size (not of scale) - cargo caches sources in ~/.cargo so they're not shared by every project on the system. Additionally, rustc uses dead code elimination by design so there's not nearly as bad an issue as in JS where it's not possible to tree shake out every unused class or method. Most of the bloat of target directories are stale incremental build artifacts, because cargo does not do any automatic cleanup.
- ViewTrick1002 3y ago
  Automatic GC of the cargo cache is available on nightly. Really looking forward to it landing on stable. For large projects I've seen the cache grow to hundreds of GB, and not noticing it until the compilation fails because the disk is full... https://blog.rust-lang.org/2023/12/11/cargo-cache-cleaning.html https://blog.rust-lang.org/2023/12/11/cargo-cache-cleaning.h...
  - galangalalgol 3y ago
    I've had the same issue with conan, and realizing only the. That is why our pipeline took so long, copying tens of GB to the runner. Cargo is mild by comparison.
- deleted 3y ago
  [deleted]
thayne 3y ago
WRT the standard library. There are tradeoffs. Yes, a big standard library that has everything you need is great, as long as it is well designed, well maintained, and has some mechanism to prevent the whole thing bloating every executable. But executing well on that is difficult for a number of reasons: - the release cycle of the standard library is (usually) tied to the compuler, which often means it can't evolve quickly. - backwards compatibility is a much bigger deal for the standard library. Which means if you have a big library you will eventually have a big pile of deprecated APIs you will probably never be able to actually remove. It also feeds into the next point - new development in stdlib can be hindered by the need to get it right the first time. - the creators of the language probably aren't experts in every area. Which means in order to include say, a good compression library you either need to recruit someone who is an expert to write and maintain the package in your stdlib, or link to a library in another language, and the latter is still really an external depency. - a large stdlib is a large maintenance burden Python has a large standard library, but it has parts that are deprecated and/or largely abandoned, including packages for technologies that are no longer widely used. And its http packages are rarely used directly because third party packages like requests or urllib3 are better. Go's standard library that is praised here isn't perfect either. The log and flag packages are often insufficient, and the implementation of net.IP is suboptimal [1]. I think probably the best balance is to start with a fairly small standard library, and when community libraries for key functionality become popular and stable, then pull them in to the standard library or otherwise make them official. And maybe have official libraries that are external to the stdlib and maybe have less rigid backwards compatibility garantees that can move faster than the language itself. [1]: https://tailscale.com/blog/netaddr-new-ip-type-for-go https://tailscale.com/blog/netaddr-new-ip-type-for-go
- klabb3 3y ago
  > The log […] There’s log/slog now. Is that sufficient? > and flag packages are often insufficient […] Yes. Flag is really primitive and awkward to use. On the other hand, in a typical “imperative shell” style CLI, sharing flags between packages is not exactly crucial, which is the main reason to put it in std (interop). I’ve never really suffered from importing a 3p flag parser in my own code. > and the implementation of net.IP is suboptimal [1]. Fixed for a long time. See net/netip (although I am annoyed it’s not universally available in all net functions). It’s based on the implementation from your link.
eviks 3y ago
> 3 years? 68 lines of code. Is it not safer at this point to vendor that code? Why, though? There is no bitrot or build flakiness with unnecessary changes if no changes happen, so what's the actual argument for this? Why is it missing from the article?
lamontcg 3y ago
> Similarly, if you go looking for compression support in Rust, there's none in the standard library. I have no idea why that is considered a criticism. Compression support is the kind of thing which is going to evolve over time and compression formats will change and performance and APIs of different libraries will get better. You avoid the problem where the compression library in the std library is the one that experienced programmers tell all the noobs that they should never use.
- aldanor 3y ago
  I'd replace "similarly" with "fortunately"
- electroly 3y ago
  > You avoid the problem where the compression library in the std library is the one that experienced programmers tell all the noobs that they should never use. This is not inevitable. In the .NET world we have great compressors in the standard library. Zip and gzip aren't going anywhere. I totally disagree with the idea that we would have been better off without those compressors in the standard library.

139 points by tim_sw 3y ago

Just a correction: most std C functions don’t allocate. strdup does but it was only recently adopted into the standard, it was previously an extension. Similarly zig’s stdlib shouldn’t allocate behind your back, except for thread spawn where it does: https://github.com/ziglang/zig/blob/5cd7fef17faa2a40c8da23f0ef2485df0af39ed4/lib/std/Thread.zig#L666 https://github.com/ziglang/zig/blob/5cd7fef17faa2a40c8da23f0... Generally speaking, it’s as mentioned just a convention. A zig library might not allow its users to pass allocators for example. In C++, stl containers can take an allocator as a template parameter. Recent C++ versions also provide several polymorphic allocators in the stdlib. You can also override the global allocator or a specific class’ allocator (override placement new).

AndyKelley 3y ago
For spawning a thread you're literally asking for the stack of the thread to be memory mapped. I fail to see how this is "behind your back". You also linked specifically to the POSIX threads implementation of thread spawning, which is by definition supposed to play nicely with the libc posix threads API, which expects you to use the libc allocator in combination with POSIX threads API, so that's what it does. You might as well accuse the mmap() function in the zig standard library of allocating behind your back.
- MindSpunk 3y ago
  The behind your back part is probably referring to the Args payload bouncing through a heap allocation. It isn't explicit on the signature it's making an allocation. The function has no choice though unless you leave it up to the user to keep the payload allocation live until the thread terminates.
  - alchemio 3y ago
    Exactly
- alchemio 3y ago
  It’s something Zig touts when compared to other languages(1). The idea is that in the end it’s a convention that an allocator needs to be passed to indicate that the function allocates, which not even the stdlib adheres religiously to. I’m fine with it since I do believe a library writer should know best what works with their library. 1. https://ziglang.org/learn/why_zig_rust_d_cpp/#no-hidden-allocations https://ziglang.org/learn/why_zig_rust_d_cpp/#no-hidden-allo...
- fsckboy 3y ago
  he was pointing out that it does, and he was not applying to it the label "behind your back, that's why he said "except for". the wording makes perfect sense.
  - rootlocus 3y ago
    > Similarly zig’s stdlib shouldn’t allocate behind your back, except for thread spawn where it does shouldn't X, exceptor for Y where it does [X] It makes perfect sense that GP was saying "thread spawn allocates behind your back".
jcelerier 3y ago
> strdup does but it was only recently adopted into the standard, it was previously an extension. Does it really matter when people were already writing code with strdup when the zig and rust creators were in middle school ? It was already there in BSD 4.3 (1986) apparently.
- alchemio 3y ago
  It really doesn’t matter and that’s my point.
- uecker 3y ago
  GNU obstacks also already exist for as long as I can remember. Everything old is new again...

khaledh 3y ago

Nim is also a strong player as a systems programming language. In terms of memory management, it's configurable, and by default you get ARC (no GC). I've written a hobby kernel (if you can call it that) in Nim[1] as well as Zig[2], and I found Nim to be much more ergonomic and approachable. The fact that Zig requires weaving an allocator through most calls that may allocate gets in the way of what I'm trying to do. I'd rather focus on core logic and have ref counting take care of dropping memory when appropriate. One thing I wish Nim had though is true sum types with payloads. I think there's an RFC for that, but it's a shame it's not in the language yet. [1] https://github.com/khaledh/axiom https://github.com/khaledh/axiom [2] https://github.com/khaledh/axiom-zig https://github.com/khaledh/axiom-zig

loctal 3y ago
Gleam is another strong candidate too
- lokelow 3y ago
  Gleam runs on the Erlang VM, and I believe it also compiles to WASM. That's pretty different than the other languages here. Cool language though, I am a big fan of the BEAM
throwawaymaths 3y ago
> The fact that Zig requires weaving an allocator through most calls It does no such thing. Why didn't you just make a global allocator? Alternatively, you can have your allocator be attached to your objects.

slily 3y ago

> People have been making jokes about node_modules for a decade now, but this problem is just as bad in Rust codebases I've seen I agree... but I think this is more indicative of a cultural problem than a language design/standard library scoping problem. A compact standard library is more resistant to scope creep. I don't want every language to end up like C++, and I think keeping the scope of the standard library small helps avoid that. On the other hand, popular third-party libraries that provide common functions have too many third-(fourth-?)party dependencies. Keeping dependency trees small should be a priority for such tools, but convenience trumps all right now so it isn't valued as it should be.

no_wizard 3y ago
one way to approach this with compromise is to keep the shipped standard library small, but have officially maintained packages from the language foundation for exceedingly common functionality. That gives an avenue for exploration, and tight controls and quality control on packages, without bloating the standard library.
- MBCook 3y ago
  What’s so bad about “bloating the standard library”? Java has an incredible amount of stuff and it’s incredibly useful. The compiler should be able to eliminate the unused parts during compilation, it’s not like adding JSON to the standard library is going to make a “hello world” program bigger.
  - shit_game 3y ago
    >The compiler should be able to eliminate the unused parts during compilation, it’s not like adding JSON to the standard library is going to make a “hello world” program bigger. I agree. I would argue against keeping standard libraries terse as an appropriate goal for a language because terse (or alternatively, inadequate) standard libraries are exactly what necessitate third party libraries for common functionality, which in turn lead to this dependency hell that so many people dislike so much. Not to say that a standard library should do absolutely everything, but the more capable a language is out of the box, the fewer third party libraries a language ecosystem will come to rely on, meaning that methodologies will be more consistent in more code and dependency management will be simpler.

Thaxll 3y ago

Zig does not have a good package manager, does it?

bsder 3y ago
I don't believe Zig wants to have "package manager" at all. It has the ability to pull in a package/module. However, I believe that having a "package manager" is explicitly an anti-goal. I'm in agreement. It's become pretty clear that having any form of "package repository" without also allocating the necessary manpower to curate that repository causes disasters.
- wavemode 3y ago
  > I don't believe Zig wants to have "package manager" at all. ? it's literally one of the project's main milestones: https://github.com/ziglang/zig/projects?type=classic https://github.com/ziglang/zig/projects?type=classic
  - bsder 3y ago
    We're probably talking past one another due to my imprecision: Zig can manage packages. It can pull in a Zig package using a reference to a Git repository on GitHub, for example. It can pin that to a hash and cache it. It can specify dependencies. This is, strictly speaking, a "package manager" but is not what people normally think of when you mention that. Zig does not have blessed mechanisms that pull from a central repository of packages. Having a central repository is normally what people think of when they talk about a "package manager" (node, cargo, deno, etc.). Yes, those managers could conceivably not use the central repository and specify everything explicitly, but nobody ever uses them like that.
    - papichulo2023 3y ago
      So same as Golang? Still package manager though.
- baby 3y ago

coldtea 3y ago

>Zig is not a mature language. But it has made enough useful choices for a number of companies to invest in it and run it in production. The useful choices make Zig worth talking about.Go and Rust are mature languages. But they have both made questionable choices that seem worth talking about. Well, Zig's native string type support, or lack thereof, is also a questionable choice. It's not like Zig did all the right choices and only Go and Rust made questionable ones.

bsder 3y ago
Everybody says they want a built-in String type. And then everybody proceeds to write their own String library anyway. "The only winning move is not to play."
- pornel 3y ago
  Rust has a nice solution of having a &str slice (equivalent of C++ std::string_view) as the lowest common denominator and a deref operator that can easily coerce various string types to the slice. This allows Rust to have all kinds of string flavors (fixed-len or NUL-terminated or growable, on stack or heap or in ROM, with SSO, with CoW, interned or refcounted, atomically or not, and so on), but they all coerce to the basic &str, so they have the same basic methods, and are compatible with most functions that just want a string without caring how it's allocated. You can use your own weird string type if you want, but usually you're not forced to convert or copy it to use it with standard library functions or 3rd party dependencies.
  - josephg 3y ago
    Yeah - although the cost of this is that it makes rust harder to learn. (Try answering the common beginner question: “How is &String different from &str?”) In rust Strings are usually passed in to functions as a &str, and returned from functions as String. It makes sense once you’re used to it, but I think philosophically Rust is much closer to C++ than C. Rust is missing the elegant minimalism of C or Zig.
    - 3y ago

Arnavion 3y ago

>Zig is practically alone in that if you write the next() method and and don't pass an allocator to any method in the next() body, nothing in that next() method will allocate. It's not quite as simple as that. This can allocate just fine: fn next(self: *Self) { self.array_list.append(5); } ... where `array_list` is of type `std.ArrayList(u32)`. `std.ArrayList(T)` takes an allocator at construction time (`array_list = std.ArrayList(u32).init(allocator);`) and stores it as a field, so its methods don't need to take an allocator parameter. (And yes, there is a version of `std.ArrayList(T)` called `std.ArrayListUnmanaged(T)` that does *not* take an allocator in its constructor and does take an allocator in all its methods that need to allocate.)

AndyKelley 3y ago
That would not compile because the append operation can fail, and errors cannot be ignored.
- tialaramex 3y ago
  The exact code written won't compile but it gets the idea across - the original claim from the blog post was just false as far as I can tell. We can't in fact be sure that no allocation took place just because we didn't provide an allocator. It's a convention at most, probably for code that's written by Zig programmers who prefer this style, it's true but you could say the same about any convention for any language.
  - Arnavion 3y ago
    Right. The "will this or won't this allocate" analysis also needs to take into account all values being used by the code because they might be using allocators internally. I haven't written zig code in a few years so I don't know if people have developed a convention for it. That said, there is a reason types like `ArrayListUnmanaged` exist. If you have some complex type with a bunch of different collections as fields, you probably want to use the same allocator for all of those collections, so there's no reason to have the collections themselves store redundant copies of the allocator. Instead you can have the allocator held by the top-level type and use the `Unmanaged` collections as fields. This also means that you do end up needing to reference `self.allocator` explicitly when calling `.append()` etc, satisfying the original claim of TFA.

rwbt 3y ago

There's also Odin[0] too. I experimented them all and Odin is pretty nice. Nim is also good too but a lot more features. But - I concluded that language matters a lot less compared to APIs. Yes, the language should have enough good features to let the programmers express themselves, but overall well designed APIs matter a lot more than language. For example -tossing most of the C stdlib and following a consistent coding style (similar to one described here -[1]), with using Arenas for memory allocation, I can be just as productive in C. [0] - https://odin-lang.org https://odin-lang.org [1] - https://nullprogram.com/blog/2023/10/08/ https://nullprogram.com/blog/2023/10/08/

duped 3y ago

> People have been making jokes about node_modules for a decade now, but this problem is just as bad in Rust codebases I've seen. Just w.r.t the issue of size (not of scale) - cargo caches sources in ~/.cargo so they're not shared by every project on the system. Additionally, rustc uses dead code elimination by design so there's not nearly as bad an issue as in JS where it's not possible to tree shake out every unused class or method. Most of the bloat of target directories are stale incremental build artifacts, because cargo does not do any automatic cleanup.

ViewTrick1002 3y ago
Automatic GC of the cargo cache is available on nightly. Really looking forward to it landing on stable. For large projects I've seen the cache grow to hundreds of GB, and not noticing it until the compilation fails because the disk is full... https://blog.rust-lang.org/2023/12/11/cargo-cache-cleaning.html https://blog.rust-lang.org/2023/12/11/cargo-cache-cleaning.h...
- galangalalgol 3y ago
  I've had the same issue with conan, and realizing only the. That is why our pipeline took so long, copying tens of GB to the runner. Cargo is mild by comparison.
deleted 3y ago
[deleted]

thayne 3y ago

WRT the standard library. There are tradeoffs. Yes, a big standard library that has everything you need is great, as long as it is well designed, well maintained, and has some mechanism to prevent the whole thing bloating every executable. But executing well on that is difficult for a number of reasons: - the release cycle of the standard library is (usually) tied to the compuler, which often means it can't evolve quickly. - backwards compatibility is a much bigger deal for the standard library. Which means if you have a big library you will eventually have a big pile of deprecated APIs you will probably never be able to actually remove. It also feeds into the next point - new development in stdlib can be hindered by the need to get it right the first time. - the creators of the language probably aren't experts in every area. Which means in order to include say, a good compression library you either need to recruit someone who is an expert to write and maintain the package in your stdlib, or link to a library in another language, and the latter is still really an external depency. - a large stdlib is a large maintenance burden Python has a large standard library, but it has parts that are deprecated and/or largely abandoned, including packages for technologies that are no longer widely used. And its http packages are rarely used directly because third party packages like requests or urllib3 are better. Go's standard library that is praised here isn't perfect either. The log and flag packages are often insufficient, and the implementation of net.IP is suboptimal [1]. I think probably the best balance is to start with a fairly small standard library, and when community libraries for key functionality become popular and stable, then pull them in to the standard library or otherwise make them official. And maybe have official libraries that are external to the stdlib and maybe have less rigid backwards compatibility garantees that can move faster than the language itself. [1]: https://tailscale.com/blog/netaddr-new-ip-type-for-go https://tailscale.com/blog/netaddr-new-ip-type-for-go

klabb3 3y ago
> The log […] There’s log/slog now. Is that sufficient? > and flag packages are often insufficient […] Yes. Flag is really primitive and awkward to use. On the other hand, in a typical “imperative shell” style CLI, sharing flags between packages is not exactly crucial, which is the main reason to put it in std (interop). I’ve never really suffered from importing a 3p flag parser in my own code. > and the implementation of net.IP is suboptimal [1]. Fixed for a long time. See net/netip (although I am annoyed it’s not universally available in all net functions). It’s based on the implementation from your link.

eviks 3y ago

> 3 years? 68 lines of code. Is it not safer at this point to vendor that code? Why, though? There is no bitrot or build flakiness with unnecessary changes if no changes happen, so what's the actual argument for this? Why is it missing from the article?

lamontcg 3y ago

> Similarly, if you go looking for compression support in Rust, there's none in the standard library. I have no idea why that is considered a criticism. Compression support is the kind of thing which is going to evolve over time and compression formats will change and performance and APIs of different libraries will get better. You avoid the problem where the compression library in the std library is the one that experienced programmers tell all the noobs that they should never use.

aldanor 3y ago
I'd replace "similarly" with "fortunately"
electroly 3y ago
> You avoid the problem where the compression library in the std library is the one that experienced programmers tell all the noobs that they should never use. This is not inevitable. In the .NET world we have great compressors in the standard library. Zip and gzip aren't going anywhere. I totally disagree with the idea that we would have been better off without those compressors in the standard library.