6 ms·

In the short term, yeah 4x slower coreutils is probably worth it for memory safe code. In the long run, I’m glad rust coreutils is slowly working through these

by josephg 11mo ago

In the short term, yeah 4x slower coreutils is probably worth it for memory safe code. In the long run, I’m glad rust coreutils is slowly working through these compatibility issues so we don’t have to pay for memory safety. Yeah, it'll have new bugs. And right now it’s slightly incompatible. But give it a few more years to cook. I’m sure they’ll get there. As an aside, I find it weird how much negativity rewrites like this get. If someone decided to make a new web browser, C compiler or kernel people would be congratulating them. I really don’t understand the conservatism when it comes to Linux. Is the current implementation perfect? Should it be preserved in amber? The gnu runtime seems like a messy, badly specified hairball of hacky, inconsistent scripts to me. Some guys in a certain room in the 70s and 80s wrote some C programs. And now every bad idea they had lives in perpetuity in my /usr/bin directory? In the decades since, these tools have sprouted hundreds of weird features that almost nobody uses. And now what, people care what language it’s all written in? This code must never be changed?? Who cares.

wavemode 11mo ago
> 4x slower coreutils I doubt this is true in practice. The majority of coreutils spend the majority of their time waiting for the results of IO/syscalls. (The exception would probably be, the hashing utilities like md5sum.)
- stefan_ 11mo ago
  I tried md5sum and sha256sum and there was exactly zero difference in runtime (the Fil-C version of sha256sum was consistently faster, in fact..)
  - josephg 11mo ago
    Why would it be faster? That seems suspicious to me. Is all the code being compiled with the same flags? Shasum probably benefits a lot from intrinsics that are only available on newer CPU targets.
    - 0cf8612b2e1e 11mo ago
      Those are also highly algorithmic tools. Probably few code paths and a lot of vector operations where competing compilers may not have much room to differentiate.
      - josephg 11mo ago
        It really depends based on how shasum is implemented. If its implemented in assembly, it'll perform the same no matter how its compiled. But if its written using C code, the compiler has a lot of latitude to vectorize based on target CPU features. And not just SSE2 and AVX. Even popcnt isn't even available in the baseline x86_64 target for llvm. The Fil-C compiler is a fork of llvm. There's no way all that garbage collection code would make fil-c faster. So if its faster, its probably using different target flags. And in that case, its not a fair benchmark comparison.
    - stefan_ 11mo ago
      I'm sure theres a difference in the binary, for a real comparison you would need to compile the same coreutils version with the same options. I just think the assertion that "compute-heavy" tools like sha256sum would be especially affected by Fil-C is not true, and if that was true given the "baseline slowdown" of 4x, surely it would show up in this sloppy test.
ottah 11mo ago
GNU utils is battle tested, well reviewed, and STABLE. That's really what I want in an OS, stability. Rust solves only one case of security issues, but it cannot solve logical errors, which there will be many of in a new software project. I just don't see what's to gain, to suffer through years of instability, waiting for a userspace suite to mature, and reach feature parity, when we have a well understood, and safe tool set know. Maybe in five years, when coreutils is complete, I'd be okay with Ubuntu replacing user land with it. But we're not there, and it's a problem we shouldn't have to tolerate. Also I can't stand we're leaving GPL code behind for MIT.
- steveklabnik 11mo ago
  Luckily, the existence of uutils doesn’t change the fact that GNU coreutils exists. In fact, it’s helped improve the stability of the GNU coreutils by clarifying intended behavior and adding test cases. So if you prefer them, you should stick to them. Nobody is taking anything from you.
  - ottah 11mo ago
    So I guess to properly clarify, I absolutely do not mind that someone wants to build coreutils in Rust. I don't have a problem with Rust Coreutils existing. The problem, and the real issue I have is that this project is being used as the default in major linux distros. Eager adoption of this project, and making it the production target does take away things from me. The interface has changed, stability is affected. Correctness is now measured against this incomplete implementation first, not the known correct, and stable GNU coreutils.
    - steveklabnik 11mo ago
      That’s not what is happening. One distro is kicking the tires on using this by default. The purpose is exactly because the GNU versions are being treated as the proper versions. Divergences from them are being fixed, so that this new version follows those. You can only do that by actually trying them out, because it’s impossible for the test suite to cover every behavior.

by josephg 11mo ago

wavemode 11mo ago

> 4x slower coreutils I doubt this is true in practice. The majority of coreutils spend the majority of their time waiting for the results of IO/syscalls. (The exception would probably be, the hashing utilities like md5sum.)

stefan_ 11mo ago
I tried md5sum and sha256sum and there was exactly zero difference in runtime (the Fil-C version of sha256sum was consistently faster, in fact..)
- josephg 11mo ago
  Why would it be faster? That seems suspicious to me. Is all the code being compiled with the same flags? Shasum probably benefits a lot from intrinsics that are only available on newer CPU targets.
  - 0cf8612b2e1e 11mo ago
    Those are also highly algorithmic tools. Probably few code paths and a lot of vector operations where competing compilers may not have much room to differentiate.
    - josephg 11mo ago
      It really depends based on how shasum is implemented. If its implemented in assembly, it'll perform the same no matter how its compiled. But if its written using C code, the compiler has a lot of latitude to vectorize based on target CPU features. And not just SSE2 and AVX. Even popcnt isn't even available in the baseline x86_64 target for llvm. The Fil-C compiler is a fork of llvm. There's no way all that garbage collection code would make fil-c faster. So if its faster, its probably using different target flags. And in that case, its not a fair benchmark comparison.
  - stefan_ 11mo ago
    I'm sure theres a difference in the binary, for a real comparison you would need to compile the same coreutils version with the same options. I just think the assertion that "compute-heavy" tools like sha256sum would be especially affected by Fil-C is not true, and if that was true given the "baseline slowdown" of 4x, surely it would show up in this sloppy test.

ottah 11mo ago

GNU utils is battle tested, well reviewed, and STABLE. That's really what I want in an OS, stability. Rust solves only one case of security issues, but it cannot solve logical errors, which there will be many of in a new software project. I just don't see what's to gain, to suffer through years of instability, waiting for a userspace suite to mature, and reach feature parity, when we have a well understood, and safe tool set know. Maybe in five years, when coreutils is complete, I'd be okay with Ubuntu replacing user land with it. But we're not there, and it's a problem we shouldn't have to tolerate. Also I can't stand we're leaving GPL code behind for MIT.

steveklabnik 11mo ago
Luckily, the existence of uutils doesn’t change the fact that GNU coreutils exists. In fact, it’s helped improve the stability of the GNU coreutils by clarifying intended behavior and adding test cases. So if you prefer them, you should stick to them. Nobody is taking anything from you.
- ottah 11mo ago
  So I guess to properly clarify, I absolutely do not mind that someone wants to build coreutils in Rust. I don't have a problem with Rust Coreutils existing. The problem, and the real issue I have is that this project is being used as the default in major linux distros. Eager adoption of this project, and making it the production target does take away things from me. The interface has changed, stability is affected. Correctness is now measured against this incomplete implementation first, not the known correct, and stable GNU coreutils.
  - steveklabnik 11mo ago
    That’s not what is happening. One distro is kicking the tires on using this by default. The purpose is exactly because the GNU versions are being treated as the proper versions. Divergences from them are being fixed, so that this new version follows those. You can only do that by actually trying them out, because it’s impossible for the test suite to cover every behavior.