47 ms·

Uutils: an attempt at writing cross-platform CLI utilities in Rust

mprovost 5y ago
I'm writing a book "Rust From the Ground Up" which teaches Rust by rewriting a coreutil per chapter, from the original BSD sources. The first 3 chapters (true/false, yes, and head) are released. wc is next, then cat, cut, rev, and uniq. https://rftgu.rs/ https://rftgu.rs/
- nickel_8448 5y ago
  The book looks very interesting. By when do you plan to complete the next set of chapters?
  - mprovost 5y ago
    Thanks! I've written the whole book - now I'm going through and editing and typesetting each chapter and trying to publish one a month.
    - nickel_8448 5y ago
      That's wonderful. Do you have twitter? I just want to be notified once the book is complete.
      - mprovost 5y ago
        Yes I’m posting updates here: https://mobile.twitter.com/rustftgu https://mobile.twitter.com/rustftgu Thanks for the interest!
mprovost 5y ago
More interesting than rewriting the GNU coreutils are the projects that are reinventing them. Like ripgrep (which built on the example of ack) is a better grep, by discarding backwards compatibility. Or fd, which abandons find's arcane syntax. We've learned a thing or two since the 70s when most of these tools were first invented.
- hencoappel 5y ago
  Agreed, it would be an interesting idea to make a new coreutils with everything modernised/cleaned up to be easier to use and more consistent.
  - DarylZero 5y ago
    https://www.nushell.sh/ https://www.nushell.sh/
    - GhettoComputers 5y ago
      Tried it too bad no shell compares to fish for ease of use. Every new shell gets asked "can it do the thing in fish" and the answer is always kind of or no, zsh did not function as well even with the configs and it seemed much slower.
MichaelBurge 5y ago
If we replace GNU coreutils[1] with this, could we finally call it "Linux" instead of "GNU/Linux"? [1] And gcc with clang when bootstrapping
- yjftsjthsd-h 5y ago
  Yes:) In fact, if you're into that, I'd argue that Alpine is already not GNU/Linux, since it uses busybox and musl.
- bayindirh 5y ago
  Is getting rid of GNU that important?
  - woodruffw 5y ago
    I’m not sure if this answers the “how important” question, but: a large part of the GNU ecosystem is functionally unmaintained. The various tools under binutils receive so few patches that they’ve been the standard punching bag for academic fuzzing research for the last 20 years, and the coreutils aren’t much better.
  - haimez 5y ago
    About as important as keeping GNU, empirically.
  - bogwog 5y ago
    I think the point of this project is less about getting rid of GNU, and more about promoting Rust. There's nothing wrong with GNU coreutils.
    - adgjlsfhk1 5y ago
      https://www.cvedetails.com/vulnerability-list.php?vendor_id=72&product_id=5075 https://www.cvedetails.com/vulnerability-list.php?vendor_id=.... 4 vulns in the past decade that rust wouldn't have had.
pxc 5y ago
This will probably sound completely insane, not least of all to people who prefer non-GNU coreutils, but I kinda hope that as uutils matures, some shells will offer an option to compile in the uutils rewrites as builtins. Especially for newer shells and shells that want to target Windows in a first-class way, that could be a real portability win.
- deleted 5y ago
  [deleted]
- smt88 5y ago
  Why is WSL[1] insufficient for Windows portability? 1. https://docs.microsoft.com/en-us/windows/wsl/about https://docs.microsoft.com/en-us/windows/wsl/about
  - dagw 5y ago
    WSL is great if you want/need a full Linux environment. It is quite heavy and probably overkill if all you want is to use some coreutil commands from the Windows terminal.
    - tyingq 5y ago
      Gitbash is a relatively nice solution for this.
    - mkl 5y ago
      WSL1 is very light in my experience.
  - pxc 5y ago
    I wasn't primarily concerned with Windows portability, but with the portability of scripts altogether, and no longer having to concern oneself with variations in coreutils versions for at least some scripts, even if they run on multiple platforms. But to answer your question about Windows: Because requiring an emulation layer is a huge step up in complexity from being able to distribute your shell as a single static binary which is just a few MB. Right now Elvish and Nushell, for example, can be deployed on Windows as binaries that don't require any external dependencies, any installation procedure, or any configuration. If they could link against uutils statically, they could also come with basic utilities without having an environment to manage. WSL (along with MSYS2 and Cygwin) is deeply stateful. Now to make sure everything is working you have to check the compatibility of your whole system, and the versions of coreutils can still vary in addition to the version of the shell interpreter. WSL2 also requires virtualization, which can complicate using it under virtual machines. You may not want everything you run under your shell to depend on WSL, which is also happens with WSL2. Even with WSL1, a script that requires a chroot environment to deploy and run it is way less portable than one that doesn't.
signa11 5y ago
why MIT license ? that is bound to be turnoff for a lot of folks. also, a minor nit: there are still a lot of work [1] that needs to be done, and imho, it is a bit premature to title this article as is presented here. [1] https://github.com/uutils/coreutils#utilities https://github.com/uutils/coreutils#utilities
- xvedejas 5y ago
  Is it? I thought it was as standard as and on pretty much equal footing with BSD and Apache licenses. Is it substantially different?
- no_wizard 5y ago
  Who, exactly? I think most don’t care, and big corps love the MIT license
  - GhettoComputers 5y ago
    Don’t they love them all? Free software in general is nice, PS3 ran Linux, was based on BSD, and I’m sure they used MIT software somewhere there too.
    - yjftsjthsd-h 5y ago
      Big corps love permissive licenses; they do not love all open source licenses, such as AGPL.
      - GhettoComputers 5y ago
        I never heard of it, and I never seen it being used, whats a big project that uses it?
        yjftsjthsd-h 5y ago
        https://en.wikipedia.org/wiki/Mastodon_(software) https://en.wikipedia.org/wiki/Mastodon_(software) would be one major example
cakrome 5y ago
MIT license is a big no. Folks still don't understand why copyleft is so important nowadays.
- wmf 5y ago
  And you're not educating them.
- ivegotnoaccount 5y ago
  Or maybe they do understand the licence and simply decided they did not agree with its ethics.
- strken 5y ago
  The MIT license is better than any proprietary software license, yet MIT-licensed software attracts so many more negative license-related comments than proprietary software. Seems a bit counterproductive.
  - drog 5y ago
    I think its because those who license software with MIT where GPL could be used are considered "traitors" and as some people believe "traitors are worse than enemies.
  - cakrome 5y ago
    False equivalency. I'm comparing permissive licenses with copyleft licenses. I'm not comparing MIT license with any proprietary licenses. And if uutils is proprietary, you won't see this discussion happn at all.
- staticassertion 5y ago
  > Folks still don't understand why copyleft is so important nowadays. True, it's incredible how badly the FSF has done in terms of actually marketing their licensing structure. At this point I doubt most developers care much.
podiki 5y ago
I wonder if distros (on the maintainers side) really want to move to more Rust in such core pieces? But maybe, as the readme here suggests, this is more for non-Linux platforms? I know in GNU Guix the more Rust is used has lead to struggles in packaging and building from source for non-x86-64 architectures (demands of building the Rust toolchain, bootstrapping), e.g. [0]. With something like librsvg [1] being a very low level dependency now, Rust has become rather integral to many parts, with any changes to these libraries requiring massive rebuilds for a source-based reproducible distro like GNU Guix. [0] https://lists.gnu.org/r/guix-devel/2021-11/msg00197.html https://lists.gnu.org/r/guix-devel/2021-11/msg00197.html [1] https://gitlab.gnome.org/GNOME/librsvg https://gitlab.gnome.org/GNOME/librsvg
- mustache_kimono 5y ago
  I've contributed a few small bits to uutils and in the testing I did, on those few small bits, the performance case is pretty exciting. Much easier to do concurrency with rayon, etc.
  - GhettoComputers 5y ago
    I’ve spoken to my friend about this and why it’s so much faster, he said it could be done in c as well but rust is much easier to write. Pretty impressed with rust, I just heard many people insulting it at first, throwing around “memory safe” and it seemed like rust programmers were trying to ingress into established programs and there was a great deal of hatred for it. I fell in love with the performance of rust CLI tools and mainly see its utility as a fast performing binary, memory safety is a bonus but if I had known about its performance earlier I would have ignored anti rust propaganda like quoting Linus saying “Nothing better than C”.
    - DenseComet 5y ago
      > he said it could be done in c as well but rust is much easier to write This article is a good example http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performance-of-c-and-rust/ http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performa...
Semaphor 5y ago
2014, 208 comments: https://news.ycombinator.com/item?id=7882211 https://news.ycombinator.com/item?id=7882211 2016, 490 comments: https://news.ycombinator.com/item?id=11337399 https://news.ycombinator.com/item?id=11337399
- agsamek 5y ago
  What is the progress over years?
  - steveklabnik 5y ago
    One fun recent development in 2021: https://sylvestre.ledru.info/blog/2021/03/09/debian-running-on-rust-coreutils https://sylvestre.ledru.info/blog/2021/03/09/debian-running-... > tldr: Rust/coreutils ( https://github.com/uutils/coreutils/ https://github.com/uutils/coreutils/ ) is now available in Debian, good enough to boot a Debian with GNOME, install the top 1000 packages, build Firefox, the Linux Kernel and LLVM/Clang.
    - GhettoComputers 5y ago
      Thanks, I heard a lot of people saying its not complete but I haven't had any issues with it so far, pretty huge endorsement to see its even a debian package, that's a huge sign of maturity.
- deleted 5y ago
  [deleted]
scaraffe 5y ago
If anyone else wants to do this as a learning exercise I found https://www.maizure.org/projects/decoded-gnu-coreutils/ https://www.maizure.org/projects/decoded-gnu-coreutils/ which goes into design and code of most of the utilities from the original coreutils.
phendrenad2 5y ago
Glad that this is MIT. Copyleft is great but it's important to have non-viral alternatives.
- xiaomai 5y ago
  "non-viral"? Sounds like late 90s Balmer talking. (I understand being worried about the license of libs you're going to deploy, but the license of cli utils isn't something you need to worry about "infecting" your proprietary code.)
  - GhettoComputers 5y ago
    MS contributed to Linux because their code was "infected" by GPL code from linux.
- Kinrany 5y ago
  Why is it important?
ridiculous_fish 5y ago
Gave it a whirl with `cp`: > ./target/debug/cp /dev/null /dev/zero thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }', src/uu/cp/src/cp.rs:1295:54 Good advice is "don't panic." Trying `mv`: > ./target/debug/mv . . ./target/debug/mv: cannot stat '.': No such file or directory oof Anyways the point is that there's lots of legitimate institutional knowledge baked into coreutils and a naive RiiR effort will re-introduce previously fixed bugs.
- atoav 5y ago
  Both are quite naive Errors I must say. The good thing about Rust is that it doesn't hide the complexities of the file system from you — it requires you to explicitly handle the edge cases. OR you know, you could just write a cp that panics when there is a PermissionDenied because you decided that never happens and happily unwrapped the Result. The last one is a different kind. Rusts std lib indeed cannot stat . Which means they would have to translate . into the coreutils meaning manually. Their tests should have cought that one tho.
  - ajross 5y ago
    > Rusts std lib indeed cannot stat . But the kernel can! I mean, stat() isn't a standard library function, it's a system call with (reasonably) well-defined semantics. And "." is a valid path, which returns a valid struct stat block representing a guaranteed-valid (cwd is always live; you're always "somewhere" even if the filesystem has removed the directory) directory. Coming at this from the perspective of "translate . into the coreutils meaning" is almost certainly the wrong way to think about it. It's rust that has the broken picture of the filesystem environment, coreutils is just doing unix.
    - user-the-name 5y ago
      On Linux. This isn't a Linux project, it is a cross-platform project.
deleted 5y ago
[deleted]
duped 5y ago
Obviously a few of these need work, but I've been in the unfortunate position of recovering a Linux system with a busted .so that broke almost all of coreutils, but cargo worked and so did the coreutils alternatives. Static linking is an absolute godsend.
- otabdeveloper4 5y ago
  Use busybox instead.
  - duped 5y ago
    Doesn't help when `sudo` is broken ...
    - pseudalopex 5y ago
      How do these utils help when sudo is broken?
      - duped 5y ago
        What I wound up doing was getting a working shared library and editing the LD_PRELOAD, then getting sudo back to use normal coreutils.
        GhettoComputers 5y ago
        I thought you would just end up doing everything as root lol
        deleted 5y ago
        [deleted]
    - mariusor 5y ago
      sudo and coreutils being broken are separate things though.
xixixao 5y ago
I was reminded yet again how terribly non intuitive the core file-system commands are for end user on the command line. Maybe having Rust implementations readily available would allow people to fork them and improve the interface in some new shell+utils combo package. (rm, cp require -r, mv doesn’t, no auto-directory creation for mv - if you don’t believe me look at git/hg which don’t copy the semantics + I’d like reliable dry run and by default confirm every destructive action)
- deleted 5y ago
  [deleted]
- zrm 5y ago
  > rm, cp require -r, mv doesn’t, no auto-directory creation for mv These are all like that on purpose. mv doesn't have -r because it's a safety feature for rm and cp, since deleting or copying an entire directory is expensive so you should have to specify it explicitly. Moving a directory is really just renaming it, which isn't expensive when it's on the same filesystem (and it usually is). The most common case for the destination directory not existing when moving something is a typo in the path. Then you'd end up creating a directory you didn't intend to, possibly on a filesystem you didn't intend to, and moving all the other arguments over there. It could be useful to make it possible to explicitly specify that you want this, similar to mkdir -p, but you could add that to the existing mv without breaking anything.
  - afiori 5y ago
    An inspectable and reliable dry run option and/or an interactive confirmation would solve some of these problems.
    - zrm 5y ago
      Not really. Dry run is useful for complex commands you're not sure the effects of. These options are so that when you intend to type: rm /home/john/some/file.txt But you accidentally type: rm /home/john some/file.txt It says: rm: cannot remove '/home/john': Is a directory Instead of john not having a home directory anymore. And people aren't going to use a dry run option or interactive confirmation every time they run a simple command like that.
siraben 5y ago
Nixpkgs has a PR for replacing the build environment of packages with uutils-coreutils[0]. From testing by building various packages over a dozen bugs were reported upstream, but a lot of software does build successfully with uutils[1][2], after upstream fixes, including chromium, vlc, nix, rustc, emacs, vim, uutils itself etc. It's an interesting experiment to replace such a fundamental dependency with a rewrite. A lot of the build failures are due to uutils not implementing GNU extensions, some of which can get pretty involved. [0] https://github.com/NixOS/nixpkgs/pull/116274 https://github.com/NixOS/nixpkgs/pull/116274 [1] https://github.com/NixOS/nixpkgs/pull/116274#issuecomment-850950790 https://github.com/NixOS/nixpkgs/pull/116274#issuecomment-85... [2] https://github.com/NixOS/nixpkgs/pull/116274#issuecomment-862131908 https://github.com/NixOS/nixpkgs/pull/116274#issuecomment-86...
- yjftsjthsd-h 5y ago
  I'm not quite sure I follow; is this a proposal to actually use this in nix, or just a PoC that it could work?
  - siraben 5y ago
    The latter! If merged the package set that uses uutils for stdenv would be accessible via an overlay (similar to the cross-compilation infra we have), so one could build pkgsUutils.hello to build GNU Hello, for instance.
    - deleted 5y ago
      [deleted]
Toaster-King 5y ago
The comments complaining about the choice of license reminded me that a FSF licensing intern did the same, and demanded it be changed due to some false claim about it being a derivative work[1]. Let me remind everyone that there was probably a good reason for using a permissive license, and no-one has the right to demand changes based on their definition of freedom - unless they stepped up and contributed. If this truly injures you, you can either do your own rewrite or, like Muse Group, outright buy the rights to the software and relicense. [1] https://github.com/uutils/coreutils/issues/834 https://github.com/uutils/coreutils/issues/834
- marcan_42 5y ago
  There's also this issue by FSF director Ian Kelling: https://github.com/uutils/coreutils/issues/1781 https://github.com/uutils/coreutils/issues/1781 Honestly, it sounds like they're deeply scared of long-standing FSF-backed software having to compete (or even be replaced) with alternatives that just happen not to be GPL (e.g. they have similar thoughts on LLVM/clang), to the point where they decide to troll the issue trackers. It feels really childish.
  - josefx 5y ago
    > with alternatives that just happen not to be GPL (e.g. they have similar thoughts on LLVM/clang) LLVM/clang is worse than just GPL, as far as I understand the design of GCC is intentionally horribly bad to make it painful to integrate with a proprietary tool chain. Of course this means that things like refactoring support end up being basically impossible to build on GCC, so people move to clang out of necessity.
    - Joker_vD 5y ago
      Something something those who give up software freedom for software's usefulness deserve neither something something.
    - marcan_42 5y ago
      It's both a technical issue and a political issue; RMS himself has repeatedly advocated against interfaces that would expose internal representations that GCC uses because they could be used to build proprietary add-ons. These days GCC does have a plugin interface, but they came up with a really funny hack to try to stop people from using proprietary plugins. Programs compiled with gcc can require linking with libgcc, and the libgcc license is "GPL, except it doesn't apply to software compiled with GCC and all-Free plugins". So their idea is that if you add a proprietary plug-in to GCC, that makes all code compiled like that have to be GPLed instead. It is, of course, questionable whether this hack would hold up in court. It is also rather useless, because it's not hard to reimplement enough of libgcc to make it work. For example, the Linux kernel only links with libgcc for a few architectures.
pixelbeat__ 5y ago
As a GNU coreutils maintainer, I'm happy to see another implementation. A lot of the coreutils development effort goes into testing, and new implementations should be able to leverage the existing GNU coreutils test suite, as it just uses whatever utilities are in the $PATH BTW some notes on how the GNU coreutils are tested are at: https://www.pixelbeat.org/docs/coreutils-testing.html https://www.pixelbeat.org/docs/coreutils-testing.html
- arcticbull 5y ago
  > A lot of the coreutils development effort goes into testing, and new implementations should be able to leverage the existing GNU coreutils test suite, as it just uses whatever utilities are in the $PATH This makes me really happy. Separately from this, I've been wondering for a long time if there's a way for standards (and de facto standards) to share test suites for other implementers to re-use. Sort of a npm but only for test suites. Does such a thing exist? I wrote a TOML parser recently and had to re-derive the test suite from the specs.
  - ahartmetz 5y ago
    OpenGL, for example, has an official test suite as well as one or more open source ones (I know of piglit).
    - arcticbull 5y ago
      Indeed, I'm picturing sort of a canonical repository for test cases (with a common interface). I would love to have a place I can obtain a test suite for a given standard, execute it, and even potentially publish my conformance. Potentially even have a badge to post on GitHub repos indicating conformance to a certain version of the standard.
  - marcodiego 5y ago
    Public tests are good but relying on public tests only encourages kludges to make tools pass the tests. A form of overfiting. That's why some compression tests do not publish their test corpora.
enriquto 5y ago
There is much to love about this! GNU coreutils is one of the pillars of our civilization. Re-implementing it in several new languages can only benefit everybody, as it will lead to a higher understanding of all the implementations, including the original one. So far, the tests of this Rust version seem to be used to compare against GNU coreutils, with a goal to attain feature parity. I hope in the near future, the careful memory management of Rust will motivate new tests that will be passed by the Rust version but failed bu the C one (e.g., some memory leak, or an input that triggers a segfault in the C version). Then, you will see a new curve in the graph concerning the number of tests failed by the C version, which at some point will cross the decreasing curve of the Rust version! The coverage of the rust implementation is still a bit sparse for that, but hopefully it will improve. There is one feature, though, that is unfortunately missing in this Rust re-implementation: the fact that users of coreutils will always be able to examine the source code of the programs that they run. This is guaranteed by the original copyleft license. Βut this Rust version uses a "stealable" license that allows distributors of these programs to strip users of that right. This is really sad.
gizzlon 5y ago
Should be "uutils is an *attempt* at rewriting coreutils in Rust" Not to shit on the project, just that the title made it seem like something ready to use .. (most of the tests are still failing)
- dang 5y ago
  Title changed to that from "Coreutils Rewritten in Rust". Thanks for the heads-up!
marcan_42 5y ago
It's unfortunate that FSF director Ian Kelling doesn't see it the same way, and thinks this project has little merit other than to work around the GPL license of coreutils... https://github.com/uutils/coreutils/issues/1781 https://github.com/uutils/coreutils/issues/1781 I'm curious as to what extent GNU coreutils governance is tied to the FSF these days.
- speedgoose 5y ago
  I don't read that he thinks the project has little merit other than changing the license.
  - marcan_42 5y ago
    The issue title literally says the most notable thing about the project is the license. That reads to me like a clear dismissal of any technical merits; if the most important thing about your project is its license, it's not a very interesting project.
    - speedgoose 5y ago
      I see your point. I also think the license is the most important thing, and probably even more for someone working on a GNU project at the FSF, but the project is still very interesting in many ways.
    - rob74 5y ago
      Maybe this should be read as "I consider the license a no-go, so any technical merits the project has are not relevant (for me)"? If you already dismiss the project because of its license, you probably wouldn't let technical merits redeem that.
    - ACS_Solver 5y ago
      The stance is probably not surprising, given the point of FSF. Whether you agree with them or not, the FSF has always been very clear that the most important thing about software is its license. From their point of view, a free program is always superior than a non-free one, regardless of the technical merits.
shomyo 5y ago
Rust and useless rewritting. An iconic combo. 5+ years later and those rust-folks still rewritting everything. Mental case of NIH.
deleted 5y ago
[deleted]
einpoklum 5y ago
I have filed an official issue with the uutils repository, asking the developers/maintainers to consider switching to GPL: https://news.ycombinator.com/item?id=29456115 https://news.ycombinator.com/item?id=29456115 I've tried making non-confrontational arguments in favor of the switch. Consider adding to them if you agree (or counter-arguing if you oppose, I guess) - but please keep the tone there less argumentative than here.
DeathArrow 5y ago
>Many GNU, Linux and other utilities are useful, and obviously some effort has been spent in the past to port them to Windows. However, those projects are written in platform-specific C, a language considered unsafe compared to Rust, and have other issues. >Rust provides a good, platform-agnostic way of writing systems utilities that are easy to compile anywhere, and this is as good a way as any to try and learn it. If you want some software to be platform agnostic, you can write it in C in a way that is not platform specific. So rewriting some software in Rust to be platform agnostic doesn't seem a good reason for a rewrite. Also, even if the C software to be rewritten isn't platform-agnostic, at least some part of it is, so by using C you don't have to rewrite everything. I'm sure that Rust can have a lot of valid use cases but the trend of "Hey, let's rewrite X in Rust just because ..." I don't agree with.
OJFord 5y ago
If you fancy living on the edge, there's already an AUR package 'coreutils-hybrid' that uses stable uutils (with unprefixed names) where possible, falling back on GNU coreutils: https://aur.archlinux.org/packages/coreutils-hybrid https://aur.archlinux.org/packages/coreutils-hybrid
_wldu 5y ago
I've done this in go. It's fun to do, but the real reason I did it was memory safety issues in C. We need to implement as much code as we can in memory safe languages.
nomemory 5y ago
> Many GNU, Linux and other utilities are useful, and obviously some effort has been spent in the past to port them to Windows. However, those projects are written in platform-specific C, a language considered unsafe compared to Rust, and have other issues. I wonder why some people in the Rust community seem to remind us every few sentences how Rust is superior to C because it's safer. /s And now it's C, yesterday it was C++... what's next, ASM ?
- c3534l 5y ago
  Because they're justifying a project whose sole difference to the other implementation is the language its written in.
  - nomemory 5y ago
    So it's more like a "Look, it's possible to do this in Rust!" than "We did it in Rust, and we believe it's better because ..." ?
    - LeonidasXIV 5y ago
      It literally says why it is better and you even quoted it: > However, those projects are written in platform-specific C, a language considered unsafe compared to Rust Or to paraphrase it using your wording: "We did it in Rust and we believe it is better because it does not have memory unsafety issues that are the number #1 reason for security issues every year"
      - nomemory 5y ago
        So without playing dumb or anything, you say they say: "We have security issues every year because we program in unsafe languages like C, and to solve this, we should start rewriting things in Rust because it's 'memory safe'." Amazing!
betimsl 5y ago
I believe that coreutils should be rewritten in Go. Much better option.
- jgilias 5y ago
  Here you 'Go': https://github.com/aisola/go-coreutils https://github.com/aisola/go-coreutils
  - GhettoComputers 5y ago
    Last commit was 2015, is it functional?
- andreynering 5y ago
  I've seem a few attempts. This is just one example: https://github.com/u-root/u-root/tree/master/cmds/core https://github.com/u-root/u-root/tree/master/cmds/core EDIT: You may also be interested in https://github.com/mvdan/sh https://github.com/mvdan/sh
- loudmax 5y ago
  Why would Go be a better option than Rust here? Go and Rust are optimized for different types of environments. There are a lot of situations when selecting Go over Rust would be a more pragmatic choice. I don't see how coreutils is one of these.
  - tyingq 5y ago
    It seems like a reasonable opinion to me. Looking at the list of what's in coreutils, I don't see anything where performance would be a huge issue other than perhaps dd, so any performance edge Rust might provide seems moot. And I would guess that Golang would be less code and faster to write/implement. I would feel differently if it were something else, like gawk, for example. Rust would be a better fit there.
npigrounet 5y ago
This is so much better than the previous, buggy, unsafe and dangerous C implementation.
xyzzy_plugh 5y ago
This is very cool. The problem with POSIX is that, while it's possible to implement the bare minimum, it's hard to not have a few extensions. There are some truly braindead ideas that GNU coreutils have absolutely improved upon. Sadly, a new coreutils collection simply brings further incompatibility. There's already significant switching logic out there to handle BSD vs GNU coreutils (have you ever used sed -i?), adding another flavor makes this sort of thing dead on arrival, at least to me. I'm not retrofitting my scripts to support these implementations too. Aside, Nix is the only real solution to this problem, as it can replace these tools wholesale, hermetically. Being able to depend on a specific implementation in a portable manner is rather compelling.

I'm writing a book "Rust From the Ground Up" which teaches Rust by rewriting a coreutil per chapter, from the original BSD sources. The first 3 chapters (true/false, yes, and head) are released. wc is next, then cat, cut, rev, and uniq. https://rftgu.rs/ https://rftgu.rs/

nickel_8448 5y ago
The book looks very interesting. By when do you plan to complete the next set of chapters?
- mprovost 5y ago
  Thanks! I've written the whole book - now I'm going through and editing and typesetting each chapter and trying to publish one a month.
  - nickel_8448 5y ago
    That's wonderful. Do you have twitter? I just want to be notified once the book is complete.
    - mprovost 5y ago
      Yes I’m posting updates here: https://mobile.twitter.com/rustftgu https://mobile.twitter.com/rustftgu Thanks for the interest!

mprovost 5y ago

More interesting than rewriting the GNU coreutils are the projects that are reinventing them. Like ripgrep (which built on the example of ack) is a better grep, by discarding backwards compatibility. Or fd, which abandons find's arcane syntax. We've learned a thing or two since the 70s when most of these tools were first invented.

hencoappel 5y ago
Agreed, it would be an interesting idea to make a new coreutils with everything modernised/cleaned up to be easier to use and more consistent.
- DarylZero 5y ago
  https://www.nushell.sh/ https://www.nushell.sh/
  - GhettoComputers 5y ago
    Tried it too bad no shell compares to fish for ease of use. Every new shell gets asked "can it do the thing in fish" and the answer is always kind of or no, zsh did not function as well even with the configs and it seemed much slower.

MichaelBurge 5y ago

If we replace GNU coreutils[1] with this, could we finally call it "Linux" instead of "GNU/Linux"? [1] And gcc with clang when bootstrapping

yjftsjthsd-h 5y ago
Yes:) In fact, if you're into that, I'd argue that Alpine is already not GNU/Linux, since it uses busybox and musl.
bayindirh 5y ago
Is getting rid of GNU that important?
- woodruffw 5y ago
  I’m not sure if this answers the “how important” question, but: a large part of the GNU ecosystem is functionally unmaintained. The various tools under binutils receive so few patches that they’ve been the standard punching bag for academic fuzzing research for the last 20 years, and the coreutils aren’t much better.
- haimez 5y ago
  About as important as keeping GNU, empirically.
- bogwog 5y ago
  I think the point of this project is less about getting rid of GNU, and more about promoting Rust. There's nothing wrong with GNU coreutils.
  - adgjlsfhk1 5y ago
    https://www.cvedetails.com/vulnerability-list.php?vendor_id=72&product_id=5075 https://www.cvedetails.com/vulnerability-list.php?vendor_id=.... 4 vulns in the past decade that rust wouldn't have had.

pxc 5y ago

This will probably sound completely insane, not least of all to people who prefer non-GNU coreutils, but I kinda hope that as uutils matures, some shells will offer an option to compile in the uutils rewrites as builtins. Especially for newer shells and shells that want to target Windows in a first-class way, that could be a real portability win.

deleted 5y ago
[deleted]
smt88 5y ago
Why is WSL[1] insufficient for Windows portability? 1. https://docs.microsoft.com/en-us/windows/wsl/about https://docs.microsoft.com/en-us/windows/wsl/about
- dagw 5y ago
  WSL is great if you want/need a full Linux environment. It is quite heavy and probably overkill if all you want is to use some coreutil commands from the Windows terminal.
  - tyingq 5y ago
    Gitbash is a relatively nice solution for this.
  - mkl 5y ago
    WSL1 is very light in my experience.
- pxc 5y ago
  I wasn't primarily concerned with Windows portability, but with the portability of scripts altogether, and no longer having to concern oneself with variations in coreutils versions for at least some scripts, even if they run on multiple platforms. But to answer your question about Windows: Because requiring an emulation layer is a huge step up in complexity from being able to distribute your shell as a single static binary which is just a few MB. Right now Elvish and Nushell, for example, can be deployed on Windows as binaries that don't require any external dependencies, any installation procedure, or any configuration. If they could link against uutils statically, they could also come with basic utilities without having an environment to manage. WSL (along with MSYS2 and Cygwin) is deeply stateful. Now to make sure everything is working you have to check the compatibility of your whole system, and the versions of coreutils can still vary in addition to the version of the shell interpreter. WSL2 also requires virtualization, which can complicate using it under virtual machines. You may not want everything you run under your shell to depend on WSL, which is also happens with WSL2. Even with WSL1, a script that requires a chroot environment to deploy and run it is way less portable than one that doesn't.

signa11 5y ago

why MIT license ? that is bound to be turnoff for a lot of folks. also, a minor nit: there are still a lot of work [1] that needs to be done, and imho, it is a bit premature to title this article as is presented here. [1] https://github.com/uutils/coreutils#utilities https://github.com/uutils/coreutils#utilities

xvedejas 5y ago
Is it? I thought it was as standard as and on pretty much equal footing with BSD and Apache licenses. Is it substantially different?
no_wizard 5y ago
Who, exactly? I think most don’t care, and big corps love the MIT license
- GhettoComputers 5y ago
  Don’t they love them all? Free software in general is nice, PS3 ran Linux, was based on BSD, and I’m sure they used MIT software somewhere there too.
  - yjftsjthsd-h 5y ago
    Big corps love permissive licenses; they do not love all open source licenses, such as AGPL.
    - GhettoComputers 5y ago
      I never heard of it, and I never seen it being used, whats a big project that uses it?
      - yjftsjthsd-h 5y ago
        https://en.wikipedia.org/wiki/Mastodon_(software) https://en.wikipedia.org/wiki/Mastodon_(software) would be one major example

cakrome 5y ago

MIT license is a big no. Folks still don't understand why copyleft is so important nowadays.

wmf 5y ago
And you're not educating them.
ivegotnoaccount 5y ago
Or maybe they do understand the licence and simply decided they did not agree with its ethics.
strken 5y ago
The MIT license is better than any proprietary software license, yet MIT-licensed software attracts so many more negative license-related comments than proprietary software. Seems a bit counterproductive.
- drog 5y ago
  I think its because those who license software with MIT where GPL could be used are considered "traitors" and as some people believe "traitors are worse than enemies.
- cakrome 5y ago
  False equivalency. I'm comparing permissive licenses with copyleft licenses. I'm not comparing MIT license with any proprietary licenses. And if uutils is proprietary, you won't see this discussion happn at all.
staticassertion 5y ago
> Folks still don't understand why copyleft is so important nowadays. True, it's incredible how badly the FSF has done in terms of actually marketing their licensing structure. At this point I doubt most developers care much.

podiki 5y ago

I wonder if distros (on the maintainers side) really want to move to more Rust in such core pieces? But maybe, as the readme here suggests, this is more for non-Linux platforms? I know in GNU Guix the more Rust is used has lead to struggles in packaging and building from source for non-x86-64 architectures (demands of building the Rust toolchain, bootstrapping), e.g. [0]. With something like librsvg [1] being a very low level dependency now, Rust has become rather integral to many parts, with any changes to these libraries requiring massive rebuilds for a source-based reproducible distro like GNU Guix. [0] https://lists.gnu.org/r/guix-devel/2021-11/msg00197.html https://lists.gnu.org/r/guix-devel/2021-11/msg00197.html [1] https://gitlab.gnome.org/GNOME/librsvg https://gitlab.gnome.org/GNOME/librsvg

mustache_kimono 5y ago
I've contributed a few small bits to uutils and in the testing I did, on those few small bits, the performance case is pretty exciting. Much easier to do concurrency with rayon, etc.
- GhettoComputers 5y ago
  I’ve spoken to my friend about this and why it’s so much faster, he said it could be done in c as well but rust is much easier to write. Pretty impressed with rust, I just heard many people insulting it at first, throwing around “memory safe” and it seemed like rust programmers were trying to ingress into established programs and there was a great deal of hatred for it. I fell in love with the performance of rust CLI tools and mainly see its utility as a fast performing binary, memory safety is a bonus but if I had known about its performance earlier I would have ignored anti rust propaganda like quoting Linus saying “Nothing better than C”.
  - DenseComet 5y ago
    > he said it could be done in c as well but rust is much easier to write This article is a good example http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performance-of-c-and-rust/ http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performa...

Semaphor 5y ago

2014, 208 comments: https://news.ycombinator.com/item?id=7882211 https://news.ycombinator.com/item?id=7882211 2016, 490 comments: https://news.ycombinator.com/item?id=11337399 https://news.ycombinator.com/item?id=11337399

agsamek 5y ago
What is the progress over years?
- steveklabnik 5y ago
  One fun recent development in 2021: https://sylvestre.ledru.info/blog/2021/03/09/debian-running-on-rust-coreutils https://sylvestre.ledru.info/blog/2021/03/09/debian-running-... > tldr: Rust/coreutils ( https://github.com/uutils/coreutils/ https://github.com/uutils/coreutils/ ) is now available in Debian, good enough to boot a Debian with GNOME, install the top 1000 packages, build Firefox, the Linux Kernel and LLVM/Clang.
  - GhettoComputers 5y ago
    Thanks, I heard a lot of people saying its not complete but I haven't had any issues with it so far, pretty huge endorsement to see its even a debian package, that's a huge sign of maturity.
deleted 5y ago
[deleted]

scaraffe 5y ago

If anyone else wants to do this as a learning exercise I found https://www.maizure.org/projects/decoded-gnu-coreutils/ https://www.maizure.org/projects/decoded-gnu-coreutils/ which goes into design and code of most of the utilities from the original coreutils.

phendrenad2 5y ago

Glad that this is MIT. Copyleft is great but it's important to have non-viral alternatives.

xiaomai 5y ago
"non-viral"? Sounds like late 90s Balmer talking. (I understand being worried about the license of libs you're going to deploy, but the license of cli utils isn't something you need to worry about "infecting" your proprietary code.)
- GhettoComputers 5y ago
  MS contributed to Linux because their code was "infected" by GPL code from linux.
Kinrany 5y ago
Why is it important?

ridiculous_fish 5y ago

Gave it a whirl with `cp`: > ./target/debug/cp /dev/null /dev/zero thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }', src/uu/cp/src/cp.rs:1295:54 Good advice is "don't panic." Trying `mv`: > ./target/debug/mv . . ./target/debug/mv: cannot stat '.': No such file or directory oof Anyways the point is that there's lots of legitimate institutional knowledge baked into coreutils and a naive RiiR effort will re-introduce previously fixed bugs.

atoav 5y ago
Both are quite naive Errors I must say. The good thing about Rust is that it doesn't hide the complexities of the file system from you — it requires you to explicitly handle the edge cases. OR you know, you could just write a cp that panics when there is a PermissionDenied because you decided that never happens and happily unwrapped the Result. The last one is a different kind. Rusts std lib indeed cannot stat . Which means they would have to translate . into the coreutils meaning manually. Their tests should have cought that one tho.
- ajross 5y ago
  > Rusts std lib indeed cannot stat . But the kernel can! I mean, stat() isn't a standard library function, it's a system call with (reasonably) well-defined semantics. And "." is a valid path, which returns a valid struct stat block representing a guaranteed-valid (cwd is always live; you're always "somewhere" even if the filesystem has removed the directory) directory. Coming at this from the perspective of "translate . into the coreutils meaning" is almost certainly the wrong way to think about it. It's rust that has the broken picture of the filesystem environment, coreutils is just doing unix.
  - user-the-name 5y ago
    On Linux. This isn't a Linux project, it is a cross-platform project.

deleted 5y ago

[deleted]

duped 5y ago

Obviously a few of these need work, but I've been in the unfortunate position of recovering a Linux system with a busted .so that broke almost all of coreutils, but cargo worked and so did the coreutils alternatives. Static linking is an absolute godsend.

otabdeveloper4 5y ago
Use busybox instead.
- duped 5y ago
  Doesn't help when `sudo` is broken ...
  - pseudalopex 5y ago
    How do these utils help when sudo is broken?
    - duped 5y ago
      What I wound up doing was getting a working shared library and editing the LD_PRELOAD, then getting sudo back to use normal coreutils.
      - GhettoComputers 5y ago
        I thought you would just end up doing everything as root lol
        deleted 5y ago
        [deleted]
  - mariusor 5y ago
    sudo and coreutils being broken are separate things though.

xixixao 5y ago

I was reminded yet again how terribly non intuitive the core file-system commands are for end user on the command line. Maybe having Rust implementations readily available would allow people to fork them and improve the interface in some new shell+utils combo package. (rm, cp require -r, mv doesn’t, no auto-directory creation for mv - if you don’t believe me look at git/hg which don’t copy the semantics + I’d like reliable dry run and by default confirm every destructive action)

deleted 5y ago
[deleted]
zrm 5y ago
> rm, cp require -r, mv doesn’t, no auto-directory creation for mv These are all like that on purpose. mv doesn't have -r because it's a safety feature for rm and cp, since deleting or copying an entire directory is expensive so you should have to specify it explicitly. Moving a directory is really just renaming it, which isn't expensive when it's on the same filesystem (and it usually is). The most common case for the destination directory not existing when moving something is a typo in the path. Then you'd end up creating a directory you didn't intend to, possibly on a filesystem you didn't intend to, and moving all the other arguments over there. It could be useful to make it possible to explicitly specify that you want this, similar to mkdir -p, but you could add that to the existing mv without breaking anything.
- afiori 5y ago
  An inspectable and reliable dry run option and/or an interactive confirmation would solve some of these problems.
  - zrm 5y ago
    Not really. Dry run is useful for complex commands you're not sure the effects of. These options are so that when you intend to type: rm /home/john/some/file.txt But you accidentally type: rm /home/john some/file.txt It says: rm: cannot remove '/home/john': Is a directory Instead of john not having a home directory anymore. And people aren't going to use a dry run option or interactive confirmation every time they run a simple command like that.

siraben 5y ago

Nixpkgs has a PR for replacing the build environment of packages with uutils-coreutils[0]. From testing by building various packages over a dozen bugs were reported upstream, but a lot of software does build successfully with uutils[1][2], after upstream fixes, including chromium, vlc, nix, rustc, emacs, vim, uutils itself etc. It's an interesting experiment to replace such a fundamental dependency with a rewrite. A lot of the build failures are due to uutils not implementing GNU extensions, some of which can get pretty involved. [0] https://github.com/NixOS/nixpkgs/pull/116274 https://github.com/NixOS/nixpkgs/pull/116274 [1] https://github.com/NixOS/nixpkgs/pull/116274#issuecomment-850950790 https://github.com/NixOS/nixpkgs/pull/116274#issuecomment-85... [2] https://github.com/NixOS/nixpkgs/pull/116274#issuecomment-862131908 https://github.com/NixOS/nixpkgs/pull/116274#issuecomment-86...

yjftsjthsd-h 5y ago
I'm not quite sure I follow; is this a proposal to actually use this in nix, or just a PoC that it could work?
- siraben 5y ago
  The latter! If merged the package set that uses uutils for stdenv would be accessible via an overlay (similar to the cross-compilation infra we have), so one could build pkgsUutils.hello to build GNU Hello, for instance.
  - deleted 5y ago
    [deleted]

Toaster-King 5y ago

The comments complaining about the choice of license reminded me that a FSF licensing intern did the same, and demanded it be changed due to some false claim about it being a derivative work[1]. Let me remind everyone that there was probably a good reason for using a permissive license, and no-one has the right to demand changes based on their definition of freedom - unless they stepped up and contributed. If this truly injures you, you can either do your own rewrite or, like Muse Group, outright buy the rights to the software and relicense. [1] https://github.com/uutils/coreutils/issues/834 https://github.com/uutils/coreutils/issues/834

marcan_42 5y ago
There's also this issue by FSF director Ian Kelling: https://github.com/uutils/coreutils/issues/1781 https://github.com/uutils/coreutils/issues/1781 Honestly, it sounds like they're deeply scared of long-standing FSF-backed software having to compete (or even be replaced) with alternatives that just happen not to be GPL (e.g. they have similar thoughts on LLVM/clang), to the point where they decide to troll the issue trackers. It feels really childish.
- josefx 5y ago
  > with alternatives that just happen not to be GPL (e.g. they have similar thoughts on LLVM/clang) LLVM/clang is worse than just GPL, as far as I understand the design of GCC is intentionally horribly bad to make it painful to integrate with a proprietary tool chain. Of course this means that things like refactoring support end up being basically impossible to build on GCC, so people move to clang out of necessity.
  - Joker_vD 5y ago
    Something something those who give up software freedom for software's usefulness deserve neither something something.
  - marcan_42 5y ago
    It's both a technical issue and a political issue; RMS himself has repeatedly advocated against interfaces that would expose internal representations that GCC uses because they could be used to build proprietary add-ons. These days GCC does have a plugin interface, but they came up with a really funny hack to try to stop people from using proprietary plugins. Programs compiled with gcc can require linking with libgcc, and the libgcc license is "GPL, except it doesn't apply to software compiled with GCC and all-Free plugins". So their idea is that if you add a proprietary plug-in to GCC, that makes all code compiled like that have to be GPLed instead. It is, of course, questionable whether this hack would hold up in court. It is also rather useless, because it's not hard to reimplement enough of libgcc to make it work. For example, the Linux kernel only links with libgcc for a few architectures.

pixelbeat__ 5y ago

As a GNU coreutils maintainer, I'm happy to see another implementation. A lot of the coreutils development effort goes into testing, and new implementations should be able to leverage the existing GNU coreutils test suite, as it just uses whatever utilities are in the $PATH BTW some notes on how the GNU coreutils are tested are at: https://www.pixelbeat.org/docs/coreutils-testing.html https://www.pixelbeat.org/docs/coreutils-testing.html

arcticbull 5y ago
> A lot of the coreutils development effort goes into testing, and new implementations should be able to leverage the existing GNU coreutils test suite, as it just uses whatever utilities are in the $PATH This makes me really happy. Separately from this, I've been wondering for a long time if there's a way for standards (and de facto standards) to share test suites for other implementers to re-use. Sort of a npm but only for test suites. Does such a thing exist? I wrote a TOML parser recently and had to re-derive the test suite from the specs.
- ahartmetz 5y ago
  OpenGL, for example, has an official test suite as well as one or more open source ones (I know of piglit).
  - arcticbull 5y ago
    Indeed, I'm picturing sort of a canonical repository for test cases (with a common interface). I would love to have a place I can obtain a test suite for a given standard, execute it, and even potentially publish my conformance. Potentially even have a badge to post on GitHub repos indicating conformance to a certain version of the standard.
- marcodiego 5y ago
  Public tests are good but relying on public tests only encourages kludges to make tools pass the tests. A form of overfiting. That's why some compression tests do not publish their test corpora.

enriquto 5y ago

There is much to love about this! GNU coreutils is one of the pillars of our civilization. Re-implementing it in several new languages can only benefit everybody, as it will lead to a higher understanding of all the implementations, including the original one. So far, the tests of this Rust version seem to be used to compare against GNU coreutils, with a goal to attain feature parity. I hope in the near future, the careful memory management of Rust will motivate new tests that will be passed by the Rust version but failed bu the C one (e.g., some memory leak, or an input that triggers a segfault in the C version). Then, you will see a new curve in the graph concerning the number of tests failed by the C version, which at some point will cross the decreasing curve of the Rust version! The coverage of the rust implementation is still a bit sparse for that, but hopefully it will improve. There is one feature, though, that is unfortunately missing in this Rust re-implementation: the fact that users of coreutils will always be able to examine the source code of the programs that they run. This is guaranteed by the original copyleft license. Βut this Rust version uses a "stealable" license that allows distributors of these programs to strip users of that right. This is really sad.

gizzlon 5y ago

Should be "uutils is an *attempt* at rewriting coreutils in Rust" Not to shit on the project, just that the title made it seem like something ready to use .. (most of the tests are still failing)

dang 5y ago
Title changed to that from "Coreutils Rewritten in Rust". Thanks for the heads-up!

marcan_42 5y ago

It's unfortunate that FSF director Ian Kelling doesn't see it the same way, and thinks this project has little merit other than to work around the GPL license of coreutils... https://github.com/uutils/coreutils/issues/1781 https://github.com/uutils/coreutils/issues/1781 I'm curious as to what extent GNU coreutils governance is tied to the FSF these days.

speedgoose 5y ago
I don't read that he thinks the project has little merit other than changing the license.
- marcan_42 5y ago
  The issue title literally says the most notable thing about the project is the license. That reads to me like a clear dismissal of any technical merits; if the most important thing about your project is its license, it's not a very interesting project.
  - speedgoose 5y ago
    I see your point. I also think the license is the most important thing, and probably even more for someone working on a GNU project at the FSF, but the project is still very interesting in many ways.
  - rob74 5y ago
    Maybe this should be read as "I consider the license a no-go, so any technical merits the project has are not relevant (for me)"? If you already dismiss the project because of its license, you probably wouldn't let technical merits redeem that.
  - ACS_Solver 5y ago
    The stance is probably not surprising, given the point of FSF. Whether you agree with them or not, the FSF has always been very clear that the most important thing about software is its license. From their point of view, a free program is always superior than a non-free one, regardless of the technical merits.

shomyo 5y ago

Rust and useless rewritting. An iconic combo. 5+ years later and those rust-folks still rewritting everything. Mental case of NIH.

deleted 5y ago

[deleted]

einpoklum 5y ago

I have filed an official issue with the uutils repository, asking the developers/maintainers to consider switching to GPL: https://news.ycombinator.com/item?id=29456115 https://news.ycombinator.com/item?id=29456115 I've tried making non-confrontational arguments in favor of the switch. Consider adding to them if you agree (or counter-arguing if you oppose, I guess) - but please keep the tone there less argumentative than here.

DeathArrow 5y ago

>Many GNU, Linux and other utilities are useful, and obviously some effort has been spent in the past to port them to Windows. However, those projects are written in platform-specific C, a language considered unsafe compared to Rust, and have other issues. >Rust provides a good, platform-agnostic way of writing systems utilities that are easy to compile anywhere, and this is as good a way as any to try and learn it. If you want some software to be platform agnostic, you can write it in C in a way that is not platform specific. So rewriting some software in Rust to be platform agnostic doesn't seem a good reason for a rewrite. Also, even if the C software to be rewritten isn't platform-agnostic, at least some part of it is, so by using C you don't have to rewrite everything. I'm sure that Rust can have a lot of valid use cases but the trend of "Hey, let's rewrite X in Rust just because ..." I don't agree with.

OJFord 5y ago

If you fancy living on the edge, there's already an AUR package 'coreutils-hybrid' that uses stable uutils (with unprefixed names) where possible, falling back on GNU coreutils: https://aur.archlinux.org/packages/coreutils-hybrid https://aur.archlinux.org/packages/coreutils-hybrid

_wldu 5y ago

I've done this in go. It's fun to do, but the real reason I did it was memory safety issues in C. We need to implement as much code as we can in memory safe languages.

nomemory 5y ago

c3534l 5y ago
Because they're justifying a project whose sole difference to the other implementation is the language its written in.
- nomemory 5y ago
  So it's more like a "Look, it's possible to do this in Rust!" than "We did it in Rust, and we believe it's better because ..." ?
  - LeonidasXIV 5y ago
    It literally says why it is better and you even quoted it: > However, those projects are written in platform-specific C, a language considered unsafe compared to Rust Or to paraphrase it using your wording: "We did it in Rust and we believe it is better because it does not have memory unsafety issues that are the number #1 reason for security issues every year"
    - nomemory 5y ago
      So without playing dumb or anything, you say they say: "We have security issues every year because we program in unsafe languages like C, and to solve this, we should start rewriting things in Rust because it's 'memory safe'." Amazing!

betimsl 5y ago

I believe that coreutils should be rewritten in Go. Much better option.

jgilias 5y ago
Here you 'Go': https://github.com/aisola/go-coreutils https://github.com/aisola/go-coreutils
- GhettoComputers 5y ago
  Last commit was 2015, is it functional?
andreynering 5y ago
I've seem a few attempts. This is just one example: https://github.com/u-root/u-root/tree/master/cmds/core https://github.com/u-root/u-root/tree/master/cmds/core EDIT: You may also be interested in https://github.com/mvdan/sh https://github.com/mvdan/sh
loudmax 5y ago
Why would Go be a better option than Rust here? Go and Rust are optimized for different types of environments. There are a lot of situations when selecting Go over Rust would be a more pragmatic choice. I don't see how coreutils is one of these.
- tyingq 5y ago
  It seems like a reasonable opinion to me. Looking at the list of what's in coreutils, I don't see anything where performance would be a huge issue other than perhaps dd, so any performance edge Rust might provide seems moot. And I would guess that Golang would be less code and faster to write/implement. I would feel differently if it were something else, like gawk, for example. Rust would be a better fit there.

npigrounet 5y ago

This is so much better than the previous, buggy, unsafe and dangerous C implementation.

xyzzy_plugh 5y ago

This is very cool. The problem with POSIX is that, while it's possible to implement the bare minimum, it's hard to not have a few extensions. There are some truly braindead ideas that GNU coreutils have absolutely improved upon. Sadly, a new coreutils collection simply brings further incompatibility. There's already significant switching logic out there to handle BSD vs GNU coreutils (have you ever used sed -i?), adding another flavor makes this sort of thing dead on arrival, at least to me. I'm not retrofitting my scripts to support these implementations too. Aside, Nix is the only real solution to this problem, as it can replace these tools wholesale, hermetically. Being able to depend on a specific implementation in a portable manner is rather compelling.