9 ms·

Rewriting essential Linux packages in Rust

98 points by worble 2y ago

saurik 2y ago
A big reason the GNU utilities were game changing is not because of their existence, or their functionality, but because of their license... a license which, in no small part, is what not merely motivated but then allowed for their continued existence and functionality: a tit-for-tat, sharing is caring, we're all in this together, fighting for the users approach to software development, one which ensures that no one is going to embrace and extend your software for use in their platform to lock people out of participation (whether directly or indirectly) in control over the hardware they own. It just really really sucks that people are thereby allocating a ton of effort into reimplementing these tools--putting good effort behind a project that even has a good reason to exist (memory safety), even if (as I'll poke at later in this comment) that apparently is explicitly not the reason they are working on this (which shocked me)--with the goal of being "bug for bug compatible" with the upstream copies from the GNU Foundation while carefully ignoring the #1 most important integration (as this affects how the software fits into the whole) test: "is this software 'free' as in freedom?". Of course, they claim that this is some kind of unproductive waste of time "debate", as if the license is the least important part of the software and doesn't matter, and I think some people want to take this narrative. Regardless, whether or not we agree with this--a position that feels a lot like "politics don't matter and are a waste of time, so stop voicing your concerns"--that's not what's going on here: if you look a bit deeper, this project actually cares deeply about its license, and is going out of its way to choose the license it is using, ignore complaints, and avoid ending up GPL. https://www.youtube.com/watch?v=5qTyyMyU2hQ https://www.youtube.com/watch?v=5qTyyMyU2hQ In an interview with FOSS Weekly, Sylvestre Ledru (the main developer, who curiously has a background working on Debian and Firefox, before ending up getting seduced by the clang/LLVM ecosystem), firmly states "it is not about security", focusing only on an interest in learning himself how the full stack of tools function and preparing for a future where new developers don't actually know enough C to contribute; this might seem to fit into the earlier narrative that the license doesn't really matter, which he later restates himself "I don't care that much, as long as it is OSI compliant". This topic comes up multiple times later in the interview, and Steven sticks to his framing that he doesn't care about the license, that this debate is a waste of time, and that he tries to avoid discussing it as it is "more philosophical than technical". Of course, this isn't preventing him from discussing it ;P... this is clearly a big issue that people have with this project, it is one that comes up in most discussions of the project, and--if it really didn't matter, and it really weren't a big deal--you would thereby expect that he'd just change it, to avoid having to discuss it again... ...only, in this interview--in no small part from the interviewer slowly leaking part of their pre-interview discussion to cause the topic to keep coming back up--we learn just how much this developer does seem to care about the license, as, to keep it all as MIT, he's having to avoid looking at the original implementation, in an attempt to avoid accidentally letting his code get infected by GPL, to support some users of the project who actively choose to use this reimplementation to avoid GPL compliance (the example we are given--by the interviewer outing it, not him--is "car manufacturers"). As someone who works in security but finds it demoralizing how often security is used as an excuse for what ends up being an effort to lock users out of a platform due to what is merely some supposedly-accidental property of the effort--including one time I was in a hearing with the US Copyright Office, sitting next to a rep from General Motors who was there to argue that we shouldn't be allowed to jailbreak a "portable all-purpose mobile computing device" because that might include a car (lol)--I found this back/forth in the comments forum on the website for this interview worth reading: https://hackaday.com/2024/07/17/floss-weekly-episode-792-rust-coreutils/ https://hackaday.com/2024/07/17/floss-weekly-episode-792-rus... <AgainAgain> the goal is to “rewrite it in x” is to move everything to permissive liscenses. then lock future changes away. just like every thing else “security” is used as pretext. <Jonathan Bennett> We chatted a bit about exactly that. They make no claim that this effort is for security, and freely admitted that some of their users are doing so precisely because it’s MIT and not GPL. So… Yes, but actually no. <Thovte> That sounds like yes, but actually, yes. No?
- panstromek 2y ago
  I feel like this shouldn't be surprising. If you use GPL, you give others strong incentive to rewrite an alternative, because GPL is just very heavy burden. I'd even go as far as to say this is an example of failure of GPL. If the intent was to set up incentives to make sure people share the improvements with the community for mutual benefit, but the result is an incentive to rather burn resources to do complete rewrites, then that seems like a failure on all sides to me.
  - codeguro 2y ago
    It's not a heavy burden at all. Releasing source code is quick and easy. Github will host it for you for free. The issue isn't the burden. The issue is that they don't want to comply with the license. Let's cut the crap. This is an attack on the four essential freedoms by means of replacement of the foundational libraries. If you want to talk about burning resources to do complete rewrites, let's start with uutils.
1238127 2y ago
[flagged]
- Curvature5868 2y ago
  Not a rust zealot, but https://github.com/firecracker-microvm/firecracker/tree/main/src https://github.com/firecracker-microvm/firecracker/tree/main... comes to mind.
- dylan604 2y ago
  If the original thing is improved by being written in Rust as everyone proclaims, then this would be a good thing. However, I have doubts that the years upon years of updates fixing odd behavior, and then forgotten about the whys&hows of those updates that the same issues will not be introduced in the Rust version.
  - kjrfghslkdjfl 2y ago
    [dead]
  - janice1999 2y ago
    > and then forgotten about the whys&hows of those updates that the same issues will not be introduced in the Rust version. Ideally that's what test suites are for, although I'm sure some deviations/gaps will be caught by users. For uutils they are preserving all the edge cases, replicating the original behaviour. There are benefits to rewrites (far less often in my experience), like doas replacing sudo in BSD distros and culling previous unused and insecure behaviour.
    - estebank 2y ago
      And in the face of gaps of the test suite there's no assurance that consistent output will be preserved across releases of the same application. This is somewhat mitigated by the slow release schedule that these kind of projects usually have, couples with a feeling of "being done" meaning that releases shouldn't have much green field feature development work. But still fixing one bug could cause a regression elsewhere.
shmerl 2y ago
Ripgrep should be included in all distros by default.
- janice1999 2y ago
  fd is also great and I install it everywhere. https://github.com/sharkdp/fd https://github.com/sharkdp/fd
  - evanwpm 2y ago
    I also really like: https://github.com/eza-community/eza https://github.com/eza-community/eza (modern ls replacement) https://github.com/BurntSushi/erd https://github.com/BurntSushi/erd (modern tree replacement) https://github.com/sharkdp/bat https://github.com/sharkdp/bat (modern cat(1) replacement) my .zshrc for every system now uses these as drop in replacements
    - mubou 2y ago
      Wrong erd! https://github.com/solidiquis/erdtree https://github.com/solidiquis/erdtree This looks really cool, though. I might try using this as an `ls` replacement, too, just because of how it shows the recursive directory size like `du`. I've always wished windows explorer did that (do any linux gui file managers?)
- konart 2y ago
  And https://github.com/ast-grep/ast-grep https://github.com/ast-grep/ast-grep maybe
- grandempire 2y ago
  It’s a great tool, but it’s not a posix compliant grep.
  - shmerl 2y ago
    I think it's more about having a stable interface. If it's not stable - you can't use it as a base tool long term. But if it's stable, what difference does it make if it's POSIX compliant or not?
malkia 2y ago
Is Rust (llvm?) supported on all platforms Linux targets?
- preisschild 2y ago
  Not yet, afaik the rust4linux devs thus want to create a rust frontend for GCC (gccrs)
  - estebank 2y ago
    gccrs (and gcc-codegen which is a project with the same objective but only implementing the backend) is an independent project to Rust4Linux (but they talk to each other).
    - preisschild 2y ago
      Ah ok i thought they were the main contributers and users.
- masklinn 2y ago
  No. What relevance does that have tho?
  - rerdavies 2y ago
    The relevance would be that Rust coreutils cannot be merged into Linux mainline. Obviously.
    - masklinn 2y ago
      Coreutils are not part of the linux project in the first place, and uutils does not aim to be merged into GNU coreutils.
      - mustache_kimono 2y ago
        >> The relevance would be that Rust coreutils cannot be merged into Linux mainline. Obviously. > Coreutils are not part of the linux project in the first place, and uutils does not aim to be merged into GNU coreutils. I think he having fun with you bro.
xixixao 2y ago
I've been using the rewritten coreutils as a reference in implementing human-utils[0]. The amount of complexity, even with pretty high-level Rust std, is still super high. So rewriting them in Rust is no small feat. For the file-system management ones: I appreciate the value of everyone knowing these tools, but they do have some terrible defaults, and I wish there was an alternative between using a GUI/TUI file manager and carefully not stabbing myself in the foot. That's why I started building human-utils (alas it's very much unfinished). https://github.com/xixixao/human-utils https://github.com/xixixao/human-utils
- linsomniac 2y ago
  I like some of the directions you're heading with that. One thing I've thought is it might be useful to have tools that create filesystem objects (like "new" or "mov foo bar/") be able to take permissions. "mov --umask 027 --owner alice:bob foo bar/" and "new --mode a=rx foo/" for example.
  - yjftsjthsd-h 2y ago
    install(1) does that: https://www.man7.org/linux/man-pages/man1/install.1.html https://www.man7.org/linux/man-pages/man1/install.1.html
    - linsomniac 2y ago
      Indeed, under very limited circumstances of copying files and the man page says "-o" and "-g" only work as superuser, though I just tested it and that seems to be more a note about how chgrp()/chown() work (if you are changing to a group/user you don't have privs for), and it works in the case of a copy. Indeed very useful in some cases; I use it in CI/CD pipelines and Dockerfiles typically.
      - yjftsjthsd-h 2y ago
        It also works for creating directories, ex. install -d -m 755 somedir And yeah, I'm pretty sure the owner/group thing is just noting that at least on GNU/Linux that's just how the OS works (I think that varies among unix-likes).
deivid 2y ago
It's a fun pasttime. I'm rewriting mdadm in rust: https://github.com/DavidVentura/mdadm-rs https://github.com/DavidVentura/mdadm-rs Mostly, I am tired of tools requiring root access, or a block device, to function, even in read only mode. If you have a file on disk (eg: a VM's disk) mdadm will refuse to show metadata, requiring root to do so.
- yjftsjthsd-h 2y ago
  > If you have a file on disk (eg: a VM's disk) mdadm will refuse to show metadata, requiring root to do so. If that's an artificial limitation, surely it should be easy to fix?
  - blueflow 2y ago
    Did the previous developers put it there for fun? Probably not.
    - nine_k 2y ago
      The previous developers might have written it at the time when VM disk RAID images were not a consideration. (Apparently it was Linux 3.0, 2011.)
      - yjftsjthsd-h 2y ago
        But even then, the device nodes should be protected by file ownership+permissions; why does the tool do its own check?
        noja 2y ago
        Chesterton’s fence. Ask the developer.
deleted 2y ago
[deleted]
brian-armstrong 2y ago
> " There are between 200 and 300 dependencies in the uutils project. He said that he understood there is always a supply-chain-attack risk, "but that's a risk we are willing to take". There is more and more tooling around to help mitigate the risk, he said. left-pad II, coming soon to a Linux distro near you
- johnny22 2y ago
  pretty sure you'd have to call it something else. I don't think the crates setup allows what exactly happened with left-pad to happen. It's much more likely to involve malicious code.
- charlotte-fyi 2y ago
  A left pad incident isn't possible on crates.io. Yanking a package from the registry doesn't remove the code if you have an existing lockfile.
  - brian-armstrong 2y ago
    left-pad is symbolic of dependency and supply chain issues generally. If all you took away from that incident is that there's risk only from someone unpublishing the module then you probably need to go back and think about it some more.
    - woodruffw 2y ago
      I think it'd be more productive to say that instead, since it's strictly more correct than comparing it to left-pad. (An interesting thing to consider: the worst "supply-chain" type attack in recent memory is probably xz, which has a much more traditional maintenance, development, and distribution model than the median Rust package does. I don't think Rust's ecosystem is even remotely immune to the risk of malicious packages, but I imagine the kinds of dependencies that exist in the current coreutils are much more appealing to a high-sophistication attacker because of their relative lack of publicity/transparency.)
    - charlotte-fyi 2y ago
marsven_422 2y ago
[dead]
noja 2y ago
Perl did this! https://perlpowertools.com/ https://perlpowertools.com/
greenheadedduck 2y ago
I wonder how linux devs feel about the rewrite in Rust. I mean surly loads of them have decades of experience in C, and Rust seems like such a different beast. Can any C developers provide insight, how is this transition?
- blueflow 2y ago
  I don't see an transition happening at all. There are some rust projects, but they are more like an addition to, not an replacement of the current ecosystem.
- krater23 2y ago
  There is no really transistion. Much developers just are ignoring Rust as they ignored D, E, Go, Ruby on Rails(the PHP developers) and some other fashion programming languages. It's a overhyped trend and in some years Rust will find his place beside all the other Languages, but will never be a widespread replacement of C/C++. We tried it in a commercially project because some hyperiders in the team wanted to do so. The truth is, a thing that would be developed in C++ in 1.5 month wasn't done in 6 months caused by things like 'All our developers are newbies in the language, no one really can do meaningful reviews, tons of dependencies(often one dependency in different versions), no good way to integrate cargo in our existing build flow and the lack of fun during programming.' When you don't start a complete new project with bloody newbies, Rust is not a good choice.
tmtvl 2y ago
Rewriting GPL software under the MIT license is a terrible thing to do. The GPL is meant to protect and preserve what should be basic human rights. So-called "permissive" licenses are meant to provide big tech with free labour.
- grg0 2y ago
  Yup, and one doesn't need to look further than FreeBSD to see what the end result is. I wonder, to what extent is the Linux Rust effort "generously subsidized" by corporations? > He is thinking about ""what we are going to leave to the next generation"". Developers starting out don't want to use COBOL, Fortran, or C, he said. They want to work with fancy stuff like Rust, Swift, Go, or Kotlin. Oh, think about the children! The wolf in sheep's clothing.
  - jmclnx 2y ago
    I did not see these posts before I posted similar. I fully believe this is a direction Corporations are pushing. Almost wonder when spyware will be added and restrictive DRM.
    - surajrmal 2y ago
      This is FUD. The folks funding this generally believe in the benefits. Not everything is 4D chess. There are plenty of not more of examples of rewrites where licenses do not change.
      - bayindirh 2y ago
        If that's no 4D chess, and if the license is not that important, why not relicense uutils to GPLv3 then?
        surajrmal 2y ago
        I don't mind GPL, but if I were to start a new project it would be MIT licensed. There is no hidden agenda in my decision to do that and it's certainly not some grand scheme by my employer to make me do that. I simply find gpl less free because it has obligations attached to it. I don't think I'm alone in feeling this way. In fact I think this may very well be the way the majority of folks feel at this point.
timewizard 2y ago
> "I'm going to state the obvious, that Rust is very good for security, for parallelism, for performance". That's not obvious. > He is "almost certain" that code he writes in Rust is going to work well on everything from Android to Windows. I'd think the problem with security in code is cocky developers who believe that some part of the environment is magical and can save them from themselves. > Ledru cited laziness as another reason for using Rust. "So if there is a crate or library doing that work, I'm going to use it. I'm not going to implement it [myself]." Precisely. Where does this "certainty" come from then? > He is thinking about "what we are going to leave to the next generation". At this rate, a complete and total mess, of two slightly incompatible libraries neither of which have any significant features which differentiate it from the other, save for in the imagination of the developers themselves.
- grg0 2y ago
  This Ledru comes across as a total noob indeed. > He is thinking about "what we are going to leave to the next generation". A history lesson in software licensing, maybe? Certainly not one he bothered to learn himself, lol.
  - sylvestre 2y ago
    +1 ;)
jmclnx 2y ago
Seems this project is MIT-licensed. That is fine, but I cannot help this is a way to get Corporations from following the GPL. I wonder if Linux is re-written i rust will it too remove GPL as a factor ? Again due to the license choice I tend to believe this can be seen as a way to move Linux to a Microsoft Type Windows System.
- jeroenhd 2y ago
  There's no specific reason for Rust code not to be GPL licensed. For whatever reason, many Rust devs choose not to use copyleft licences in their projects, but that doesn't preclude GPL projects from using Rust. I would've preferred projects like these to be GPL licensed too, but as the author writes in the comments (https://lwn.net/Articles/1009647/ https://lwn.net/Articles/1009647/): what's the real-world impact of this specific project being MIT? Commercial UNIX systems are pretty much dead, and I doubt the ones that still want to ship customised versions of these tools will respect the GPLv3 licence. I believe copyleft is essential for things like kernels, but for userspace tooling where plenty of alternatives exist, I don't think it's as important.
  - jillesvangurp 2y ago
    There's no good reason for people to use GPL either. People project all sort of idealistic stuff on this license but the reality is that healthy projects with a diverse contributor based are not at any real risk of their code being hijacked. There are plenty of permissively licensed projects that have been around for decades. Copyleft licenses don't provide much additional protection. They impose a requirement on people that modify the software to provide those modifications under the same license. That's about it. Imposing that requirement is important to some people but not really that essential for the long term health of open source projects. If you want to create a fork of OpenBSD kernel and call it DrEvilBSD and release it under the 100% proprietary DrEvil License 1.0, you can do that of course. The license allows you to do that. You are required to preserve the license and copyright notice, of course. For copyleft proponents, this is a wrong that needs to be corrected and they'll use big words like theft and stealing. For permissive software people this is a feature, not a bug. Do whatever you want with the software. These are both valid points of view to hold. In practice, long lived OSS projects get more protection from the fact that they have a large amount of copyright holders (everybody that ever contributed to the code) which makes any form of re-licensing impractical. The Linux kernel will never re-license. Nor will the OpenBSD kernel. It would take the permission of many thousands/tens of thousands of developers; or their surviving relatives (quite a few are no longer alive). Not going to happen. Anyway, the MIT license is a perfectly good license. It's widely used, well understood, easy to understand, etc. It has been around for decades. Countless of OSS projects use it. Perfectly fine choice if your goal is to facilitate others to use your software in whatever way works for them. Whether that's bundling into some proprietary firmware or distributing it in some OSS linux distribution. Giving users of the software that freedom is a fine choice. And kind of important for operating systems. Unless your goal is to keep it out of the hands of companies that sell products to customers based on these operating systems. Which might be why there aren't many AGPL or GPLv3 licensed operating systems. For a project like this coreutils rewrite, the MIT license makes sense. It maximizes usability across diverse systems Linux, BSD, proprietary UNIX, Windows, etc. without imposing restrictions. That’s likely an intentional choice to ensure broad adoption. There's no good reason to restrict that. Probably the developers want to see this go wherever it can go.
jll29 2y ago
I wonder what lessons were learned that could benefit others who want to port command line tools from C to Rust, e.g. particular idioms or re-usable functions (error handling, logging, defaults/dot-file management, command line option parsing). There was a book called "Dr. Dobb's C-tools", which had the commented source code of a C compiler, assembler, linker and std library, and it greatly benefitted me to go beyond K&R's book towards understand the idioms of C programing.
WhereIsTheTruth 2y ago
This article is funny > "I'm going to state the obvious, that Rust is very good for security, for parallelism, for performance". > The idea to replace GNU coreutils with Rust versions was not about security, though, because the GNU versions were already quite secure. "They did an amazing job. They almost don't have any security issues in their code base." And it's not about the licensing, he said. "I'm not interested in that debate." > One of the reasons that Ledru liked Rust for this project, he said, is that it's very portable. He is "almost certain" that code he writes in Rust is going to work well on everything from Android to Windows. > Ledru cited laziness as another reason for using Rust. "So if there is a crate or library doing that work, I'm going to use it. I'm not going to implement it [myself]." There are between 200 and 300 dependencies in the uutils project. He said that he understood there is always a supply-chain-attack risk, "but that's a risk we are willing to take". There is more and more tooling around to help mitigate the risk, he said. People who keep promote this fraud are fraudsters too
- krater23 2y ago
  We just should ignore the evangelists that just reimplement something existing in rust again. Maybe this language will die or find his way in the corner where someone is doing soemthing useful with it.
1vuio0pswjnm7 2y ago
Memory management programming errors is not a problem I am having with "essential Linux packages". I use a custom userland and rely on busybox and toybox for most basic utilities. However, the size and resource requirements of a Rust toolchain and endless dependencies might introduce new problems for me when compiling essential Linux packages.

98 points by worble 2y ago

A big reason the GNU utilities were game changing is not because of their existence, or their functionality, but because of their license... a license which, in no small part, is what not merely motivated but then allowed for their continued existence and functionality: a tit-for-tat, sharing is caring, we're all in this together, fighting for the users approach to software development, one which ensures that no one is going to embrace and extend your software for use in their platform to lock people out of participation (whether directly or indirectly) in control over the hardware they own. It just really really sucks that people are thereby allocating a ton of effort into reimplementing these tools--putting good effort behind a project that even has a good reason to exist (memory safety), even if (as I'll poke at later in this comment) that apparently is explicitly not the reason they are working on this (which shocked me)--with the goal of being "bug for bug compatible" with the upstream copies from the GNU Foundation while carefully ignoring the #1 most important integration (as this affects how the software fits into the whole) test: "is this software 'free' as in freedom?". Of course, they claim that this is some kind of unproductive waste of time "debate", as if the license is the least important part of the software and doesn't matter, and I think some people want to take this narrative. Regardless, whether or not we agree with this--a position that feels a lot like "politics don't matter and are a waste of time, so stop voicing your concerns"--that's not what's going on here: if you look a bit deeper, this project actually cares deeply about its license, and is going out of its way to choose the license it is using, ignore complaints, and avoid ending up GPL. https://www.youtube.com/watch?v=5qTyyMyU2hQ https://www.youtube.com/watch?v=5qTyyMyU2hQ In an interview with FOSS Weekly, Sylvestre Ledru (the main developer, who curiously has a background working on Debian and Firefox, before ending up getting seduced by the clang/LLVM ecosystem), firmly states "it is not about security", focusing only on an interest in learning himself how the full stack of tools function and preparing for a future where new developers don't actually know enough C to contribute; this might seem to fit into the earlier narrative that the license doesn't really matter, which he later restates himself "I don't care that much, as long as it is OSI compliant". This topic comes up multiple times later in the interview, and Steven sticks to his framing that he doesn't care about the license, that this debate is a waste of time, and that he tries to avoid discussing it as it is "more philosophical than technical". Of course, this isn't preventing him from discussing it ;P... this is clearly a big issue that people have with this project, it is one that comes up in most discussions of the project, and--if it really didn't matter, and it really weren't a big deal--you would thereby expect that he'd just change it, to avoid having to discuss it again... ...only, in this interview--in no small part from the interviewer slowly leaking part of their pre-interview discussion to cause the topic to keep coming back up--we learn just how much this developer does seem to care about the license, as, to keep it all as MIT, he's having to avoid looking at the original implementation, in an attempt to avoid accidentally letting his code get infected by GPL, to support some users of the project who actively choose to use this reimplementation to avoid GPL compliance (the example we are given--by the interviewer outing it, not him--is "car manufacturers"). As someone who works in security but finds it demoralizing how often security is used as an excuse for what ends up being an effort to lock users out of a platform due to what is merely some supposedly-accidental property of the effort--including one time I was in a hearing with the US Copyright Office, sitting next to a rep from General Motors who was there to argue that we shouldn't be allowed to jailbreak a "portable all-purpose mobile computing device" because that might include a car (lol)--I found this back/forth in the comments forum on the website for this interview worth reading: https://hackaday.com/2024/07/17/floss-weekly-episode-792-rust-coreutils/ https://hackaday.com/2024/07/17/floss-weekly-episode-792-rus... <AgainAgain> the goal is to “rewrite it in x” is to move everything to permissive liscenses. then lock future changes away. just like every thing else “security” is used as pretext. <Jonathan Bennett> We chatted a bit about exactly that. They make no claim that this effort is for security, and freely admitted that some of their users are doing so precisely because it’s MIT and not GPL. So… Yes, but actually no. <Thovte> That sounds like yes, but actually, yes. No?

panstromek 2y ago
I feel like this shouldn't be surprising. If you use GPL, you give others strong incentive to rewrite an alternative, because GPL is just very heavy burden. I'd even go as far as to say this is an example of failure of GPL. If the intent was to set up incentives to make sure people share the improvements with the community for mutual benefit, but the result is an incentive to rather burn resources to do complete rewrites, then that seems like a failure on all sides to me.
- codeguro 2y ago
  It's not a heavy burden at all. Releasing source code is quick and easy. Github will host it for you for free. The issue isn't the burden. The issue is that they don't want to comply with the license. Let's cut the crap. This is an attack on the four essential freedoms by means of replacement of the foundational libraries. If you want to talk about burning resources to do complete rewrites, let's start with uutils.

1238127 2y ago

[flagged]

Curvature5868 2y ago
Not a rust zealot, but https://github.com/firecracker-microvm/firecracker/tree/main/src https://github.com/firecracker-microvm/firecracker/tree/main... comes to mind.
dylan604 2y ago
If the original thing is improved by being written in Rust as everyone proclaims, then this would be a good thing. However, I have doubts that the years upon years of updates fixing odd behavior, and then forgotten about the whys&hows of those updates that the same issues will not be introduced in the Rust version.
- kjrfghslkdjfl 2y ago
  [dead]
- janice1999 2y ago
  > and then forgotten about the whys&hows of those updates that the same issues will not be introduced in the Rust version. Ideally that's what test suites are for, although I'm sure some deviations/gaps will be caught by users. For uutils they are preserving all the edge cases, replicating the original behaviour. There are benefits to rewrites (far less often in my experience), like doas replacing sudo in BSD distros and culling previous unused and insecure behaviour.
  - estebank 2y ago
    And in the face of gaps of the test suite there's no assurance that consistent output will be preserved across releases of the same application. This is somewhat mitigated by the slow release schedule that these kind of projects usually have, couples with a feeling of "being done" meaning that releases shouldn't have much green field feature development work. But still fixing one bug could cause a regression elsewhere.

shmerl 2y ago

Ripgrep should be included in all distros by default.

janice1999 2y ago
fd is also great and I install it everywhere. https://github.com/sharkdp/fd https://github.com/sharkdp/fd
- evanwpm 2y ago
  I also really like: https://github.com/eza-community/eza https://github.com/eza-community/eza (modern ls replacement) https://github.com/BurntSushi/erd https://github.com/BurntSushi/erd (modern tree replacement) https://github.com/sharkdp/bat https://github.com/sharkdp/bat (modern cat(1) replacement) my .zshrc for every system now uses these as drop in replacements
  - mubou 2y ago
    Wrong erd! https://github.com/solidiquis/erdtree https://github.com/solidiquis/erdtree This looks really cool, though. I might try using this as an `ls` replacement, too, just because of how it shows the recursive directory size like `du`. I've always wished windows explorer did that (do any linux gui file managers?)
konart 2y ago
And https://github.com/ast-grep/ast-grep https://github.com/ast-grep/ast-grep maybe
grandempire 2y ago
It’s a great tool, but it’s not a posix compliant grep.
- shmerl 2y ago
  I think it's more about having a stable interface. If it's not stable - you can't use it as a base tool long term. But if it's stable, what difference does it make if it's POSIX compliant or not?

malkia 2y ago

Is Rust (llvm?) supported on all platforms Linux targets?

preisschild 2y ago
Not yet, afaik the rust4linux devs thus want to create a rust frontend for GCC (gccrs)
- estebank 2y ago
  gccrs (and gcc-codegen which is a project with the same objective but only implementing the backend) is an independent project to Rust4Linux (but they talk to each other).
  - preisschild 2y ago
    Ah ok i thought they were the main contributers and users.
masklinn 2y ago
No. What relevance does that have tho?
- rerdavies 2y ago
  The relevance would be that Rust coreutils cannot be merged into Linux mainline. Obviously.
  - masklinn 2y ago
    Coreutils are not part of the linux project in the first place, and uutils does not aim to be merged into GNU coreutils.
    - mustache_kimono 2y ago
      >> The relevance would be that Rust coreutils cannot be merged into Linux mainline. Obviously. > Coreutils are not part of the linux project in the first place, and uutils does not aim to be merged into GNU coreutils. I think he having fun with you bro.

xixixao 2y ago

I've been using the rewritten coreutils as a reference in implementing human-utils[0]. The amount of complexity, even with pretty high-level Rust std, is still super high. So rewriting them in Rust is no small feat. For the file-system management ones: I appreciate the value of everyone knowing these tools, but they do have some terrible defaults, and I wish there was an alternative between using a GUI/TUI file manager and carefully not stabbing myself in the foot. That's why I started building human-utils (alas it's very much unfinished). https://github.com/xixixao/human-utils https://github.com/xixixao/human-utils

linsomniac 2y ago
I like some of the directions you're heading with that. One thing I've thought is it might be useful to have tools that create filesystem objects (like "new" or "mov foo bar/") be able to take permissions. "mov --umask 027 --owner alice:bob foo bar/" and "new --mode a=rx foo/" for example.
- yjftsjthsd-h 2y ago
  install(1) does that: https://www.man7.org/linux/man-pages/man1/install.1.html https://www.man7.org/linux/man-pages/man1/install.1.html
  - linsomniac 2y ago
    Indeed, under very limited circumstances of copying files and the man page says "-o" and "-g" only work as superuser, though I just tested it and that seems to be more a note about how chgrp()/chown() work (if you are changing to a group/user you don't have privs for), and it works in the case of a copy. Indeed very useful in some cases; I use it in CI/CD pipelines and Dockerfiles typically.
    - yjftsjthsd-h 2y ago
      It also works for creating directories, ex. install -d -m 755 somedir And yeah, I'm pretty sure the owner/group thing is just noting that at least on GNU/Linux that's just how the OS works (I think that varies among unix-likes).

deivid 2y ago

It's a fun pasttime. I'm rewriting mdadm in rust: https://github.com/DavidVentura/mdadm-rs https://github.com/DavidVentura/mdadm-rs Mostly, I am tired of tools requiring root access, or a block device, to function, even in read only mode. If you have a file on disk (eg: a VM's disk) mdadm will refuse to show metadata, requiring root to do so.

yjftsjthsd-h 2y ago
> If you have a file on disk (eg: a VM's disk) mdadm will refuse to show metadata, requiring root to do so. If that's an artificial limitation, surely it should be easy to fix?
- blueflow 2y ago
  Did the previous developers put it there for fun? Probably not.
  - nine_k 2y ago
    The previous developers might have written it at the time when VM disk RAID images were not a consideration. (Apparently it was Linux 3.0, 2011.)
    - yjftsjthsd-h 2y ago
      But even then, the device nodes should be protected by file ownership+permissions; why does the tool do its own check?
      - noja 2y ago
        Chesterton’s fence. Ask the developer.

deleted 2y ago

[deleted]

brian-armstrong 2y ago

> " There are between 200 and 300 dependencies in the uutils project. He said that he understood there is always a supply-chain-attack risk, "but that's a risk we are willing to take". There is more and more tooling around to help mitigate the risk, he said. left-pad II, coming soon to a Linux distro near you

johnny22 2y ago
pretty sure you'd have to call it something else. I don't think the crates setup allows what exactly happened with left-pad to happen. It's much more likely to involve malicious code.
charlotte-fyi 2y ago
A left pad incident isn't possible on crates.io. Yanking a package from the registry doesn't remove the code if you have an existing lockfile.
- brian-armstrong 2y ago
  left-pad is symbolic of dependency and supply chain issues generally. If all you took away from that incident is that there's risk only from someone unpublishing the module then you probably need to go back and think about it some more.
  - woodruffw 2y ago
    I think it'd be more productive to say that instead, since it's strictly more correct than comparing it to left-pad. (An interesting thing to consider: the worst "supply-chain" type attack in recent memory is probably xz, which has a much more traditional maintenance, development, and distribution model than the median Rust package does. I don't think Rust's ecosystem is even remotely immune to the risk of malicious packages, but I imagine the kinds of dependencies that exist in the current coreutils are much more appealing to a high-sophistication attacker because of their relative lack of publicity/transparency.)
  - charlotte-fyi 2y ago

marsven_422 2y ago

[dead]

noja 2y ago

Perl did this! https://perlpowertools.com/ https://perlpowertools.com/

greenheadedduck 2y ago

I wonder how linux devs feel about the rewrite in Rust. I mean surly loads of them have decades of experience in C, and Rust seems like such a different beast. Can any C developers provide insight, how is this transition?

blueflow 2y ago
I don't see an transition happening at all. There are some rust projects, but they are more like an addition to, not an replacement of the current ecosystem.
krater23 2y ago
There is no really transistion. Much developers just are ignoring Rust as they ignored D, E, Go, Ruby on Rails(the PHP developers) and some other fashion programming languages. It's a overhyped trend and in some years Rust will find his place beside all the other Languages, but will never be a widespread replacement of C/C++. We tried it in a commercially project because some hyperiders in the team wanted to do so. The truth is, a thing that would be developed in C++ in 1.5 month wasn't done in 6 months caused by things like 'All our developers are newbies in the language, no one really can do meaningful reviews, tons of dependencies(often one dependency in different versions), no good way to integrate cargo in our existing build flow and the lack of fun during programming.' When you don't start a complete new project with bloody newbies, Rust is not a good choice.

tmtvl 2y ago

Rewriting GPL software under the MIT license is a terrible thing to do. The GPL is meant to protect and preserve what should be basic human rights. So-called "permissive" licenses are meant to provide big tech with free labour.

grg0 2y ago
Yup, and one doesn't need to look further than FreeBSD to see what the end result is. I wonder, to what extent is the Linux Rust effort "generously subsidized" by corporations? > He is thinking about ""what we are going to leave to the next generation"". Developers starting out don't want to use COBOL, Fortran, or C, he said. They want to work with fancy stuff like Rust, Swift, Go, or Kotlin. Oh, think about the children! The wolf in sheep's clothing.
- jmclnx 2y ago
  I did not see these posts before I posted similar. I fully believe this is a direction Corporations are pushing. Almost wonder when spyware will be added and restrictive DRM.
  - surajrmal 2y ago
    This is FUD. The folks funding this generally believe in the benefits. Not everything is 4D chess. There are plenty of not more of examples of rewrites where licenses do not change.
    - bayindirh 2y ago
      If that's no 4D chess, and if the license is not that important, why not relicense uutils to GPLv3 then?
      - surajrmal 2y ago
        I don't mind GPL, but if I were to start a new project it would be MIT licensed. There is no hidden agenda in my decision to do that and it's certainly not some grand scheme by my employer to make me do that. I simply find gpl less free because it has obligations attached to it. I don't think I'm alone in feeling this way. In fact I think this may very well be the way the majority of folks feel at this point.

timewizard 2y ago

> "I'm going to state the obvious, that Rust is very good for security, for parallelism, for performance". That's not obvious. > He is "almost certain" that code he writes in Rust is going to work well on everything from Android to Windows. I'd think the problem with security in code is cocky developers who believe that some part of the environment is magical and can save them from themselves. > Ledru cited laziness as another reason for using Rust. "So if there is a crate or library doing that work, I'm going to use it. I'm not going to implement it [myself]." Precisely. Where does this "certainty" come from then? > He is thinking about "what we are going to leave to the next generation". At this rate, a complete and total mess, of two slightly incompatible libraries neither of which have any significant features which differentiate it from the other, save for in the imagination of the developers themselves.

grg0 2y ago
This Ledru comes across as a total noob indeed. > He is thinking about "what we are going to leave to the next generation". A history lesson in software licensing, maybe? Certainly not one he bothered to learn himself, lol.
- sylvestre 2y ago
  +1 ;)

jmclnx 2y ago

Seems this project is MIT-licensed. That is fine, but I cannot help this is a way to get Corporations from following the GPL. I wonder if Linux is re-written i rust will it too remove GPL as a factor ? Again due to the license choice I tend to believe this can be seen as a way to move Linux to a Microsoft Type Windows System.

jeroenhd 2y ago
There's no specific reason for Rust code not to be GPL licensed. For whatever reason, many Rust devs choose not to use copyleft licences in their projects, but that doesn't preclude GPL projects from using Rust. I would've preferred projects like these to be GPL licensed too, but as the author writes in the comments (https://lwn.net/Articles/1009647/ https://lwn.net/Articles/1009647/): what's the real-world impact of this specific project being MIT? Commercial UNIX systems are pretty much dead, and I doubt the ones that still want to ship customised versions of these tools will respect the GPLv3 licence. I believe copyleft is essential for things like kernels, but for userspace tooling where plenty of alternatives exist, I don't think it's as important.
- jillesvangurp 2y ago
  There's no good reason for people to use GPL either. People project all sort of idealistic stuff on this license but the reality is that healthy projects with a diverse contributor based are not at any real risk of their code being hijacked. There are plenty of permissively licensed projects that have been around for decades. Copyleft licenses don't provide much additional protection. They impose a requirement on people that modify the software to provide those modifications under the same license. That's about it. Imposing that requirement is important to some people but not really that essential for the long term health of open source projects. If you want to create a fork of OpenBSD kernel and call it DrEvilBSD and release it under the 100% proprietary DrEvil License 1.0, you can do that of course. The license allows you to do that. You are required to preserve the license and copyright notice, of course. For copyleft proponents, this is a wrong that needs to be corrected and they'll use big words like theft and stealing. For permissive software people this is a feature, not a bug. Do whatever you want with the software. These are both valid points of view to hold. In practice, long lived OSS projects get more protection from the fact that they have a large amount of copyright holders (everybody that ever contributed to the code) which makes any form of re-licensing impractical. The Linux kernel will never re-license. Nor will the OpenBSD kernel. It would take the permission of many thousands/tens of thousands of developers; or their surviving relatives (quite a few are no longer alive). Not going to happen. Anyway, the MIT license is a perfectly good license. It's widely used, well understood, easy to understand, etc. It has been around for decades. Countless of OSS projects use it. Perfectly fine choice if your goal is to facilitate others to use your software in whatever way works for them. Whether that's bundling into some proprietary firmware or distributing it in some OSS linux distribution. Giving users of the software that freedom is a fine choice. And kind of important for operating systems. Unless your goal is to keep it out of the hands of companies that sell products to customers based on these operating systems. Which might be why there aren't many AGPL or GPLv3 licensed operating systems. For a project like this coreutils rewrite, the MIT license makes sense. It maximizes usability across diverse systems Linux, BSD, proprietary UNIX, Windows, etc. without imposing restrictions. That’s likely an intentional choice to ensure broad adoption. There's no good reason to restrict that. Probably the developers want to see this go wherever it can go.

jll29 2y ago

I wonder what lessons were learned that could benefit others who want to port command line tools from C to Rust, e.g. particular idioms or re-usable functions (error handling, logging, defaults/dot-file management, command line option parsing). There was a book called "Dr. Dobb's C-tools", which had the commented source code of a C compiler, assembler, linker and std library, and it greatly benefitted me to go beyond K&R's book towards understand the idioms of C programing.

WhereIsTheTruth 2y ago

This article is funny > "I'm going to state the obvious, that Rust is very good for security, for parallelism, for performance". > The idea to replace GNU coreutils with Rust versions was not about security, though, because the GNU versions were already quite secure. "They did an amazing job. They almost don't have any security issues in their code base." And it's not about the licensing, he said. "I'm not interested in that debate." > One of the reasons that Ledru liked Rust for this project, he said, is that it's very portable. He is "almost certain" that code he writes in Rust is going to work well on everything from Android to Windows. > Ledru cited laziness as another reason for using Rust. "So if there is a crate or library doing that work, I'm going to use it. I'm not going to implement it [myself]." There are between 200 and 300 dependencies in the uutils project. He said that he understood there is always a supply-chain-attack risk, "but that's a risk we are willing to take". There is more and more tooling around to help mitigate the risk, he said. People who keep promote this fraud are fraudsters too

krater23 2y ago
We just should ignore the evangelists that just reimplement something existing in rust again. Maybe this language will die or find his way in the corner where someone is doing soemthing useful with it.

1vuio0pswjnm7 2y ago

Memory management programming errors is not a problem I am having with "essential Linux packages". I use a custom userland and rely on busybox and toybox for most basic utilities. However, the size and resource requirements of a Rust toolchain and endless dependencies might introduce new problems for me when compiling essential Linux packages.