10 ms·
Was there something wrong with the old coreutils that needed improvement?
by trollbridge 1y ago
Was there something wrong with the old coreutils that needed improvement?
- arresin 1y agoIt wasn’t… “safe”
- throitallaway 1y agoIt seems like I'm probably preaching to the choir, but what really is the attack surface with coreutils? I can't imagine there have been a lot of pwns as a result of the `date` command.
- denkmoon 1y agoTo play devil's advocate, who knows what kind of madness people are handing off to subprocess.run(["date"]) et al. They shouldn't, but I'd bet my last dollar it's out there.
- ls65536 1y agoI can certainly understand it for something like sudo or for other tools where the attack surface is larger and certain security-critical interactions are happening, but in this case it really seems like a questionable tradeoff, where the benefits in this specific case are abstract (theoretically no more possibility of any memory-safety bugs) but the costs are very concrete (incompatibility issues; and possibly other, new, non-memory-safety bugs being introduced with new code). EDIT: Just to be clear, I'm otherwise perfectly happy that these experiments are being done, and we should all be better off for it and learn something as a result. Obviously somebody has assessed that this tradeoff has at least a decent probability of being a net positive here in some timeframe, and if others are unhappy about it then I suppose they're welcome to install another implementation of coreutils, or use a different distro, or write their own, or whatever.
- johnisgood 1y ago> "sudo" Hence "doas". OpenBSD has a lot of new stuff throughout the codebase. No need for adding a bloated dependency (e.g. Rust) just because you want to re-implement "yes" in a "memory-safe language" when you probably have no reasons to.
- deleted 1y ago[deleted]
- JuniperMesos 1y agoI'd prefer it if all software was written in languages that made it as easy as possible to avoid bugs, including memory-safety bugs, regardless of whether it seems like it has a large attack surface or not.
- pdimitar 1y agoI view `uutils` as a good opportunity to get rid of legacy baggage that might be used by just 0.03% of the community but has to sit there and it impedes certain feature adding or bug fixing. F.ex. `sudo-rs` does not support most of what the normal `sudo` does... and it turned out that most people did not need most of `sudo` in the first place. Less code leads to less bugs.
- mort96 1y agoUntrusted input is often stored in files. Coreutils tools are often used to operate on those files. As an obvious example, I sometimes download files from the Internet, then run coreutils sha256sum or the like on those files to verify that they're trustworthy. That means they're untrusted at the time where I use them as input to sha256sum. If there's an RCE in sha256sum (unlikely, but this is a thought experiment to demonstrate an attack vector), then that untrusted file can just exploit that RCE directly. If there's a bug in sha256sum which allows a malicious file to manipulate the result, then a malicious file could potentially make itself look like a trusted file and therefore get past a security barrier. Maybe there's no bug in sha256sum, but I need to base64 decode the file before running sha256sum on it, using the base64 tool from coreutils. If you use your imagination, I'm sure you yourself can think up plenty more use cases where you might run a program from GNU coreutils against untrusted user input. If it helps, here's a Wikipedia article which lists all commands from GNU coreutils: https://en.wikipedia.org/wiki/GNU_Core_Utilities#Commands https://en.wikipedia.org/wiki/GNU_Core_Utilities#Commands EDIT: To be clear, this comment is only intended to explain what the attack surface is, not to weigh in on whether rewriting the tools in Rust improves security. One could argue that it's more likely that the freshly rewritten sha256sum from uutils has a bug than that GNU sha256sum has a bug. The statement "tools from coreutils are sometimes used to operate on untrusted input and therefore have an attack surface worth exploring" is not the same as the statement "rewriting coreutils in Rust improves security". Personally, I'm excited for the uutils stuff, but not primarily because I believe it alone will directly result in significant security improvements in Ubuntu 25.10.
- pantalaimon 1y agoBut if there is a bug in the date command that prevents security updates from being installed, you've got your vulnerability right there. Rust is not a silver bullet.
- LtWorf 1y agoI reported a segfault in "tac" a number of years ago.
- ajross 1y agoYou don't attack coreutils. You attack the scripts. In this case it was an update script that failed because of an incompatibility. It's not too hard at all to imagine one failing in an exploitable way. Honestly, Rust-related hilarity aside, this project was a terrible, terrible idea. Unix shell environments have always been ad hoc and poorly tested, and anything that impacts compatibility is going to break historical code that may literally be decades old. See also the recent insanity of GNU grep suddenly tossing an error when invoked as "fgrep". You just don't do that folks.
- johnny22 1y agoand yet coreutils continues to receive updates in ways that could break things.
- collinfunk 1y ago> See also the recent insanity of GNU grep suddenly tossing an error when invoked as "fgrep". You just don't do that folks. The 'fgrep' and 'egrep' didn't throw errors, it would just send a warning to standard error before behaving as expected. Those commands were never standardized, and everyone is better off using 'grep -F' and 'grep -E' respectively.
- ajross 1y ago> didn't throw errors, it would just send a warning to standard error Noted without comment. Except to say that I've had multiple scripts of my own break via "just" discovering garbage in the output streams. > Those commands were never standardized "Those commands" were present in v7 unix in 1979!
- larusso 1y agoI think he means POSIX. Didn’t check but in some cases posix only covers some options a tool provides not all. It’s a hard lesson I learned while keeping shell scripts portable between Linux and macOS.
- wvh 1y agoA thousand badly written shell scripts might disagree.
- agumonkey 1y agowhen your bug it fully typed
- jwhitlark 1y agoSafer threading for performance improvements was part of it, as I understand.
- cozzyd 1y ago$ /usr/bin/time date Fri Oct 24 10:20:17 AM CDT 2025 0.00user 0.00system 0:00.00elapsed 0%CPU (0avgtext+0avgdata 2264maxresident)k 0inputs+0outputs (0major+93minor)pagefaults 0swaps Imagine how much faster it will be with threading!
- rurban 1y agoRather think of cp, mv, install, backup. Copying files faster is still an ongoing effort, also in gnu coreutils
- 1718627440 1y agoWhen I want to do stuff in parallel at the OS level, I rather user processes and e.g. GNU parallel.
- dwroberts 1y agoProbably stuff like https://www.cvedetails.com/cve/CVE-2015-4042/ https://www.cvedetails.com/cve/CVE-2015-4042/
- hitekker 1y agoDidn't that bug get fixed before it went public?
- kasabali 1y ago"Denial of service" In sort command Is this the best they could come up with?
- xxs 1y agoIn some cases it was possible to crash (overflow) sort.c, not just DoS. I did try to look more info the issue - it was not handled for quite some time however I did not find any real world impact.
- collinfunk 1y agoMinor correction, but that bug was never in any "official" coreutils release. The bug was in a multi-byte character patch that many distributions use (and still use). There have been other CVEs in that patch [1]. But the worst you can do is crash 'sort' with that. Note that uutils also has crashes. Here is one due to unbounded recursion: $ ./target/release/coreutils mkdir -p `python3 -c 'print("./" + "a/" * 32768)'` Segmentation fault (core dumped) Not saying that both issues don't deserve fixing. But I wouldn't really panic over either of them. [1] https://lwn.net/Articles/535735/ https://lwn.net/Articles/535735/
- username223 1y agoNot enough Rust. The thought of rewriting anything as intricate, foundational, and battle-tested as GNU coreutils from scratch scares me. Maybe I'd try it with a mature automatic C-to-Rust translator, but I would still expect years of incompatibilities and reintroduced bugs. See also the "cascade of attention-deficit teenagers" development model.
- kstrauser 1y agoEh. People have written replacements for glibc because they didn't like something or another about it, and that seems to me to be way more fraught with risk than coreutils.
- username223 1y agoFair enough. My gut sense is that C functions are simpler than shell commands, with a handful of parameters rather than a dozen or more flags, and this bug supports that -- they forgot to implement a flag in "date." But I haven't tried to do either, so I could be wrong.
- elcritch 1y agoFolks also run into compatibility issues with musl as well. The biggest I recall was an issue with DNS breaking because musl didn’t implement some piece.
- kasabali 1y agoTBF DNS handling of glibc is crazy.
- School-Cotton 1y agoFWIW, GNU coreutils is itself a rewrite of stuff that existed before, and which has been rewritten multiple other times.
- JuniperMesos 1y ago> The thought of rewriting anything as intricate, foundational, and battle-tested as GNU coreutils from scratch scares me. Maybe I'd try it with a mature automatic C-to-Rust translator, but I would still expect years of incompatibilities and reintroduced bugs. It is extremely bad that it's not a relatively straightforward process for any random programmer to rewrite coreutils from scratch as a several week project. That means that the correct behavior of coreutils is not specified well enough and and it's not easy enough to understand it by reading the source code.
- bilekas 1y agoIt wasn't rewritten in rust yet. Therefore it wasn't complete. /s
- pepa65 1y agoThey hadn't implemented the `-r` flag of `date`... But worse than that, they didn't squeek on the unimplemented flag (because the interface was already accepting it...). This is an incompetent implementer (and project management?)
- baq 1y agoThis joke is getting more worn out than ‘fizz buzz saas in rust’ hn post titles have ever been
- kasabali 1y agoYeah. Except it isn't a joke. Rustafarians are dead serious.
- pdimitar 1y agoSome actual links would be nice. I have not seen a Rust zealot in HN for like at least 3 years at this point. (On Reddit I've seen plenty, but who takes Reddit seriously?)
- kasabali 1y agoHere's a link: https://news.ycombinator.com/item?id=45686919 https://news.ycombinator.com/item?id=45686919
- rolandog 1y agoI have a suspicion it's about the license, like this commenter [0] did a year ago. [0]: https://news.ycombinator.com/item?id=38853429 https://news.ycombinator.com/item?id=38853429
- LtWorf 1y agoI've had the same suspicion since I read about it the first time.
- ndiddy 1y agoAgreed. Since GNU Coreutils is GPLv3 but uutils is MIT, my guess is eventually Canonical will start using "works like the GNU software except you don't have to comply with GPLv3" as a selling point for Ubuntu Core (their IoT focused distro). This would let them sell to companies who want to only permit signed firmware images to run on their devices, which isn't allowed under GPLv3.
- mjmas 1y agoIf it was only for that, they could use/improve busybox, which has the same license as the kernel (GPLv2). Perhaps it is also so they can be used in closed source systems (I have uutils installed on my Windows system which works nicely).
- mort96 1y agoBusybox is frankly a horrible user experience, and will never be a good one. Its niche is to be as small as possible, as a single static executable, while providing most tools you need to get the job done in an embedded system. Bells and whistles like a shell that's nice to use, or a vi implementation with working undo/redo, or extensive built-in documentation in the form of --help output, are non-features which would make busybox worse for its primary use case.
- deleted 1y ago[deleted]
- jcranmer 1y agoIf you're the maintainer of OpenBSD, then implementing coreutils in a given language is a necessary requirement for it to be considered a viable systems language: https://marc.info/?l=openbsd-misc&m=151233345723889&w=2 https://marc.info/?l=openbsd-misc&m=151233345723889&w=2
- _zoltan_ 1y agono, just the usual... people want to rewrite stuff in Rust "just because". it's getting annoying.
- simonask 1y agoOther people are allowed to do whatever they want.
- deleted 1y ago[deleted]
- _zoltan_ 1y agocertainly! I didn't say otherwise. however once software that has been only rewritten for the sake of being written in Rust starts affecting large distributions like Ubuntu, that's a different issue... however one could argue that Ubuntu picking up the brand new Rust based coreutils instead of the old one is a 2nd order effect of "let's rewrite everything in Rust, whether it makes sense of not"
- Ygg2 1y agoSo rewrites in Rust can happen as long as they don't have practical usage? This isn't something that affected Ubuntu. It's something Ubuntu wanted to test in day to day usage.
- _zoltan_ 1y agoideally stupid rewrites never happen, but alas...
- Ygg2 1y agoIirc it started as simple exercise. It just aimed at high compatibility with original coreutils. Which part of that is stupid? License is chosen because Rust is more static linkage friendly. Which leaves exercise part or high compatibility. You might as well say Linux is a stupid rewrite that will never achieve anything circa 1998.
- IshKebab 1y agoI think it's mainly that it's a fun project and Rust is a lot nicer to work with than C. You're way more likely to see modern niceties and UX improvements in these ones than the old ones.
- hactually 1y ago> Rust is a lot nicer to work with than C What? How??
- gpm 1y agoModern conveniences such as compiler support for - Tagged unions so you can easily and correctly return "I have one of these things". - Generics so you can reuse datastructures other people wrote easily and correctly. And a modern toolchain with a package manager that makes it easy to correctly do this. - Compile time reference counting so you don't have to worry about freeing things/unlocking mutex's/... (sometimes also called RAII + a borrow checker). - Type inference - Things that are changed are generally syntactically tagged as mutable which makes it a lot easier to quickly read code - Iterators... And so on and so forth. Rust is in large part "take all the good ideas that came before it and put it in a low level language". In the last 50 years there's been a lot of good ideas, and C doesn't really incorporate any of them.
- simonask 1y agoI agree with your points, except that "compile time reference counting" is not a thing. I'm not sure what that would even mean. :-)
- gpm 1y agoThe borrow tracker tracks whether there is 1, more than 1, or no references to a pointer at any particular time and rust automatically drops it when that last reference (the owning one) goes away. Sounds like compile time reference counting to me :P I didn't invent this way of referring to it, though I don't recall who I stole it from. It's not entirely accurate, but it's a close enough description to capture how rust's mostly automatic memory management works from a distance. If you want a more literal interpretation of compile time reference counting see also: https://docs.rs/static-rc/0.7.0/static_rc/ https://docs.rs/static-rc/0.7.0/static_rc/
- tecoholic 1y agoThis should give you the necessary background https://discourse.ubuntu.com/t/carefully-but-purposefully-oxidising-ubuntu/56995 https://discourse.ubuntu.com/t/carefully-but-purposefully-ox...
- vlz 1y agoTo summarize, a Jon Seager from Canonical says it’s for safety and resilience. > Performance is a frequently cited rationale for “Rewrite it in Rust” projects. While performance is high on my list of priorities, it’s not the primary driver behind this change. These utilities are at the heart of the distribution - and it’s the enhanced resilience and safety that is more easily achieved with Rust ports that are most attractive to me.
- kybernetyk 1y agoThey weren't written in Rust. But I wonder why the borrow checker wouldn't catch the date bug...
- viraptor 1y ago> where date ignores the -r/--reference=file argument This has nothing to do with memory ownership, so borrow checker is irrelevant. Ubuntu just shipped before that argument's handling was implemented.
- egorfine 1y agoAbsolutely nothing. But systemd projects and Rust rewrites have this one thing in common: them being pure virtue signaling they absolutely have to be noticed. And what's a better way to get noticed if not going for something important and core? To me, Rust rewrites look like "just stop oil" road blocks - the more people suffer, the better. PS: Disclaimer: I love Rust. I hate fanboys.
- mustache_kimono 1y ago> To me, Rust rewrites look like "just stop oil" road blocks - the more people suffer, the better. Then blame Canonical? Quit it with the Rust hate.
- egorfine 1y agorust coreutils had zero chance of being even noticed until canonical decided to replace working tools with this.
- rurban 1y agocoreutils once refused to add more options. Like cp, mv --progress, which would have been extremely useful. They've added internal features though, like better hardware support in copying and moving files.