9 ms·

Link to the actual implementation: https://github.com/uutils/coreutils https://github.com/uutils/coreutils I love these re-implementations of things. Technolo

Link to the actual implementation: https://github.com/uutils/coreutils https://github.com/uutils/coreutils I love these re-implementations of things. Technology advances by the continued reinvention of the wheel, and even if some efforts end up being of merely didactic interest, it is still important to make them. Coreutils is one of the pillars of our civilization, and thus it should be re-implemented several times on many programming languages. Looking at the source code, it is impressive that this re-implementation is essentially complete! Look at this: https://github.com/uutils/coreutils/tree/main/src/uu https://github.com/uutils/coreutils/tree/main/src/uu It's only missing some obscure and fringe things. An important feature of GNU coreutils is, somewhat sadly, not reproduced by this re-implementation: the copyleft license of the original implementation. This takes out some freedom of the users. For example, when I use GNU "ls", I know that I can always look at its source code and change it to my whim (or hire a programmer to change it for me). However, if I use uutils "ls", this is not necessarily the case, for it may be a re-distribution by some middle-men that has modified it slightly and does not provide the source code. I suppose that the removal of this freedom is accidental, because it was a nice thing.

codetrotter 4y ago
> I suppose that the removal of this freedom is accidental, because it was a nice thing. Not necessarily. Some people prefer the MIT/BSD/ISC family of licenses over GPL.
- jordigh 4y ago
  It should be forbidden to forbid. That's what I don't like. Most of us have employers who are afraid of copyleft. That has made us afraid of copyleft, because we feel like our job is threatened by copyleft. But the erosion of copyleft over the decades has produced phones that track us, websites that control our dating lives, our food, the entertainment we consume, social media that manipulates our emotions and our friendships, and countless other things without us having any recourse or control. For a while, copyleft was good. It gave us Linux on laptops, where we have the source code for nearly every part of the stack. Without copyleft, we have locked phones, tablets, game consoles, e-readers, refrigerators, cars, pacemakers, and many other devices that are difficult to really call our own because they have software that ultimately obeys the manufacturer, not the consumer.
  - graphe 4y ago
    You can have it be open source and locked down. Most of those devices you mention run Linux, BSD, or RTOS.
    - jordigh 4y ago
      Open source, yes. Copyleft, no. GPLv3 forbids what many of these devices do. It is why GPLv3 is so feared by many companies, like Apple, why the GPL was gradually purged from macOS.
      - ludocode 4y ago
        Apple doesn't "fear" the GPLv3. The GPLv3 is simply incompatible with their walled garden. They would not be able to lock down their devices to the extent that they do if they used GPLv3 code, and they care a lot more about their digital locks than they do about any GPLv3 software. Painting it as some kind of irrational fear is misleading and counter-productive. It is perfectly logical for Apple to forbid GPLv3 code. Quite frankly, GCC committed suicide by switching to the GPLv3. Both Apple and Google have entirely abandoned it. Apple invested tens of millions of dollars into building Clang as a direct result of the license change. And now even the BSDs and many embedded Linux platforms have switched to Clang, leaving desktop Linux and a few esoteric systems as the only remaining use cases for GCC. Yes, GCC is still around, but no serious company will invest in GCC for anything new ever again.
        johnny22 4y ago
        except for C++ where it seems that it is advancing faster than clang is.
  - burntsushi 4y ago
    Nice. Recasting "prefer MIT" to "afraid of copyleft." I'm pretty sure we've had this discussion before on lobste.rs, yet you continue to mischaracterize and lump all opposition of copyleft into some irrational position based on fear, or that we somehow can't think for ourselves because our employers don't like copyleft. My full position: https://github.com/BurntSushi/notes/blob/master/2020-10-29_licensing-and-copyleft.md https://github.com/BurntSushi/notes/blob/master/2020-10-29_l...
- medler 4y ago
  I don’t think it was accidental. There’s a long thread in the uutils repo with a discussion (slash flame war) about the license choice: https://github.com/uutils/coreutils/issues/834 https://github.com/uutils/coreutils/issues/834
  - arp242 4y ago
    Thank you; that was an entertaining thread. "GPL is inherently fascist" is a new one.
    - peoplefromibiza 4y ago
      Ballmer would be proud. We've gone full circle.
- cwp 4y ago
  Yup. Whenever I release my code to the world, I use MIT. I'm not brainwashed, I know what I'm doing. If someone incorporates my code into a proprietary product, that's fine. I hope they're successful! I just want the bragging rights.
deleted 4y ago
[deleted]
charcircuit 4y ago
>if I use uutils "ls", this is not necessarily the case, for it may be a re-distribution by some middle-men that has modified it slightly and does not provide the source code You are still free to go and work off of the upstream version of uutils.
- yjftsjthsd-h 4y ago
  How does that help when I want to work on the thing actually running on my machine?
  - charcircuit 4y ago
    And how does GNU coreutils being GPL help you if the ls you have didn't come with source? Even if the ls you have is a fork of GNU coreutils's ls there is nothing you can do to get the source. The FSF can't either. But the FSF can stop it from being distributed and collect money from damages.
    - yjftsjthsd-h 4y ago
      At a minimum, it tilts the scales towards you getting source in the first place.
yjftsjthsd-h 4y ago
> Looking at the source code, it is impressive that this re-implementation is essentially complete! Look at this: https://github.com/uutils/coreutils/tree/main/src/uu https://github.com/uutils/coreutils/tree/main/src/uu It's only missing some obscure and fringe things. It's not just a matter of having all the individual tools, it's a matter of having them be functionally complete and behaving either the same as GNU or POSIX - note the graph part way down the readme that talks about having only slightly better than half of the tests passing. And the tests probably aren't truly comprehensive, especially for things that people didn't realize were implementation specific or non-guaranteed behaviors. It's good to see them making progress, I just want to someone temper the idea that they're close to done.
- dho 4y ago
  You have to be careful with the interpretation of the graph. It tracks the number of files where all tests pass.
  - enriquto 4y ago
    Yes, it seems that they re-implemented essentially all coreutils, and nearly all command line options on all of them. That's the reason why there are so many failures in the test suite, because some obscure option here and there is still not yet implemented. The main functionality of all the tools is already there. This is a huge project, and really impressive. (as a side note, I did not want to turn this thread into a licensing flamewar, even if I dislike the license change. It is more interesting to talk about the tools themselves! How does the performance compare to the GNU implementation? Are they faster? Do they use less memory? Are the executables smaller (e.g., when compiled statically)? Do they compile faster?)
1vuio0pswjnm7 4y ago
"However, if I use uutils "ls", this is not necessarily the case, for it may be a re-distribution by some middle-men that has modified it slightly and does not provide the source code." Solution: Do not use "re-distributions" by middlemen. For example, when I use BSD UNIX I get it directly from one of the two original BSD projects. I compile everything from source. NB. I use both Linux and BSD.
- ihattendorf 4y ago
  That's not always an option on a router or appliance that comes with the software pre-installed.
  - 1vuio0pswjnm7 4y ago
    Solution: If care about control, then do not buy routers or appliances that come with software pre-installed. Building a router kernel and userland from source is easy to do with BSD or something like OpenWRT. Usually open source routers use a single "crunched" or "multicall" binary for utilties such as those in Linux coreutils, the subject of this thread. The process for making crunched binaries in NetBSD is really easy. Compiling busybox is probably not too difficult either. The router (gateway) is the most important computer on the network IMHO. If one buys a router that one cannot control, then to me that suggests the buyer does not really care about controlling the computers that use it. Maybe a computer owner must use a commercial OS that includes pre-installed software. It might be nigh impossible for the computer owner compile that OS from source themselves, let alone the pre-installed software (e.g., humongous web browser). However a router is a much smaller system, no need for graphics or a web browser, and one can compile this from source relatively easily.
est31 4y ago
> An important feature of GNU coreutils is, somewhat sadly, not reproduced by this re-implementation: the copyleft license of the original implementation. This takes out some freedom of the users. Copyleft is very nice but I think the competition that liberally licensed software means to proprietary alternatives is way higher than the competition that liberally licensed software means to copylefted software. Meaning, if there is GNU coreutils around but no liberally licensed alternative, there will be a market for alternatives that proprietary vendors can use. Thus, somewhere out there will maintain a proprietary coreutils library and charge money for it. This code base will always exist as a risk to the GPL licensed coreutils package. If there is a free (as in money) alternative however, the proprietary vendor will have a much harder time to convince some company to buy their product. I think copyleft shines the most when there is a lot of competition with proprietary solutions. Without copyleft, the competing solutions could just incorporate improvements of some package into their own. With copyleft, the copylefted software package has a chance to fight. But in a domain where Free Software has won, and I would consider coreutils to be such an environment, one can also have open source software.
- mik1998 4y ago
  It makes no sense to make a dichotomy between permissively licensed software and proprietary software when any MIT-licensed software can be trivially repurposed as proprietary software, which is in fact standard practice in many fields (i.e. web, anti-personal computers).
  - est31 4y ago
    You can also repurpose MIT licensed software as GPL licensed.
- phkahler 4y ago
  >> Meaning, if there is GNU coreutils around but no liberally licensed alternative, there will be a market for alternatives that proprietary vendors can use. Thus, somewhere out there will maintain a proprietary coreutils library and charge money for it. And where is that?
Gigachad 4y ago
Who would bother repackaging and modifying some trivial shell utilities from the 70s. Copyleft fanatics feel a lot like libertarians thinking they need to bring a utility belt full of weapons for their trip to the supermarket.
LeFantome 4y ago
I am not so sure it is accidental. Many people prefer BSD, MIT, or Apache style licenses. There is open debate which is more or less free ( them or GPL ). In practice, when the Open Source distribution is the dominant implementation, the source tends to be available and modifiable even when the source is from a for profit company. One might look at all the .NET stuff from Microsoft as an example. Clang and LLVM would be other great examples. For the BSD license, the operating system family it was designed for has been available as Open Source since before the GPL has existed. You can still get their equivalent of the core utils in source form today. In fact, there is a Linux distribution going Alpha very soon ( Chimera ) that is going to use them. If most distributions were to adopt Uutils ( this Rust reimplementation ), I fully expect that not only would the source remain available for them but that there would remain essentially a single version that they would all share. I do not think that the GPL is require to maintain that momentum. The big difference is that some commercial company could take this and extend it as their own as you say. In that case though, they would likely just have gone their own way anyway which would mean less sharing, not more. You could argue that MacOS for example is what is wrong with BSD as they took some BSD stuff and have not shared everything back. They have shared some stuff though, likely more than if there was no BSD code to borrow. With the BSD code, it seems very unlikely that MacOS would have used GPL. Anyway, it is all just opinions and preferences. I may not be any more right than you. Who knows? What you are saying though is not 100% obviously right to me either.

codetrotter 4y ago

> I suppose that the removal of this freedom is accidental, because it was a nice thing. Not necessarily. Some people prefer the MIT/BSD/ISC family of licenses over GPL.

jordigh 4y ago
It should be forbidden to forbid. That's what I don't like. Most of us have employers who are afraid of copyleft. That has made us afraid of copyleft, because we feel like our job is threatened by copyleft. But the erosion of copyleft over the decades has produced phones that track us, websites that control our dating lives, our food, the entertainment we consume, social media that manipulates our emotions and our friendships, and countless other things without us having any recourse or control. For a while, copyleft was good. It gave us Linux on laptops, where we have the source code for nearly every part of the stack. Without copyleft, we have locked phones, tablets, game consoles, e-readers, refrigerators, cars, pacemakers, and many other devices that are difficult to really call our own because they have software that ultimately obeys the manufacturer, not the consumer.
- graphe 4y ago
  You can have it be open source and locked down. Most of those devices you mention run Linux, BSD, or RTOS.
  - jordigh 4y ago
    Open source, yes. Copyleft, no. GPLv3 forbids what many of these devices do. It is why GPLv3 is so feared by many companies, like Apple, why the GPL was gradually purged from macOS.
    - ludocode 4y ago
      Apple doesn't "fear" the GPLv3. The GPLv3 is simply incompatible with their walled garden. They would not be able to lock down their devices to the extent that they do if they used GPLv3 code, and they care a lot more about their digital locks than they do about any GPLv3 software. Painting it as some kind of irrational fear is misleading and counter-productive. It is perfectly logical for Apple to forbid GPLv3 code. Quite frankly, GCC committed suicide by switching to the GPLv3. Both Apple and Google have entirely abandoned it. Apple invested tens of millions of dollars into building Clang as a direct result of the license change. And now even the BSDs and many embedded Linux platforms have switched to Clang, leaving desktop Linux and a few esoteric systems as the only remaining use cases for GCC. Yes, GCC is still around, but no serious company will invest in GCC for anything new ever again.
      - johnny22 4y ago
        except for C++ where it seems that it is advancing faster than clang is.
- burntsushi 4y ago
  Nice. Recasting "prefer MIT" to "afraid of copyleft." I'm pretty sure we've had this discussion before on lobste.rs, yet you continue to mischaracterize and lump all opposition of copyleft into some irrational position based on fear, or that we somehow can't think for ourselves because our employers don't like copyleft. My full position: https://github.com/BurntSushi/notes/blob/master/2020-10-29_licensing-and-copyleft.md https://github.com/BurntSushi/notes/blob/master/2020-10-29_l...
medler 4y ago
I don’t think it was accidental. There’s a long thread in the uutils repo with a discussion (slash flame war) about the license choice: https://github.com/uutils/coreutils/issues/834 https://github.com/uutils/coreutils/issues/834
- arp242 4y ago
  Thank you; that was an entertaining thread. "GPL is inherently fascist" is a new one.
  - peoplefromibiza 4y ago
    Ballmer would be proud. We've gone full circle.
cwp 4y ago
Yup. Whenever I release my code to the world, I use MIT. I'm not brainwashed, I know what I'm doing. If someone incorporates my code into a proprietary product, that's fine. I hope they're successful! I just want the bragging rights.

deleted 4y ago

[deleted]

charcircuit 4y ago

>if I use uutils "ls", this is not necessarily the case, for it may be a re-distribution by some middle-men that has modified it slightly and does not provide the source code You are still free to go and work off of the upstream version of uutils.

yjftsjthsd-h 4y ago
How does that help when I want to work on the thing actually running on my machine?
- charcircuit 4y ago
  And how does GNU coreutils being GPL help you if the ls you have didn't come with source? Even if the ls you have is a fork of GNU coreutils's ls there is nothing you can do to get the source. The FSF can't either. But the FSF can stop it from being distributed and collect money from damages.
  - yjftsjthsd-h 4y ago
    At a minimum, it tilts the scales towards you getting source in the first place.

yjftsjthsd-h 4y ago

> Looking at the source code, it is impressive that this re-implementation is essentially complete! Look at this: https://github.com/uutils/coreutils/tree/main/src/uu https://github.com/uutils/coreutils/tree/main/src/uu It's only missing some obscure and fringe things. It's not just a matter of having all the individual tools, it's a matter of having them be functionally complete and behaving either the same as GNU or POSIX - note the graph part way down the readme that talks about having only slightly better than half of the tests passing. And the tests probably aren't truly comprehensive, especially for things that people didn't realize were implementation specific or non-guaranteed behaviors. It's good to see them making progress, I just want to someone temper the idea that they're close to done.

dho 4y ago
You have to be careful with the interpretation of the graph. It tracks the number of files where all tests pass.
- enriquto 4y ago
  Yes, it seems that they re-implemented essentially all coreutils, and nearly all command line options on all of them. That's the reason why there are so many failures in the test suite, because some obscure option here and there is still not yet implemented. The main functionality of all the tools is already there. This is a huge project, and really impressive. (as a side note, I did not want to turn this thread into a licensing flamewar, even if I dislike the license change. It is more interesting to talk about the tools themselves! How does the performance compare to the GNU implementation? Are they faster? Do they use less memory? Are the executables smaller (e.g., when compiled statically)? Do they compile faster?)

1vuio0pswjnm7 4y ago

"However, if I use uutils "ls", this is not necessarily the case, for it may be a re-distribution by some middle-men that has modified it slightly and does not provide the source code." Solution: Do not use "re-distributions" by middlemen. For example, when I use BSD UNIX I get it directly from one of the two original BSD projects. I compile everything from source. NB. I use both Linux and BSD.

ihattendorf 4y ago
That's not always an option on a router or appliance that comes with the software pre-installed.
- 1vuio0pswjnm7 4y ago
  Solution: If care about control, then do not buy routers or appliances that come with software pre-installed. Building a router kernel and userland from source is easy to do with BSD or something like OpenWRT. Usually open source routers use a single "crunched" or "multicall" binary for utilties such as those in Linux coreutils, the subject of this thread. The process for making crunched binaries in NetBSD is really easy. Compiling busybox is probably not too difficult either. The router (gateway) is the most important computer on the network IMHO. If one buys a router that one cannot control, then to me that suggests the buyer does not really care about controlling the computers that use it. Maybe a computer owner must use a commercial OS that includes pre-installed software. It might be nigh impossible for the computer owner compile that OS from source themselves, let alone the pre-installed software (e.g., humongous web browser). However a router is a much smaller system, no need for graphics or a web browser, and one can compile this from source relatively easily.

est31 4y ago

> An important feature of GNU coreutils is, somewhat sadly, not reproduced by this re-implementation: the copyleft license of the original implementation. This takes out some freedom of the users. Copyleft is very nice but I think the competition that liberally licensed software means to proprietary alternatives is way higher than the competition that liberally licensed software means to copylefted software. Meaning, if there is GNU coreutils around but no liberally licensed alternative, there will be a market for alternatives that proprietary vendors can use. Thus, somewhere out there will maintain a proprietary coreutils library and charge money for it. This code base will always exist as a risk to the GPL licensed coreutils package. If there is a free (as in money) alternative however, the proprietary vendor will have a much harder time to convince some company to buy their product. I think copyleft shines the most when there is a lot of competition with proprietary solutions. Without copyleft, the competing solutions could just incorporate improvements of some package into their own. With copyleft, the copylefted software package has a chance to fight. But in a domain where Free Software has won, and I would consider coreutils to be such an environment, one can also have open source software.

mik1998 4y ago
It makes no sense to make a dichotomy between permissively licensed software and proprietary software when any MIT-licensed software can be trivially repurposed as proprietary software, which is in fact standard practice in many fields (i.e. web, anti-personal computers).
- est31 4y ago
  You can also repurpose MIT licensed software as GPL licensed.
phkahler 4y ago
>> Meaning, if there is GNU coreutils around but no liberally licensed alternative, there will be a market for alternatives that proprietary vendors can use. Thus, somewhere out there will maintain a proprietary coreutils library and charge money for it. And where is that?

Gigachad 4y ago

Who would bother repackaging and modifying some trivial shell utilities from the 70s. Copyleft fanatics feel a lot like libertarians thinking they need to bring a utility belt full of weapons for their trip to the supermarket.

LeFantome 4y ago

I am not so sure it is accidental. Many people prefer BSD, MIT, or Apache style licenses. There is open debate which is more or less free ( them or GPL ). In practice, when the Open Source distribution is the dominant implementation, the source tends to be available and modifiable even when the source is from a for profit company. One might look at all the .NET stuff from Microsoft as an example. Clang and LLVM would be other great examples. For the BSD license, the operating system family it was designed for has been available as Open Source since before the GPL has existed. You can still get their equivalent of the core utils in source form today. In fact, there is a Linux distribution going Alpha very soon ( Chimera ) that is going to use them. If most distributions were to adopt Uutils ( this Rust reimplementation ), I fully expect that not only would the source remain available for them but that there would remain essentially a single version that they would all share. I do not think that the GPL is require to maintain that momentum. The big difference is that some commercial company could take this and extend it as their own as you say. In that case though, they would likely just have gone their own way anyway which would mean less sharing, not more. You could argue that MacOS for example is what is wrong with BSD as they took some BSD stuff and have not shared everything back. They have shared some stuff though, likely more than if there was no BSD code to borrow. With the BSD code, it seems very unlikely that MacOS would have used GPL. Anyway, it is all just opinions and preferences. I may not be any more right than you. Who knows? What you are saying though is not 100% obviously right to me either.