7 ms·

Decoded: GNU coreutils (2019)

288 points by pcr910303 6y ago

gautamcgoel 6y ago
I'm blown away by this project. What a great way to learn about the coreutils, and also see how C is written in the real world! I'm curious how the author made the diagrams explaining each utility - did he use Inkscape?
tyingq 6y ago
Really lovely work. I'm curious if the png flowcharts are generated from data, or hand drawn. Edit: Also, some easter egg looking thing at the bottom right of the page: <div class="copyright col-md-6"> ##*#**##****#*#**/\##*###*****#**#*#*#**#******#**#*#*####*#*##* </div> Edit: Fixed asterisks, I think.
- jedimastert 6y ago
  ##*#**##****#*#**/\##*###*****#**#*#*#**#******#**#*#*####*#*##* HN's formatter is having a time with that many asterisks I don't recognize the format, can someone help me out?
  - tyingq 6y ago
    It's 64 characters. I'd guess binary if that /\ wasn't in the middle.
    - bluesign 6y ago
      seems like morse code First part is maizure Ps: hn messed up with stars :)
      - deleted 6y ago
        [deleted]
      - tyingq 6y ago
        Ah, good call. HN ate your asterisks, but yeah, the first bit before the / is "maizure" in morse. Though having no spaces between the letters makes for some ambiguity...hard to decode the rest. ## *# ** ##** **# *#* * == maizure I can find some long words with a dictionary approach in there, like: ARRIVE -> .-.-..-......-. CLEVER -> -.-..-......-..-. DESTINY -> -......-..-.-.-- FENCED -> ..-..-.-.-..-.. MEMBER -> --.---.....-. MISTER -> --.....-..-. (etc) But, too many variations that direction too.
        bluesign 6y ago
        Yeah millions of combinations, I tried but was not patient enough. for the curious: https://www.jbowman.com/remorse/ https://www.jbowman.com/remorse/
        JdeBP 6y ago
        You have certainly got further than people did in https://news.ycombinator.com/item?id=17116855 https://news.ycombinator.com/item?id=17116855 .
        tyingq 6y ago
        update...pretty sure the ending is "MEMBER.FSF.ORG" Maybe an email address?
        chias 6y ago
        Nicely done!
        JdeBP 6y ago
        Now read http://maizure.org/projects/faq.html http://maizure.org/projects/faq.html . (-:
mshockwave 6y ago
Very interesting way to visualize some of the most important cornerstones in *nix systems
ojnabieoot 6y ago
Very nice work and much easier than trawling through the repository. Some ignorant and probably cliched musing: when I look at small utilities like these I am always struck by a seeming distinction between best practices for little C programs versus best practices for large C applications (the author of the post touches on this ad well). In particular, the explicit flow (including goto) and “pedantic” style is actually quite appropriate for something < 1000 lines and where the expected behavior is extremely well understood. In cases like pwd, mkdir, etc, trying to abstract too much is arguably a mistake for maintainability and understanding. I say all this as an immutable functional-first dev who hasn’t done much native code :) And I think the various type-safe / memory-safe / etc versions of these tools are worth developing. But there’s something to be said about well-optimized native code that clearly “does what it says on the box” in a way that’s accessible to anyone who understands basic Linux programming - even if they can only contextually read C code. (My only real gripe is typographic / linting related, mostly due to being a whippersnapper).
- setpatchaddress 6y ago
  Could not disagree more about goto. Small programs always turn into larger ones. And what you have at the end if you're not from the beginning using practices appropriate for larger programs is spaghetti code. I'm not criticizing it in context -- a lot of this code dates back to the mid 80's if I'm not mistaken. But always write new code using scalable idioms.
  - overboard2 6y ago
    If this program has remained small for 40 years, then maybe not all small programs turn into larger ones.
  - monocasa 6y ago
    I'd like to see the use of goto. There's two 'allowed' uses in C that are common and represent good code even today. goto error cleanup stubs, and goto in virtual machine dispatch loops. The size of the codebase doesn't really matter for those cases; they're largely considered the idiomatic way to go about the problems they're trying to solve.
ufo 6y ago
The biggest takeaway for me is that I learned about the existence of some utilities that I had never known were there. Specially "factor" and "tsort".
- vram22 6y ago
  There are many cool / useful less-known utilities in GNU / Linux. Check man7.org for good, though brief info on many of them. I had explored many of them a while ago. Maintained by Michael Kerrisk, author of The Linux Programming Interface, a kind of reference bible for Linux APIs and system calls. Edit: many of which are used in making such utilities.
  - MaxBarraclough 6y ago
    Don't forget recutils. https://www.gnu.org/software/recutils/manual/A-Little-Example.html#A-Little-Example https://www.gnu.org/software/recutils/manual/A-Little-Exampl... https://en.wikipedia.org/wiki/Recfiles https://en.wikipedia.org/wiki/Recfiles
    - vram22 6y ago
      Right, good point. It has also been discussed on HN a few times: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=recutils&sort=byPopularity&type=all https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
  - rustyminnow 6y ago
    Here's the list of coreutils pages for anybody else interested: https://man7.org/linux/man-pages/dir_by_project.html#coreutils https://man7.org/linux/man-pages/dir_by_project.html#coreuti...
- rwmj 6y ago
  There's also "moreutils"[1] which is a set of useful additional tools. "errno" is indispensable if you're a Linux programmer. [1] https://joeyh.name/code/moreutils/ https://joeyh.name/code/moreutils/
mraza007 6y ago
OMG I’m so surprised I was going to post a question on HN yesterday that how can I learn about GNU Coreutils and today I wake up see this What a coincidence!!! Truly an amazing resource on GNU coreutils
gbajson 6y ago
I have just spent 5 minutes trying to find any useful use cases for 'vdir'. Does anyone of have idea why other 'ls' has been created?
- JonathonW 6y ago
  Someone years ago wanted a shortcut for `ls -lb`, didn't want to change aliases on every machine they use, and had access to the coreutils source? In GNU coreutils, it's not really "another" ls; it's the same source code with a preprocessor macro set differently (ls, dir, and vdir all share the same source).
deleted 6y ago
[deleted]
dang 6y ago
Discussed at the time: Decoded: GNU Coreutils - https://news.ycombinator.com/item?id=20328650 https://news.ycombinator.com/item?id=20328650 - July 2019 (55 comments)
psychoslave 6y ago
"How is GNU `yes` so fast?" was already discussed on this topic: https://news.ycombinator.com/item?id=14542938 https://news.ycombinator.com/item?id=14542938
bogomipz 6y ago
What a fantastic post! If the author is reading - please post more of these Linux "decoded" series. You have a great visual style and the content is wonderfully succinct. I highly recommend reading the "Decoded: The 'top' utility (procps)" post as well: http://maizure.org/projects/decoded-top-procps/index.html http://maizure.org/projects/decoded-top-procps/index.html

288 points by pcr910303 6y ago

I'm blown away by this project. What a great way to learn about the coreutils, and also see how C is written in the real world! I'm curious how the author made the diagrams explaining each utility - did he use Inkscape?

tyingq 6y ago

Really lovely work. I'm curious if the png flowcharts are generated from data, or hand drawn. Edit: Also, some easter egg looking thing at the bottom right of the page: <div class="copyright col-md-6"> ##*#**##****#*#**/\##*###*****#**#*#*#**#******#**#*#*####*#*##* </div> Edit: Fixed asterisks, I think.

jedimastert 6y ago
##*#**##****#*#**/\##*###*****#**#*#*#**#******#**#*#*####*#*##* HN's formatter is having a time with that many asterisks I don't recognize the format, can someone help me out?
- tyingq 6y ago
  It's 64 characters. I'd guess binary if that /\ wasn't in the middle.
  - bluesign 6y ago
    seems like morse code First part is maizure Ps: hn messed up with stars :)
    - deleted 6y ago
      [deleted]
    - tyingq 6y ago
      Ah, good call. HN ate your asterisks, but yeah, the first bit before the / is "maizure" in morse. Though having no spaces between the letters makes for some ambiguity...hard to decode the rest. ## *# ** ##** **# *#* * == maizure I can find some long words with a dictionary approach in there, like: ARRIVE -> .-.-..-......-. CLEVER -> -.-..-......-..-. DESTINY -> -......-..-.-.-- FENCED -> ..-..-.-.-..-.. MEMBER -> --.---.....-. MISTER -> --.....-..-. (etc) But, too many variations that direction too.
      - bluesign 6y ago
        Yeah millions of combinations, I tried but was not patient enough. for the curious: https://www.jbowman.com/remorse/ https://www.jbowman.com/remorse/
      - JdeBP 6y ago
        You have certainly got further than people did in https://news.ycombinator.com/item?id=17116855 https://news.ycombinator.com/item?id=17116855 .
        tyingq 6y ago
        update...pretty sure the ending is "MEMBER.FSF.ORG" Maybe an email address?
        chias 6y ago
        Nicely done!
        JdeBP 6y ago
        Now read http://maizure.org/projects/faq.html http://maizure.org/projects/faq.html . (-:

mshockwave 6y ago

Very interesting way to visualize some of the most important cornerstones in *nix systems

ojnabieoot 6y ago

Very nice work and much easier than trawling through the repository. Some ignorant and probably cliched musing: when I look at small utilities like these I am always struck by a seeming distinction between best practices for little C programs versus best practices for large C applications (the author of the post touches on this ad well). In particular, the explicit flow (including goto) and “pedantic” style is actually quite appropriate for something < 1000 lines and where the expected behavior is extremely well understood. In cases like pwd, mkdir, etc, trying to abstract too much is arguably a mistake for maintainability and understanding. I say all this as an immutable functional-first dev who hasn’t done much native code :) And I think the various type-safe / memory-safe / etc versions of these tools are worth developing. But there’s something to be said about well-optimized native code that clearly “does what it says on the box” in a way that’s accessible to anyone who understands basic Linux programming - even if they can only contextually read C code. (My only real gripe is typographic / linting related, mostly due to being a whippersnapper).

setpatchaddress 6y ago
Could not disagree more about goto. Small programs always turn into larger ones. And what you have at the end if you're not from the beginning using practices appropriate for larger programs is spaghetti code. I'm not criticizing it in context -- a lot of this code dates back to the mid 80's if I'm not mistaken. But always write new code using scalable idioms.
- overboard2 6y ago
  If this program has remained small for 40 years, then maybe not all small programs turn into larger ones.
- monocasa 6y ago
  I'd like to see the use of goto. There's two 'allowed' uses in C that are common and represent good code even today. goto error cleanup stubs, and goto in virtual machine dispatch loops. The size of the codebase doesn't really matter for those cases; they're largely considered the idiomatic way to go about the problems they're trying to solve.

ufo 6y ago

The biggest takeaway for me is that I learned about the existence of some utilities that I had never known were there. Specially "factor" and "tsort".

vram22 6y ago
There are many cool / useful less-known utilities in GNU / Linux. Check man7.org for good, though brief info on many of them. I had explored many of them a while ago. Maintained by Michael Kerrisk, author of The Linux Programming Interface, a kind of reference bible for Linux APIs and system calls. Edit: many of which are used in making such utilities.
- MaxBarraclough 6y ago
  Don't forget recutils. https://www.gnu.org/software/recutils/manual/A-Little-Example.html#A-Little-Example https://www.gnu.org/software/recutils/manual/A-Little-Exampl... https://en.wikipedia.org/wiki/Recfiles https://en.wikipedia.org/wiki/Recfiles
  - vram22 6y ago
    Right, good point. It has also been discussed on HN a few times: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=recutils&sort=byPopularity&type=all https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
- rustyminnow 6y ago
  Here's the list of coreutils pages for anybody else interested: https://man7.org/linux/man-pages/dir_by_project.html#coreutils https://man7.org/linux/man-pages/dir_by_project.html#coreuti...
rwmj 6y ago
There's also "moreutils"[1] which is a set of useful additional tools. "errno" is indispensable if you're a Linux programmer. [1] https://joeyh.name/code/moreutils/ https://joeyh.name/code/moreutils/

mraza007 6y ago

OMG I’m so surprised I was going to post a question on HN yesterday that how can I learn about GNU Coreutils and today I wake up see this What a coincidence!!! Truly an amazing resource on GNU coreutils

gbajson 6y ago

I have just spent 5 minutes trying to find any useful use cases for 'vdir'. Does anyone of have idea why other 'ls' has been created?

JonathonW 6y ago
Someone years ago wanted a shortcut for `ls -lb`, didn't want to change aliases on every machine they use, and had access to the coreutils source? In GNU coreutils, it's not really "another" ls; it's the same source code with a preprocessor macro set differently (ls, dir, and vdir all share the same source).

deleted 6y ago

[deleted]

dang 6y ago

Discussed at the time: Decoded: GNU Coreutils - https://news.ycombinator.com/item?id=20328650 https://news.ycombinator.com/item?id=20328650 - July 2019 (55 comments)

psychoslave 6y ago

"How is GNU `yes` so fast?" was already discussed on this topic: https://news.ycombinator.com/item?id=14542938 https://news.ycombinator.com/item?id=14542938

bogomipz 6y ago

What a fantastic post! If the author is reading - please post more of these Linux "decoded" series. You have a great visual style and the content is wonderfully succinct. I highly recommend reading the "Decoded: The 'top' utility (procps)" post as well: http://maizure.org/projects/decoded-top-procps/index.html http://maizure.org/projects/decoded-top-procps/index.html