7 ms·

I'm a 4th year student in a well-respected CS program, I have used linux off and on for over three years now, and I would consider myself perfectly capable of m

I'm a 4th year student in a well-respected CS program, I have used linux off and on for over three years now, and I would consider myself perfectly capable of maintaining a shell script using any of the utilities you mention. However, without checking the man pages I could not tell you the behavior of the -n flag for sed, the -dc flags for tr, the -f flag for awk, or the -tr flags for ls, and I only have a guess as to what comm -23 does. This is not the bread and butter of programming, where everyone worth a damn should be able to decipher "int i = 10;" even if it's not their brand of syntax. There are people who are perfectly capable of understanding and maintaining a piece of software that uses "tr --delete --complement" but would be dumbfounded when confronted with "tr -dc" until they looked it up.

rlpb 14y ago
> This is not the bread and butter of programming... This is, however, the bread and butter of shell programming. > ... "tr --delete --complement"... And, likewise, I'm not sure of the meaning of --delete and --complement, but I do understand "tr -dc". I'd have to look up --delete and --complement to work out what they're the equivalent of.
- YokoZar 14y ago
  These days I think it's clear that the dominant use case for shell scripts is not a full time responsibility but rather edge work for deploying/installing code written in other languages. Most people looking at shell code aren't going to be shell masters, they're going to be engineers of more modern things that have to look at the shell script for some reason.
  - rlpb 14y ago
    > ...they're going to be engineers of more modern things... Have you read TAOUP? The entire Unix philosophy hinges around the shell, since the shell is the tool that lets you glue different pieces together. The shell is not outdated. If you live entirely within one language ecosystem, then you may be tempted to consider the shell an afterthought, and make all services available via API calls instead. But this is inefficient, slows down development, reduces flexibility and limits the capabilities of your programs. This fallacy is old and certainly is not "modern". See TAOUP for details; it explains this far better than I ever could.
    - aidenn0 14y ago
      But a lot of the defaults in the bourne shell and various posix utilities are wrong. One example: space as a field separator is clearly the wrong default, since it's nearly impossible to not find file-names with spaces in them these days.
- danielvinson 14y ago
  "This is, however, the bread and butter of shell programming." This is really old-fashioned thinking. The people who have been in this industry for 10+ years all grew up without computers - and thus had to memorize all of the flags, obscure shell commands, and weird regular expression. What was taught in school back then was to do things perfectly the first time, because CPU cycles were expensive, and bugs were time consuming. The generation that graduated from college about 2 years ago is the first one that had universal access to a search engine since they knew how to use a computer. This generation (and all future ones for the rest of time) sees memorizing things like this as more of a waste of mental space. What is taught in Universities now is problem solving, research, and larger concepts. There is very little straight memorization, and almost zero programming by hand/whiteboard programming. No current college grad will be able to write a shell script without reference material, but every college grab should be able to write any simple script in less than 5 minutes with Google.
  - ralph 14y ago
    They probably think they invented sex too. ;-) Memorisation isn't the issue. Knowing the tools available and their options is. That comes with study. Unix command line is a boon for problem solving and experimentation, just go and read Jon Bentley's _Programming Pearls_. One can't Google for what one doesn't realise can be done.
  - kragen 14y ago
    You're going to have a hard time making it through Hemingway if you have to reach for a dictionary every time you come across a word like "middleweight" or "khaki", even if your dictionary is online and answers your query in less than a second. You need to memorize the meanings of a vocabulary of words in order to read fluently or write well. That's not "old-fashioned thinking". It's just common sense. And it's just as true of shell scripts as it is of electrical engineering, mechanical engineering, poetry, or math.
    - rrreese 14y ago
      While both the above words are known to me, I do like that when reading a book in Kindle on my iPad, I can tap a word, and have its definition show.
ralph 14y ago
Is that a signed integer or unsigned? How many bits? What's the lifetime of i? One needs to know more than is inferred by a few characters. Seeing the long options just makes you think you understand what it's doing without knowing what it does. The latter comes from study; there's no short-cut. Fortunately, Unix is worthy of study unlike for example the Windows API.
- Zancarius 14y ago
  I can't help but agree with this sentiment. Generally, if the operator doesn't understand what a command line option does and isn't willing to read the man page, they're open to a whole slew of unpleasant surprises (although I think this applies to many things, not just userland utilities). Long options simply provide a false sense of security and may not always do what someone thinks they're supposed to do. Worse, they may not always be available as others have pointed out insofar as the BSD flavors of specific tools provide. I like what you said about "study." There truly is no shortcut for reading documentation available with the system, and assuming by inference what a command is supposed to do based on its options when neither the command nor options are understood just seems to be horribly, horribly bad practice in my mind.
  - sirclueless 14y ago
    Unless I am doing code review, I don't read code with the intent to find latent bugs or insecure idioms. I assume the programmer who wrote the code was competent, and wrote a correctly functioning program. (If I cannot assume this, then why am I using the code at all?) If I am doing code review, then I have the man pages open anyways. If I am trying to write bug-free code myself, then I have the man pages open anyways. In these cases it doesn't much matter what form of flag I use. But the most common case of reading code is a brief scan trying to grok what the code does, which shouldn't require frequent reference to man-pages in much the same way that reading a novel shouldn't require frequent reference to a dictionary. I understand your concerns about false sense of security and unexpected behavior in corner cases. I've browsed the IOCCC, I've browsed the CVE database, I know how easy it is to hide nasty behavior in unexpected corner case interactions, and that the only defense against them is vigilant attention to documented behavior. To my mind the most common interactions with code, in decreasing order, are as follows: 1. Executing it. (Flag agnostic, portability issues aside.) 2. Reading it. (Descriptive flags >>> Cryptic flags.) 3. Maintaining it. (Standard flags >> Obscure flags.) 4. Writing it. (Short flags > Verbose flags.) So I would say the most important thing is that flags are descriptive of their behavior. The second most important thing is that they are common and standardized, but this isn't as important as descriptiveness -- a less common flag that describes its operation better wins. The least important thing is brevity, it only matters once.
mbillie1 14y ago
If you aren't sure what the "-<x>" flag does for a particular command, but you are a competent Unix user, you should be fine with the man page or 10 seconds of Google. If you aren't used to Unix, then I think it's dangerous to expect the verbose flags to explain things simply to you. For example, what would --mmap or --null-data mean to you in the context of the grep command? Are they immediately obvious?
iheartfisting 14y ago
Yawn, people who start comments with credential parading... Let your insight speak for itself, no one gives a fuck how well-respected you find yourself.

rlpb 14y ago

> This is not the bread and butter of programming... This is, however, the bread and butter of shell programming. > ... "tr --delete --complement"... And, likewise, I'm not sure of the meaning of --delete and --complement, but I do understand "tr -dc". I'd have to look up --delete and --complement to work out what they're the equivalent of.

YokoZar 14y ago
These days I think it's clear that the dominant use case for shell scripts is not a full time responsibility but rather edge work for deploying/installing code written in other languages. Most people looking at shell code aren't going to be shell masters, they're going to be engineers of more modern things that have to look at the shell script for some reason.
- rlpb 14y ago
  > ...they're going to be engineers of more modern things... Have you read TAOUP? The entire Unix philosophy hinges around the shell, since the shell is the tool that lets you glue different pieces together. The shell is not outdated. If you live entirely within one language ecosystem, then you may be tempted to consider the shell an afterthought, and make all services available via API calls instead. But this is inefficient, slows down development, reduces flexibility and limits the capabilities of your programs. This fallacy is old and certainly is not "modern". See TAOUP for details; it explains this far better than I ever could.
  - aidenn0 14y ago
    But a lot of the defaults in the bourne shell and various posix utilities are wrong. One example: space as a field separator is clearly the wrong default, since it's nearly impossible to not find file-names with spaces in them these days.
danielvinson 14y ago
"This is, however, the bread and butter of shell programming." This is really old-fashioned thinking. The people who have been in this industry for 10+ years all grew up without computers - and thus had to memorize all of the flags, obscure shell commands, and weird regular expression. What was taught in school back then was to do things perfectly the first time, because CPU cycles were expensive, and bugs were time consuming. The generation that graduated from college about 2 years ago is the first one that had universal access to a search engine since they knew how to use a computer. This generation (and all future ones for the rest of time) sees memorizing things like this as more of a waste of mental space. What is taught in Universities now is problem solving, research, and larger concepts. There is very little straight memorization, and almost zero programming by hand/whiteboard programming. No current college grad will be able to write a shell script without reference material, but every college grab should be able to write any simple script in less than 5 minutes with Google.
- ralph 14y ago
  They probably think they invented sex too. ;-) Memorisation isn't the issue. Knowing the tools available and their options is. That comes with study. Unix command line is a boon for problem solving and experimentation, just go and read Jon Bentley's _Programming Pearls_. One can't Google for what one doesn't realise can be done.
- kragen 14y ago
  You're going to have a hard time making it through Hemingway if you have to reach for a dictionary every time you come across a word like "middleweight" or "khaki", even if your dictionary is online and answers your query in less than a second. You need to memorize the meanings of a vocabulary of words in order to read fluently or write well. That's not "old-fashioned thinking". It's just common sense. And it's just as true of shell scripts as it is of electrical engineering, mechanical engineering, poetry, or math.
  - rrreese 14y ago
    While both the above words are known to me, I do like that when reading a book in Kindle on my iPad, I can tap a word, and have its definition show.

ralph 14y ago

Is that a signed integer or unsigned? How many bits? What's the lifetime of i? One needs to know more than is inferred by a few characters. Seeing the long options just makes you think you understand what it's doing without knowing what it does. The latter comes from study; there's no short-cut. Fortunately, Unix is worthy of study unlike for example the Windows API.

Zancarius 14y ago
I can't help but agree with this sentiment. Generally, if the operator doesn't understand what a command line option does and isn't willing to read the man page, they're open to a whole slew of unpleasant surprises (although I think this applies to many things, not just userland utilities). Long options simply provide a false sense of security and may not always do what someone thinks they're supposed to do. Worse, they may not always be available as others have pointed out insofar as the BSD flavors of specific tools provide. I like what you said about "study." There truly is no shortcut for reading documentation available with the system, and assuming by inference what a command is supposed to do based on its options when neither the command nor options are understood just seems to be horribly, horribly bad practice in my mind.
- sirclueless 14y ago
  Unless I am doing code review, I don't read code with the intent to find latent bugs or insecure idioms. I assume the programmer who wrote the code was competent, and wrote a correctly functioning program. (If I cannot assume this, then why am I using the code at all?) If I am doing code review, then I have the man pages open anyways. If I am trying to write bug-free code myself, then I have the man pages open anyways. In these cases it doesn't much matter what form of flag I use. But the most common case of reading code is a brief scan trying to grok what the code does, which shouldn't require frequent reference to man-pages in much the same way that reading a novel shouldn't require frequent reference to a dictionary. I understand your concerns about false sense of security and unexpected behavior in corner cases. I've browsed the IOCCC, I've browsed the CVE database, I know how easy it is to hide nasty behavior in unexpected corner case interactions, and that the only defense against them is vigilant attention to documented behavior. To my mind the most common interactions with code, in decreasing order, are as follows: 1. Executing it. (Flag agnostic, portability issues aside.) 2. Reading it. (Descriptive flags >>> Cryptic flags.) 3. Maintaining it. (Standard flags >> Obscure flags.) 4. Writing it. (Short flags > Verbose flags.) So I would say the most important thing is that flags are descriptive of their behavior. The second most important thing is that they are common and standardized, but this isn't as important as descriptiveness -- a less common flag that describes its operation better wins. The least important thing is brevity, it only matters once.

mbillie1 14y ago

If you aren't sure what the "-<x>" flag does for a particular command, but you are a competent Unix user, you should be fine with the man page or 10 seconds of Google. If you aren't used to Unix, then I think it's dangerous to expect the verbose flags to explain things simply to you. For example, what would --mmap or --null-data mean to you in the context of the grep command? Are they immediately obvious?

iheartfisting 14y ago

Yawn, people who start comments with credential parading... Let your insight speak for itself, no one gives a fuck how well-respected you find yourself.