5 ms·

UNIX tips: Learn 10 good UNIX usage habits

64 points by luccastera 19y ago

bayareaguy 19y ago
Not that I ever plan to use it, but every time I see a list like this I wonder when we'll start seeing similar lists for Microsoft's PowerShell (aka monad).
- foonamefoo 19y ago
  Why would we ever need such a list; PowerShell is so tightly integrated with Clippy that you can always rely on him to give you the appropriate wizard for your task.
ojbyrne 19y ago
11. Don't ask for overtime ;-)
deleted 19y ago
[deleted]
brent 19y ago
> Make directory trees in a single swipe. Really? How often will I need to make a directory structure that is several levels deep, but only one directory at each level? >Match certain fields in output, not just lines For someone who was trying to save a few keystrokes, instead of trying waste time getting the right field for awk, just use grep. Its easy, you can't mess it up, and unless the formatting really matters its the best option.
- fendale 19y ago
  mkdir -p is really useful. If I have a script that needs to be a file in (security concerns aside) /basedir/some/path/thats/not/there/ I can just do mkdir -p /basedir/some/path/thats/not/there and not worry about which of the lower level directories exist or not - it will just create them or not complain if they don't exist. Before I knew of the -p switch, I used to split up the whole path and check for/create each directory level in turn - while its not difficult, its not much fun either!
- ralph 19y ago
  > How often will I need to make a directory structure that is several levels deep, but only one directory at each level? When you want to put a file into a directory structure where theres several levels of directories to avoid one directory containing many files, e.g. com/ycombinator/news/foo. > For someone who was trying to save a few keystrokes, instead of trying waste time getting the right field for awk, just use grep. Its easy, you can't mess it up, and unless the formatting really matters its the best option. The FA shows grep fails because 'Dec' matches part of the filename instead of just on the date. awk's a worthy solution in that case and formatting of the output wasn't relevant to the point made.
  - brent 19y ago
    By formatting I was referring to the fact that he had to match the 6th field. If grep would return 100k and only 10 in the desired field awk is obviously handy (the exception). In the general case where you are just trying to hunt things down who stops and counts which field their target string is in. I could have run grep and analyzed the results by the time you count to the right field. If you miscounted however... how do you know?
    - ralph 19y ago
      The example was using awk to pick "Dec" out in the month column whilst pointing out a grep for "Dec" would also match the filename "December" which didn't have "Dec" in the month column. awk's clearly better than grep in this case. It's not hard to count the column and you can always do a ... | awk '{print $6}' | sort | uniq -c to see if you've got it right. In fact, a common way to develope the pipeline for a one-off task is to keep appending a pipe and another command to the previous pipeline command. That way, you can see a sample of what's being produced instead of imagining it. You couldn't have analysed the results of grep, discarding the non-month matches of "Dec" in the time taken to count up to field 6 if the number of lines was at all significant. Don't be silly. If I did miscount and no lines matched I may be suspicious and look more closely. If some lines matched but on a different column than intended I may notice. I can re-run the command after correction or try the {print $6} idea. How would you notice if your manual* analysis failed to spot the extra result from grep here and there. Would you do your manual analysis twice? Ask someone else to do it too? Or just think "good enough" and use the data for the next stage?
- mark-t 19y ago
  > Really? How often will I need to make a directory structure that is several levels deep, but only one directory at each level? Didn't take me long to find one in my scripts directory. Just imagine you're doing something similar by hand: #!/bin/bash # This script strips debugging symbols from programs and stores them separately. # They will be loaded by gdb when needed but will not take up memory when # running the program normally. # determine where to save debugging symbols DEBUG_PATH="/usr/lib/debug" if [[ ${EUID} != 0 ]]; then DEBUG_PATH="${HOME}/lib/debug"; fi # go through list of command line arguments and split each one for x in "$@" do echo "Splitting ${x}" y="${DEBUG_PATH}${x}.debug" mkdir -p "$(dirname $y)" objcopy --only-keep-debug "${x}" "${y}" || continue objcopy --add-gnu-debuglink="${y}" "${x}" || continue chmod a-x,o-w "${y}" strip "${x}" done
jsnx 19y ago
Maybe it would be better if people always used cat, always used wc, &c. Sure, performance is better the other way -- but multiplying the functionality of each command comes at a price, namely that only grognards can do things "the right way". If we instead emphasized the way of distinctly separate commands for distinctly separate things...
- eru 19y ago
  The way of Unix.
ralph 19y ago
"It is generally a good idea to enclose variable calls in double quotation marks". No, it's generally a good idea to learn what quotes do and use the appropriate kind when required. Else you're just adding noise to your code. The backslashes aren't needed in listing 9 since the shell expects more input after `&&' or `||' and automatically switches to PS2. Using xargs is more subtle than they suggest. If may kick off the given command multiple times. This would make a difference, e.g. `xargs wc' would give multiple totals.
dreish 19y ago
Also handy for your .zshrc or .bashrc: mkcd () { mkdir -p $1 cd $1 } Also, moving a tarball before unpacking it is indeed brain-damaged, but, cd /target/directory tar xzvf /source/dir/sourcefile.tar.gz works for me. I don't know why you would need -C, though it saves five keystrokes ("cd -") if you didn't want to end up in the target directory for some reason. Listing 6 looks like a bad idea; it either goes to or makes a directory, depending on whether it existed. I realize he's just illustrating the use of "||", but don't copy the command. Usually you would want pwd to be an invariant. It looks like he really wanted to write: [ -d tmp/a/b/c ] || mkdir -p tmp/a/b/c GNU mkdir -p will not complain if the target directory already existed, so you could just run mkdir -p tmp/a/b/c but then that doesn't illustrate the use of "||". Overuse of 'cat' is a hard habit to break for some reason, especially since it's so harmless. I found it useful as an intermediate step that Z shell is more flexible than other shells about where to put redirection, so: < foo.txt | wc -l works in zsh instead of cat foo.txt (to get line count without a filename, which is useful in shell scripts and long one-liners), and seeing that, I would see more obviously that the pipeline is unnecessary.
- foonamefoo 19y ago
  Your mkcd script breaks for directories with spaces in their names; you've got to put $1 in quotes. Also, never heard of a tar bomb? =)
  - dreish 19y ago
    It works in Z shell, and breaks in Bash -- thanks, I didn't realize that. I don't see how -C protects against a tar bomb. Generally when I untar a tarball from a new source whose packaging habits I'm not familiar with, I tar tzvf it first.
    - foonamefoo 19y ago
      -C protects if you create an empty directory and then use it as the argument to -C; saves you a command line cd and a cd back to where you were, though it doesn't save much if you use your mkcd command to make the directory. (I'm talking about GNU tar where any leading slash(es?) is stripped off of the path of any files in the tar (and .. is disabled))
BrandonM 19y ago
I honestly knew most of this, but I did pick up a few tips that I didn't know; for example, I tend to use grep <whatever> | wc -l Not that it's really harmful or anything, but -c is certainly less keystrokes.

64 points by luccastera 19y ago

Not that I ever plan to use it, but every time I see a list like this I wonder when we'll start seeing similar lists for Microsoft's PowerShell (aka monad).

foonamefoo 19y ago
Why would we ever need such a list; PowerShell is so tightly integrated with Clippy that you can always rely on him to give you the appropriate wizard for your task.

ojbyrne 19y ago

11. Don't ask for overtime ;-)

deleted 19y ago

[deleted]

brent 19y ago

> Make directory trees in a single swipe. Really? How often will I need to make a directory structure that is several levels deep, but only one directory at each level? >Match certain fields in output, not just lines For someone who was trying to save a few keystrokes, instead of trying waste time getting the right field for awk, just use grep. Its easy, you can't mess it up, and unless the formatting really matters its the best option.

fendale 19y ago
mkdir -p is really useful. If I have a script that needs to be a file in (security concerns aside) /basedir/some/path/thats/not/there/ I can just do mkdir -p /basedir/some/path/thats/not/there and not worry about which of the lower level directories exist or not - it will just create them or not complain if they don't exist. Before I knew of the -p switch, I used to split up the whole path and check for/create each directory level in turn - while its not difficult, its not much fun either!
ralph 19y ago
> How often will I need to make a directory structure that is several levels deep, but only one directory at each level? When you want to put a file into a directory structure where theres several levels of directories to avoid one directory containing many files, e.g. com/ycombinator/news/foo. > For someone who was trying to save a few keystrokes, instead of trying waste time getting the right field for awk, just use grep. Its easy, you can't mess it up, and unless the formatting really matters its the best option. The FA shows grep fails because 'Dec' matches part of the filename instead of just on the date. awk's a worthy solution in that case and formatting of the output wasn't relevant to the point made.
- brent 19y ago
  By formatting I was referring to the fact that he had to match the 6th field. If grep would return 100k and only 10 in the desired field awk is obviously handy (the exception). In the general case where you are just trying to hunt things down who stops and counts which field their target string is in. I could have run grep and analyzed the results by the time you count to the right field. If you miscounted however... how do you know?
  - ralph 19y ago
    The example was using awk to pick "Dec" out in the month column whilst pointing out a grep for "Dec" would also match the filename "December" which didn't have "Dec" in the month column. awk's clearly better than grep in this case. It's not hard to count the column and you can always do a ... | awk '{print $6}' | sort | uniq -c to see if you've got it right. In fact, a common way to develope the pipeline for a one-off task is to keep appending a pipe and another command to the previous pipeline command. That way, you can see a sample of what's being produced instead of imagining it. You couldn't have analysed the results of grep, discarding the non-month matches of "Dec" in the time taken to count up to field 6 if the number of lines was at all significant. Don't be silly. If I did miscount and no lines matched I may be suspicious and look more closely. If some lines matched but on a different column than intended I may notice. I can re-run the command after correction or try the {print $6} idea. How would you notice if your manual* analysis failed to spot the extra result from grep here and there. Would you do your manual analysis twice? Ask someone else to do it too? Or just think "good enough" and use the data for the next stage?
mark-t 19y ago
> Really? How often will I need to make a directory structure that is several levels deep, but only one directory at each level? Didn't take me long to find one in my scripts directory. Just imagine you're doing something similar by hand: #!/bin/bash # This script strips debugging symbols from programs and stores them separately. # They will be loaded by gdb when needed but will not take up memory when # running the program normally. # determine where to save debugging symbols DEBUG_PATH="/usr/lib/debug" if [[ ${EUID} != 0 ]]; then DEBUG_PATH="${HOME}/lib/debug"; fi # go through list of command line arguments and split each one for x in "$@" do echo "Splitting ${x}" y="${DEBUG_PATH}${x}.debug" mkdir -p "$(dirname $y)" objcopy --only-keep-debug "${x}" "${y}" || continue objcopy --add-gnu-debuglink="${y}" "${x}" || continue chmod a-x,o-w "${y}" strip "${x}" done

jsnx 19y ago

Maybe it would be better if people always used cat, always used wc, &c. Sure, performance is better the other way -- but multiplying the functionality of each command comes at a price, namely that only grognards can do things "the right way". If we instead emphasized the way of distinctly separate commands for distinctly separate things...

eru 19y ago
The way of Unix.

ralph 19y ago

"It is generally a good idea to enclose variable calls in double quotation marks". No, it's generally a good idea to learn what quotes do and use the appropriate kind when required. Else you're just adding noise to your code. The backslashes aren't needed in listing 9 since the shell expects more input after `&&' or `||' and automatically switches to PS2. Using xargs is more subtle than they suggest. If may kick off the given command multiple times. This would make a difference, e.g. `xargs wc' would give multiple totals.

dreish 19y ago

Also handy for your .zshrc or .bashrc: mkcd () { mkdir -p $1 cd $1 } Also, moving a tarball before unpacking it is indeed brain-damaged, but, cd /target/directory tar xzvf /source/dir/sourcefile.tar.gz works for me. I don't know why you would need -C, though it saves five keystrokes ("cd -") if you didn't want to end up in the target directory for some reason. Listing 6 looks like a bad idea; it either goes to or makes a directory, depending on whether it existed. I realize he's just illustrating the use of "||", but don't copy the command. Usually you would want pwd to be an invariant. It looks like he really wanted to write: [ -d tmp/a/b/c ] || mkdir -p tmp/a/b/c GNU mkdir -p will not complain if the target directory already existed, so you could just run mkdir -p tmp/a/b/c but then that doesn't illustrate the use of "||". Overuse of 'cat' is a hard habit to break for some reason, especially since it's so harmless. I found it useful as an intermediate step that Z shell is more flexible than other shells about where to put redirection, so: < foo.txt | wc -l works in zsh instead of cat foo.txt (to get line count without a filename, which is useful in shell scripts and long one-liners), and seeing that, I would see more obviously that the pipeline is unnecessary.

foonamefoo 19y ago
Your mkcd script breaks for directories with spaces in their names; you've got to put $1 in quotes. Also, never heard of a tar bomb? =)
- dreish 19y ago
  It works in Z shell, and breaks in Bash -- thanks, I didn't realize that. I don't see how -C protects against a tar bomb. Generally when I untar a tarball from a new source whose packaging habits I'm not familiar with, I tar tzvf it first.
  - foonamefoo 19y ago
    -C protects if you create an empty directory and then use it as the argument to -C; saves you a command line cd and a cd back to where you were, though it doesn't save much if you use your mkcd command to make the directory. (I'm talking about GNU tar where any leading slash(es?) is stripped off of the path of any files in the tar (and .. is disabled))

BrandonM 19y ago

I honestly knew most of this, but I did pick up a few tips that I didn't know; for example, I tend to use grep <whatever> | wc -l Not that it's really harmful or anything, but -c is certainly less keystrokes.