19 ms·

Unix Commands I Wish I’d Discovered Years Earlier

480 points by micahalles 13y ago

JimmaDaRustla 13y ago
I'm always surprised at the lack of general knowledge of Regular Expressions, so egrep would be a huge one for newcomers.
foobeer 13y ago
Never knew about the "cal" command. I especially like that you can quickly look up a previous or future month/year. >> cal 3 1973 quickly shows a calendar of March 1973
- yogo 13y ago
  Aliasing to 'cal -3' can also be good if you want a little continuity.
  - xrt 13y ago
    The -3 arg is not available on OS X. Odd, since it's been in BSD Unix since forever. http://www.freebsd.org/cgi/man.cgi?query=cal http://www.freebsd.org/cgi/man.cgi?query=cal
    - clarry 13y ago
      It's hardly ubiquitous. NetBSD has had it for 11 years. FreeBSD has had it for three years and six months. DragonFly doesn't have it, OpenBSD doesn't have it... http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/cal/cal.c?rev=1.12&content-type=text/x-cvsweb-markup&only_with_tag=MAIN http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/cal/cal.c?re... http://svnweb.freebsd.org/base?view=revision&revision=204697 http://svnweb.freebsd.org/base?view=revision&revision=204697
- Shish2k 13y ago
  you can even look up `cal sept 1752`
  - quchen 13y ago
    Funny, 3-13 are missing here. > cal sep 1752 September 1752 Su Mo Tu We Th Fr Sa 1 2 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Edit: Oh, colour me surprised. https://en.wikipedia.org/wiki/September_14 https://en.wikipedia.org/wiki/September_14
    - ghotli 13y ago
      This was the best thing I learned today
- mturmon 13y ago
  Right, you can use this to easily find the number of days in an arbitrary month in a script: % cal 2 1972 | tail +3 | wc -w 29
  - parennoob 13y ago
    tail: cannot open +3 for reading: No such file or directory What am I doing wrong? This worked like a charm though: cal dec 2002 | tail -6 | wc -w (since it always takes up 6 lines, even sep 1752)
    - typicalrunt 13y ago
      It might be system-dependent. I'm on OSX and the command works properly.
    - jbarber 13y ago
      The "+n" syntax is not standards compliant, from the tail info page (on fedora 19): On older systems, the leading '-' can be replaced by '+' in the obsolete option syntax with the same meaning as in counts, and obsolete usage overrides normal usage when the two conflict. This obsolete behavior can be enabled or disabled with the '_POSIX2_VERSION' environment variable (*note Standards conformance::).
happywolf 13y ago
I recommend Quicksilver on mac, free and powerful. It lets you to find a lot of stuff (application, contact, email, etc.) and manipulate them (send, forward, print, etc.) http://qsapp.com/ http://qsapp.com/
- jentulman 13y ago
  Also in that vein on Mac it's worth having a look at Alfred http://alfredapp.com http://alfredapp.com
  - actionscripted 13y ago
    Alread rocks. You can develop workflows for common groups of tasks and it integrates well with iTunes, 1Password, Terminal and other system actions/apps.
    - adamnemecek 13y ago
      It really does. My computer would be like 30% less usable without it. The file search is particularly amazing.
- 9999 13y ago
  I second the Quicksilver recommendation. When you combine Quicksilver triggers (keyboard shortcut available regardless of what application you're in) with applescript, and fabric it's very powerful. I have a trigger that that restarts all of my local services using fabric and opens up new terminal windows running tails of the relevant logs. The Applescript is this if anyone wants to do something similar: tell application "Terminal" do script "tail -F /somepath/some.log" do script "tail -F /anotherpath/another.log" do script "sudo fab -f '/pathto/fabfile.py' local deploy" activate end tell Here's a detailed explanation of setting up Triggers: http://leafraker.com/2007/09/17/how-to-create-a-quicksilver-trigger/ http://leafraker.com/2007/09/17/how-to-create-a-quicksilver-...
ck2 13y ago
I want a command that counts files in a tree like du does sizes but without having to pipe find through wc which is crazy for hundred thousand files. Can't they just directly access inodes for high speed counting?
- casca 13y ago
  Counting number of files is probably not a sufficiently frequently used case to justify a new standard command. What part of piping concerns you?
  - sp332 13y ago
    It can be really slow, even for a few tens of thousands of tiles. I remember watching Nautilus's "properties" panel slowly count up the number of files and directories.
    - jonhohle 13y ago
      I'm not sure what this has to do with piping (why would nautilus pipe the output of find when doing a directory listing?), but I doubt it's the pipe that's causing performance issues. Uncached file system access has a tendency to be slow[0]. 0 - https://gist.github.com/nahurst/3711264 https://gist.github.com/nahurst/3711264
- njharman 13y ago
  > Can't they just directly access inodes for high speed counting? Don't know what you mean by that exactly. But, I'd be stunned if find is actually reading the actual files rather than just reading the directory entries in directory "files". Which is what I'd call accessing inodes "directly"
- bendaid 13y ago
  tree does this, don't know about suppressing the actual tree, but it tells you number of files and dirs printed at the end.
  - 13y ago
brown9-2 13y ago
I'm unreasonably excited about discovering xxd.
- Luyt 13y ago
  xxd is on OSX, but not on FreeBSD. Under FreeBSD, I always use 'hd somefile.bin'.
  - D9u 13y ago
    I have xxd on my FreeBSD setup. uname -rms FreeBSD 9.1-RELEASE-p19 amd64 Although the xxd man page doesn't have the BSD label, and is from 1996, so I'm not sure if xxd only recently found its way into FBSD. This manual page documents xxd version 1.7 AUTHOR (c) 1990-1997 by Juergen Weigert
    - laumars 13y ago
      It maybe in ports, but it's not part of the base install: $ xxd xxd: Command not found. $ uname -rms FreeBSD 9.1-RELEASE amd64 edit: fixed typo where I put xdd instead of xxd
      - pessimizer 13y ago
        s/xdd/xxd/g
        laumars 13y ago
        Sorry, that was a typo when posting on here. I did originally run it as xxd: root@primus:/usr/ports # xxd xxd: Command not found. edit: It doesn't show up in my ports either: root@primus:/usr/ports # make search name=xxd Port: textproc/xxdiff Moved: Date: 2013-07-26 Reason: Has expired: Depends on Qt 3.x Port: textproc/xxdiff-scripts Moved: Date: 2013-07-26 Reason: Has expired: Depends on Qt 3.x Obviously xxdiff is a different (if you pardon the pun) utility, but xxd should show up in that list also. edit 2: Another comment on here might explain why you're seeing xxd and I'm not: https://news.ycombinator.com/item?id=6360814 https://news.ycombinator.com/item?id=6360814 (it's not a separate utility, it's part of vim) Since I didn't install vim on that box (just use basic vi), xxd wouldn't have installed as part of it. Where as in a jail on the same box, xxd does exist: root@primus:/usr/ports # which vim xxd vim: Command not found. xxd: Command not found. root@primus:/usr/ports # jexec 1 which vim xxd /usr/local/bin/vim /usr/local/bin/xxd
casca 13y ago
TL;DR: man ascii, cal, xxd, ssh, mdfind
- deleted 13y ago
  [deleted]
  - js2 13y ago
    It's too bad that Linux and OS X don't ship with a very useful intro man page. Here's what SunOS's used to look like: http://www.manpages.info/sunos/intro.1.html http://www.manpages.info/sunos/intro.1.html Note the list of commands. I guess you could ls /usr/bin and just start reading their man pages.
  - AlecSchueler 13y ago
    `man 7 charsets` is a great read which recommends checking out the pages for console(4), console_codes(4), console_ioctl(4), ascii(7), iso_8859-1(7), unicode(7) and utf-8(7).
  - casca 13y ago
    Thanks, updated.
- chrisdotcode 13y ago
  `mdfind' in non-Mac UNIXs is spelled `locate' (and the corresponding `updatedb' updates the indexing db, obviously (although you don't have to do this often, as it runs as a cron job)).
  - Camillo 13y ago
    mdfind is incomparably more powerful than locate.
- mkramlich 13y ago
  and for readers age 30-40+: Summary: man ascii, cal, xxd, ssh, mdfind :-)
mpyne 13y ago
I believe xxd is actually part of vim and not a Unix command itself. It's used to support hex-editing binary files, but you can see why it's useful in other roles as well.
- D9u 13y ago
  The origins of xxd seem to be a bit murky, as I've seen others say the same thing about xxd being available only if vim is installed, but I've never seen a Unix system that didn't have vi(m) included. $ man xxd | grep vi Use xxd as a filter within an editor such as vim(1) to hexdump a region Use xxd as a filter within an editor such as vim(1) to recover a binary Use xxd as a filter within an editor such as vim(1) to recover one line http://grail.cba.csuohio.edu/~somos/xxd.c http://grail.cba.csuohio.edu/~somos/xxd.c From what I've been able to find, vim was released in 1991, and xxd has been around since at least 1990, whereas vi was written by Bill Joy in 1976. (according to Wikipedia) Interesting bit of trivia, perhaps someone with a good memory can clear it up for us?
  - clarry 13y ago
    There are no hits for xxd on the Unix Tree (http://minnie.tuhs.org/cgi-bin/utree.pl http://minnie.tuhs.org/cgi-bin/utree.pl) so it's definitely not a traditional Unix tool. I'm sitting in front of a BSD system which includes nvi. I don't have xxd.
  - Amadan 13y ago
    Well, at least on Debian it's in `vim-common` package. http://packages.debian.org/search?searchon=contents&keywords=xxd&mode=exactfilename&suite=stable&arch=any http://packages.debian.org/search?searchon=contents&keywords...
  - vidarh 13y ago
    There was a time when I, annoyed at being thrown into vi(m) whenever something spawned an editor, rather than setting the EDITOR environment variable on login, removed vi and replaced it with symlinks to emacs on the servers I managed. I'm a bit more tolerant of vi now, but just because I "never" see it courtesy of suitable EDITOR entries.
rabino 13y ago
I recently discovered whatis and I love it.
- peterwwillis 13y ago
  Yup, it's a great way to review a lot of commands without reading every man page. Want to find all the tools on your system that use the curses library (for user-friendly interfaces)? Try this: for i in `\ls /bin/* /sbin/* /usr/bin/* /usr/sbin/* | sort -u` ; do ldd $i 2>/dev/null | grep -q curses && whatis `basename $i` ; done | grep -v 'nothing appropriate' Don't know how your current path finds a program? Use which: pwillis@zippy:~$ which df /usr/bin/df pwillis@zippy:~$ which mount /bin/mount Annoyed that /sbin and /usr/sbin aren't in your path? Use whereis: pwillis@zippy:~$ which ifconfig which: no ifconfig in (/usr/local/bin:/usr/bin:/bin:/usr/games:/usr/lib64/kde4/libexec:/usr/kerberos/bin:/usr/lib64/java/bin:/usr/lib64/java/jre/bin:/usr/lib64/qt/bin:/usr/share/texmf/bin:.) pwillis@zippy:~$ whereis ifconfig ifconfig: /sbin/ifconfig /usr/man/man8/ifconfig.8.gz /usr/share/man/man8/ifconfig.8.gz /usr/X11/man/man8/ifconfig.8.gz
  - jlgreco 13y ago
    The zsh builtin 'where' is pretty good too: > where ls ls: aliased to ls --color=tty /usr/bin/ls /bin/ls
  - Wicher 13y ago
    Apropos 'which', let me show 'type -a': % type -a which which is a shell builtin which is /usr/bin/which Convenient if you really want to know what you'll be calling.
hannibal5 13y ago
Unix and Common Lisp are similar in many ways: 1. Both are operating systems 2. Both are conceptually simple. 3. There is always nice command that does what you want but you somehow forget it. I can't believe that Common Lisp has just 900+ symbols but I routinely forget some of them when programming.
- samatman 13y ago
  Indeed! Apropos is our friend.
  - qu4z-2 13y ago
    Or if apropos isn't on your system, man -k does the same thing. ... I think it works when apropos isn't installed. Possibly it just calls out to it.
- Keyframe 13y ago
  My favorite part is when you implement something you could've had running in a minute or so, but instead developed it for hours or days and it's still not as good as the existing solution. I often get that with C and libraries. There has to be a better way to search through existing stuff which is good and proven, not just for C of course.
  - philsnow 13y ago
    Haskell has hoogle [0], that lets you search by type signature. If you already know roughly what kind of function you want, it's a lot easier to sift through a tiny handful of (or even a single) functions that match a given type signature. ISTR Forth had something similar, or maybe I'm just thinking that in Forth, words are almost uniquely defined by their stack annotations. [0] http://www.haskell.org/hoogle/ http://www.haskell.org/hoogle/
grimgrin 13y ago
Replaces the current day of the month with []: $ cal | sed "s/.*/ & /;s/ $(date +%e) / [] /" September 2013 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 [] 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
- res0nat0r 13y ago
  Gcal will highlight the current date for you. http://www.gnu.org/software/gcal/ http://www.gnu.org/software/gcal/
- nawitus 13y ago
  Hmm, the day is already highlighted for me (inverted color).
  - ape4 13y ago
    You can turn that off with: cal | cat
    - aeon10 13y ago
      can you please explain how this works?
      - shabble 13y ago
        it's quite a common pattern in commandline apps to check if the stdout is a tty or not, and do things differently if so. So when you pipe it through `cat', it does the no-fancy-included output, much like the colouring that `ls' will apply interactively, but not when piped/redirected.
        ajuc 13y ago
        It's important for grepping and sedding - the colors are escape codes and would mess with your regular expressions otherwise (you want to find foobar , but bar was blue so you got foo\e[1;34mbar\e[0m instead, and it doesn't match with foobar).
acheron 13y ago
ctrl-R in bash was mine.
- claudius 13y ago
  Try C-x C-e – especially helpful if you want to paste stuff from websites.
- acomjean 13y ago
  nice! did not know that.
- reginaldjcooper 13y ago
  the last time one of these came up I learned about bash's vi mode: set -o vi it's awesome
fsckin 13y ago
One of the more useful bits of ssh is not mentioned: remotely running commands. Example: ssh username@host "echo $HOSTNAME && sudo somecommand && cat somecommand.log" There's probably a better way to do this, but in a pinch I can fix a problem on dozens of machines just by altering the host string.
- Kurtz79 13y ago
  Still about ssh : "typing a password on your local machine once, then using a secure identity to login to several remote machines without having to retype your password by using an ssh key agent. This is awesome. " Sounds awesome, but I really didn't understand what he means. Is he talking about using a certificate (-i option) to log in ?
  - oftenwrong 13y ago
    I am fairly sure they are referring to ssh-add
  - laumars 13y ago
    Yeah, you have a cert set up and put the public cert in the identities file for each server you want to connect to. Then just SSH into each box with agent forwarding ('-a' I think, but you're best to check that). So you'll need to create an SSH cert first: ssh-keygen -b 4096 -t rsa -C "$USER created on $(hostname)" -f $USER.key Keep the private key, that's essential you keep this safe as it's literally your key to access any servers you do the next step for. On each server SSH, append your public key to the of /home/$USER/.ssh/authorized_keys (creating the file if it doesn't exist). It must be the public key! There'll be better guides online for this stuff so I suggest you have a read through them before blindly pasting the stuff I've posted into your terminal. But it's all quite simple stuff once you've grasped the difference between a public and private key (if you weren't already aware - you may well be)
    - larrys 13y ago
      Keeping in mind that the ssh agent is root accessible on the machine in the /tmp directory looking something like this: "ssh-ebT23030" Meaning that anyone with root access who know the other machines that someone is accessing as well as their username can access those machines. I've seen cases where in abnormally terminated sessions that file stays around instead of getting deleted when the session ends.
PuercoPop 13y ago
Protip, you can make emacs use mdfind with (setq locate-command "mdfind")[1] [1]: http://emacsredux.com/blog/2013/07/05/locate/ http://emacsredux.com/blog/2013/07/05/locate/ (setq locate-command "mdfind")
WestCoastJustin 13y ago
If anyone is interested there were several great posts on Hacker News a while about about useful UNIX commands [1, 2, 3]. I have also created several screencasts about command commands like the following [4], and one about the Filesystem Hierarchy Standard [5]. ls man pwd cd top ps df du cp mv rm mkdir rmdir less cat vi [1] https://news.ycombinator.com/item?id=6046682 https://news.ycombinator.com/item?id=6046682 [2] https://news.ycombinator.com/item?id=5022457 https://news.ycombinator.com/item?id=5022457 [3] https://news.ycombinator.com/item?id=4985393 https://news.ycombinator.com/item?id=4985393 [4] http://sysadmincasts.com/episodes/13-crash-course-on-common-commands http://sysadmincasts.com/episodes/13-crash-course-on-common-... [5] http://sysadmincasts.com/episodes/12-crash-course-on-the-filesystem-hierarchy-standard http://sysadmincasts.com/episodes/12-crash-course-on-the-fil...
- jlgreco 13y ago
  You should do a screencast of such infinitely useful classics as tac, tsort, and sl. (Actually, I have found use for those first two on occasion. ;)
  - WestCoastJustin 13y ago
    Thanks for the suggestion! Actually, I initially wanted to do one on more advanced commands, but I thought I'd better lay a foundation first. I also want to cover using pipe (|) to chain these commands together. I'll make sure to include these.
    - nileshtrivedi 13y ago
      Don't miss xargs.
  - jfb 13y ago
    tsort is one of my favorites, mostly because it's totally pointless until you need it and then it works exactly right. I love it.
    - 13y ago
simonreed 13y ago
man units
Kurtz79 13y ago
I know it's not exactly an unknown command, but I didn't know about "sort" until last week. It's freaking fast and convenient, sorts hadoop reduce results like a champ.
- danieldk 13y ago
  Wow, I use it (almost) daily. Especially useful with the '-u' flag or uniq.
- WestCoastJustin 13y ago
  One of my favorite things about unix is that you can chain commands together with pipe (|). Say for example that you have a example data file that looks like this: # Deflection Col-Force Beam-Force 0.000 0 0 0.001 104 51 0.002 202 101 0.003 298 148 0.0031 104 149 0.004 289 201 0.0041 291 209 0.005 104 250 0.010 311 260 0.020 104 240 You can chain commands together to quickly get ball park figures. I do this all the time when reviewing log data to get simple counts and look for anomalies. $ cat data.txt | grep -v '#' | awk '{print $2}' | sort | uniq -c 1 0 4 104 1 202 1 289 1 291 1 298 1 311 Let me break this down a little. I cat data.txt, which just prints the contents to the console, I use grep to remove the # header, then use awk to print the second data column, the sort the values for my next procedure, and lastly I use uniq -c, which takes the input and counts all the unique values. So you can see that "4 104" means that 4 instances of 104 were found. I use this often to look at httpd logs for IP addresses or to review sudo commands, etc. Very handy! ps. there are optimizations that could be done here, but that what is great about unix and "one function" utilities. There are many ways to the same destination!
  - oftenwrong 13y ago
    Yes, there are some ways to simplify. You can get rid of the "UUOC". Instead of cat data.txt | grep -v '#' you can do: grep -v '#' data.txt or you can simplify further by moving the grepping into awk, so that the full command becomes awk '!/#/{print $2}' data.txt | sort -n | uniq -c sort -n is good to use for sorting numbers. You could probably pull the sort | uniq -c into awk somehow, but I have already demonstrated the full extent of my awk knowledge. One of these days I will get around to actually learning it.
rfatnabayeff 13y ago
There are some other useful commands for organizing the stream of datas for pretty printing: column - columnify the incoming stream into columns pr - set up incoming stream for pretty printing including columnification along or across the screen tr - sanitize input, collapse several delimiters into one
- epsylon 13y ago
  I use tr all the time when I want to check $PATH: echo $PATH | tr : '\n'
joshcorbin 13y ago
One of my favorites: column -t Examples include: $ df -h | column -t $ column -t -s: /etc/passwd
sebnukem2 13y ago
How can one use a Unix/Linux system without knowing SSH? How to login remotely without it?
- codegeek 13y ago
  you can use telnet as an alternative even though it is not secure like ssh.
- jfb 13y ago
  There are a large percentage of people who first encounter unix on single-interactive-user machines.
- Toenex 13y ago
  Wow, now I feel old. I thought everyone got started with SunOS on a Sparcstation 2? Seriously, don't most people encounter it either through installing a Linux distribution or firing up a terminal on OSX? Or is even my-first-Unix a virtual experience?
- Nick_C 13y ago
  There are free unix shells out there, google will find them. Don't expect anything fancy, but you will be able to try all the commands talked about here. To connect to the shell from Windows, you can use the Putty program (google for it), although the shell documentation will almost certainly tell you how to do it.
codegeek 13y ago
I love "tail -f" to see realtime updates on a file that I am writing to!!
- shawkinaw 13y ago
  If you don't want to clog up your terminal, try 'less' and then hit 'F' to follow.
  - zipperhead 13y ago
    Better yet use 'less +F'. It's as easy to remember as tail -f. The advantage of using less here is that you can easily interrupt the tailing (ctrl-c). For example, to search. Then just hit 'F' again to restart tailing. But now your search pattern will be highlighted as the data streams by.
    - reuven 13y ago
      I have been using both "tail -f" and less for years, and never knew about this. Thank you. In three lines of text, you have made my logfile-centric life oh-so-much easier to deal with.
    - jghrng 13y ago
      I'm using Ctrl+S and Ctrl+Q for interrupting tail -f, but that's missing the search feature of course.
shawkinaw 13y ago
I really love pushd/popd.
- clarry 13y ago
  For OpenBSD ksh I've got this hack alias d='[ ${#DIRSTACK[@]} -gt 0 ] && for i in `jot ${#DIRSTACK[@]} 1 ${#DIRSTACK[@]} 1`; do echo ${DIRSTACK[$i-1]}; done; echo $PWD' alias popd='[ ${#DIRSTACK[@]} -gt 0 ] && { cd "${DIRSTACK[${#DIRSTACK[@]}-1]}"; unset DIRSTACK[${#DIRSTACK[@]}-1]; }; d' alias pushd='DIRSTACK[${#DIRSTACK[@]}]="$PWD"; cd'
- TomAnthony 13y ago
  Ok, that is pretty freaking awesome. Thanks. For others who don't know it: http://en.wikipedia.org/wiki/Pushd_and_popd http://en.wikipedia.org/wiki/Pushd_and_popd. It essentially is a stack for the working directory, so you can get back to where you were previously quickly and easily.
- guard-of-terra 13y ago
  zsh has AUTO_PUSHD, guess what it does.
  - mafro 13y ago
    jump just made it into oh-my-zsh: http://jeroenjanssens.com/2013/08/16/quickly-navigate-your-filesystem-from-the-command-line.html http://jeroenjanssens.com/2013/08/16/quickly-navigate-your-f...
thejosh 13y ago
One of my favourite traceroute replacements, that is installed by default on Ubuntu: mtr !
salgernon 13y ago
Mac OS: pbpaste & pbcopy For instance: pbpaste | fgrep -i "`pbpaste -pboard find`" To search the copy clipboard with the find clipboard.
- greyfade 13y ago
  On X11 systems, there's also xclip: xclip -o -selection clipboard
  - deathanatos 13y ago
    There's also xsel, which seems to be installed by default more often than xclip. … | xsel -b # Copy. xsel -b | … # Paste.
- dredmorbius 13y ago
  I first ran into clipboard copy/paste on Mac. Discovered "xclip" on Linux. Aliased copy to 'xc' and paste to 'xp', later added 'xpc' to paste from the secondary (application) clipboard. Makes all sorts of hackery possible.
- FigBug 13y ago
  In 2005 I wrote similar utilities for windows called ccopy, ccut & ppaste. I recently dug them up and put them on GitHub. https://github.com/FigBug/ccopyppaste https://github.com/FigBug/ccopyppaste
bndr 13y ago
I recently discovered ncdu (http://dev.yorhel.nl/ncdu/scr http://dev.yorhel.nl/ncdu/scr) as an alternative to du. Quite handy.
- webreac 13y ago
  baobab ?
  - jlgreco 13y ago
    baobab is nice when you have X around, but ncdu has allowed me to get back up and running on more than one occasion.
deleted 13y ago
[deleted]
why-el 13y ago
`cd -` to go back to the previous directory. Saved me LOTS of time over the past few months.
- bizarref00l 13y ago
  pushd and popd are very useful too.
- nja 13y ago
  Wow. Thanks for this. I always cursed myself for forgetting to pushd, and now it turns out I didn't need to!
  - jamie_ca 13y ago
    I recently found out that zsh lets you magically popd as well, as long as you're cding deeper into the tree. If you jump to a different subtree it will pop its internal stack until it finds a parent of the target dir.
- kibwen 13y ago
  And `git checkout -` to checkout the previously checked-out branch.
tptacek 13y ago
Do yourself a favor: unzip the source for ascii.7 and change the order of the tables: hex, then decimal, then octal. You'll thank me later.
- jlgreco 13y ago
  I kind of like how the hex and char columns are next to each other though. Makes it easier to read those two columns together quickly.
ben0x539 13y ago
bash(?) thing I wish I'd discovered years earlier: diff <(command1) <(command2) instead of command1 > file1; command2 > file2; diff file1 file2; rm file1 file2
- gnaritas 13y ago
  Yup, cool use of named pipes.
  - tripa 13y ago
    Named pipes are filesystem objects you create with mkfifo. What's in use here is more widely known as “process substitution”.
    - gnaritas 13y ago
      Which works by creating two named pipes and feeding them to diff and pumping your processes data into them. Process substitution is built on named pipes, they are the enabling mechanism that makes it work. Try this in a terminal: echo <(true) And you'll see the anonymous named pipe created by the subshell and passed to echo.
dredmorbius 13y ago
Rather than 'man 7 ascii', there's the 'ascii' command itself. It will also provide encodings for single characters: $ ascii a ASCII 6/1 is decimal 097, hex 61, octal 141, bits 01100001: prints as `a' Official name: Miniscule a Other names: Small a, Lowercase a ASCII 0/10 is decimal 010, hex 0a, octal 012, bits 00001010: called ^J, LF, NL Official name: Line Feed Other names: Newline, \n
- clarry 13y ago
  Which Unix includes that command?
  - dredmorbius 13y ago
    Debian GNU/Linux. It's the 'ascii' package.
tmcb 13y ago
I'm quite impressed no one mentioned 'fg', 'bg', and 'jobs'. Since I got used to them, the number of open ptys in my screen dropped drastically.
talles 13y ago
Didn't know about the ascii one.
vectorpush 13y ago
Command line globbing! For the uninitiated: Let's say my pwd is `/home/projects` and i want to edit `/home/projects/a_huge_sprawling_app/940j_394/lol/flight_controller.rb` `vim **/flight_controller.rb` opens our flight_controller.rb straight away. In terms of effort, this allows you to basically omit a `find -name` when you're in a rush to edit some damn file where the hell is it again damn we need to reevaluate this directory str... ---- Double bang to re-use your last terminal entry. One great use, taking all the pain out of forgetting your sudos `rm -rf /var/log` rm: cannot remove `/var/log': Permission denied `sudo !!` All evidence of wrongdoing is now destroyed. ----- Ok, here is an awesome one for users of the ultimate cloud IDE: Vim. In your local ~/.ssh/config: `ForwardX11 yes` `SendEnv WINDOWID` In your remote server's /etc/ssh/sshd_config: `AcceptEnv WINDOWID` In your vimrc: `set clipboard=unnamedplus` This has the effect of seamless yank and paste between local and remote vim sessions, no need for ctrl+shift+v. I love linux.
- ilikepi 13y ago
  FWIW, the double-star recursive globbing was added in Bash 4.0. My reading of the manual[1] suggests it is disabled by default, and can be enabled by setting the 'globstar' shell option. I believe it's enabled by default in zsh. [1]: https://www.gnu.org/software/bash/manual/bash.html#The-Shopt-Builtin https://www.gnu.org/software/bash/manual/bash.html#The-Shopt...
  - vectorpush 13y ago
    Thanks. Sometimes I forget that zsh is not what everyone uses. I should add zsh to the list, hah!
  - Typhon 13y ago
    I just tried it on bash 4.2.25, and it worked, even though I just learned about it. Maybe not every OS enables the same default options for Bash (I'm using Lubuntu 12.10 at the moment)
    - ilikepi 13y ago
ja27 13y ago
xargs. Can't tell you how many times I wrote quickie C programs and awk scripts to do what xargs does. http://sidvind.com/wiki/Xargs_by_example http://sidvind.com/wiki/Xargs_by_example
- marcosdumay 13y ago
  And find, that's a nice alternative to xargs when your args are a huge set of files.
  - penland 13y ago
    Yeah, this is one of my favorite tricks as well . . .
  - smagch 13y ago
    You can also use parallel instead of xargs when files are huge.
keithpeter 13y ago
history | grep <keyword> I use this for <keyword> rsync or rdesktop when I have forgotten the long list of options & directories etc I also use ctrl R <keyword> at the bash prompt but that only gives me the most recent match.
- nja 13y ago
  Actually, after hitting C-R <keyword>, you can keep pressing C-R to search backwards through the history. Or C-S to search forwards. More info: (bash uses emacs editing mode by default) http://www.gnu.org/software/emacs/manual/html_node/emacs/Basic-Isearch.html http://www.gnu.org/software/emacs/manual/html_node/emacs/Bas...
- pbhjpbhj 13y ago
  >I also use ctrl R <keyword> at the bash prompt but that only gives me the most recent match. // Are you sure, I'm using BASH on Kubuntu and I get previous matches by repeated use of ctrl+R. Actually inspired by a previous post here I made a script "hs" that searches archived history files and shows me all past match lines. Before that I did what you do now.
  - keithpeter 13y ago
    Indeed it does! Excellent
general_failure 13y ago
my favorite is apt-file
- Toenex 13y ago
  Thank you.
dagw 13y ago
GNU Parallel. My new favorite command. I have probably written at least two or three ad-hoc, buggy, feature poor versions of that command before discovering it.
kibwen 13y ago
I recently used xxd to illustrate the concept of text encoding, and put together a little vimscript to make it easier to visualize: "Toggle hex edit mode nmap <Leader>h :call ToggleHex()<CR> let g:hex_mode_on = 0 function! ToggleHex() if g:hex_mode_on execute "%!xxd -r" let g:hex_mode_on = 0 else execute "%!xxd" let g:hex_mode_on = 1 endif endfunction Stick this in your .vimrc, type something, use \h to convert it to hex, change a value, then \h to convert it back and observe how the text has changed. Not super useful, but a neat party trick.
wpietri 13y ago
A personal favorite: the file command. Through deep magic(5), it can describe the content of a lot of files.
jbaiter 13y ago
A great one I found out about recently is 'watch'. Periodically calls a command and refreshes the screen with its stdout.
csense 13y ago
My favorite is the parallel jobs feature of xargs. For example, say you want to run a script you wrote called process-video.sh to do some processing on all the video files in a directory (extracting audio to MP3, converting format, etc.). You want to use all 8 of your cores. You could write a Makefile and run it with -j9, or you can do this: find . -name "*.flv" | xargs -n 1 -P 9 ./process-video.sh This immediately forks 9 instances of process-video.sh on the first 9 .flv files in the current directory, then starts a new instance whenever a running instance completes, so 9 instances are always in flight. (I usually set to number of cores plus one for CPU-bound tasks, hence 9 for my i7 with eight cores [1].) If you add -print0 to the find command and -0 to the xargs command, it uses null-terminated filenames (which does the right thing when filenames contain whitespace). [1] Logical cores. Most i7's have four physical cores which become eight logical cores through the magic of hyperthreading.
- thret 13y ago
  May I ask, what do you gain by setting it to +1 cores?
  - csense 13y ago
    Even though CPU is usually the resource that limits throughput for video processing in my experience, each process will presumably do some amount of I/O as well. If you have 8 cores running 8 jobs, then whenever one of those jobs needs to do I/O you have a core sitting idle while that job waits for the disk. If you have 8 cores running 9 jobs, then all your cores will still be fully utilized when a single job is doing I/O. I think you actually want to do plus a small percentage, perhaps 5%, so ceil(1.05*N) jobs on N cores. That is, I have a gut feeling that 64 cores doing 65 jobs would still result in underutilization, 67-68 jobs would be better (provided the workload doesn't become I/O-limited with that much CPU power, and provided the number of videos to be converted is still much larger than the number of cores, so you don't run afoul of Amdahl's law). These are really just rules of thumb based on my gut feelings and mental models of how the system works; it might be fun to actually benchmark it and see if my ideas correspond to reality. (You probably want to reboot first, or a bunch of unrelated I/O, to flush the disk cache.)
social_monad 13y ago
look at: man utf8 man syscalls ... and the best hint imo to process malformed filenames per 'find -print0' is: find ... -print0 | { while read -d $'\0' fil ... # use variable fil } Regards
guard-of-terra 13y ago
xmlstarlet is a very nice obscure utility which you will love if you touch moderate amounts of XML.
- gnaritas 13y ago
  Yes it is!
radioxid 13y ago
xxd: it was "made you look\n"
helloTree 13y ago
My biggest improvement was about finding out about .inputrc and configuring the Bash to use VIM-keybindings which is pretty handy if you are used to the editor. Also the following mapping from ESC to pressing jf via imap jf <ESC> was very nice as I find it much more ergonomic. And to use it in Bash I have set editing-mode vi set keymap vi $if mode=vi set keymap vi-insert "jf": vi-movement-mode $endif set show-all-if-ambiguous on
- aikinai 13y ago
  I have my Bash set to vi keybindings, but I had no idea it was possible to remap them. Now I can finally stop hitting escape in Bash too. Thank you so much!
  - helloTree 13y ago
    Yeah it is great if you can use similar commands across applications. Btw. there is also a nice history inside VIM (implemented as a separate buffer) that can be accessed using q: so you can scroll through the history with jfkl. E.g. I use that for accessing files: If you are working on a project you will probably edit similar files. If you use one "programming" vim sessions where you always edit files via :tabe you will find all the recent edited files in your history and using q:/ you can search for them pretty fast. So over time you have automatically a working environment adapted to your current project.
malkia 13y ago
I have the bad habit of trying out everything in /usr/bin (linux, OX) and in c:\windows\system32 (and further on, digging Program Files / Applications / etc.). This is how I discovered myself "fsutil" (Windows) :)
wildmXranat 13y ago
Less +F with search highlighting mentioned in comments is my new fave command. I was beholden to 'tail -f', not that it sucked.
mpu 13y ago
ssh, come on...
gajomi 13y ago
Glad to learn of cal!
cheese1756 13y ago
For the ssh section, I'd also include sshfs. It's super handy when you want to use local (or GUI) applications for a file on a server, or just to mount your home computer as a drive while you are away. I personally use it to mount my backup server's drive on my Chromebook, effectively giving myself a 500GB hard drive whenever I need it.
DavidWanjiru 13y ago
I'm feeling rather good about myself because although I'm a complete novice at linux, I knew the cal command. Yay me! I read about it here: http://linuxcommand.org/tlcl.php http://linuxcommand.org/tlcl.php
Kiro 13y ago
When do you need man ascii and xxd? I'm a front-end developer so I have no clue about these low-level things but I'm interested and would like to learn.
- F30 13y ago
  There are a lot of usecases for bot of them, mostly debugging I's say. So I'm gonna tell you what I used them for in the past months: The ASCII table might be useful when you're dealing with encoding problems and want to know e.g. what exactly happened in a broken URL encoding. I used both of them for binary exploitation in the security class I took at university: When utilizing buffer overflows, it's important to get every byte at the right place of your payload. Sometimes, you also need to treat it as a string and that's where ASCII comes in. These are only two specific usecases. As I said, there are a lot more. Development of binary file parsers could be another.
- cmhamill 13y ago
  This is a very silly use, but I use `ascii` when I can't distinguish between 0 and O in a font ? `ascii 0`, for example, clears that right up.
doktorn 13y ago
strings - find the printable strings in a object, or other binary, file. Good command to know of if you want to snoop around in binary files.
ngcazz 13y ago
Brace expansion to create a backup: cp {file,.bak} reduces to cp file file.bak
- qu4z-2 13y ago
  Brace expansion is great! My understanding, though, is that you'd need to write: cp file{,.bak} Was it a typo, or is there some feature/special-case I don't know about?
  - ngcazz 13y ago
    Oh, you're right. Thanks. (Can't edit the original comment anymore)
ruok0101 13y ago
One little known, but very handy tip: At any password prompt, if you knowingly mis-keyed your password, rather than pressing return, getting the "your an idiot" message (often after an annoying 1 - 2 second delay), and having to retype the command, you can actually press ctrl-U, which is the "Clear the line" bash command, and start over typing your password again.
- rockybutler 13y ago
  For those of us using emacs-style bash keybinds. For vi-style keybinds, it's something more like "esc dd". PS if you are a Vim user, you owe it to yourself to try the vim-style keybinds. They make a world of difference.
mncolinlee 13y ago
Along with the SSH recommendations, I would include some of the great helpers for ssh agent like keychain. https://wiki.gentoo.org/wiki/Keychain https://wiki.gentoo.org/wiki/Keychain

480 points by micahalles 13y ago

I'm always surprised at the lack of general knowledge of Regular Expressions, so egrep would be a huge one for newcomers.

foobeer 13y ago

Never knew about the "cal" command. I especially like that you can quickly look up a previous or future month/year. >> cal 3 1973 quickly shows a calendar of March 1973

yogo 13y ago
Aliasing to 'cal -3' can also be good if you want a little continuity.
- xrt 13y ago
  The -3 arg is not available on OS X. Odd, since it's been in BSD Unix since forever. http://www.freebsd.org/cgi/man.cgi?query=cal http://www.freebsd.org/cgi/man.cgi?query=cal
  - clarry 13y ago
    It's hardly ubiquitous. NetBSD has had it for 11 years. FreeBSD has had it for three years and six months. DragonFly doesn't have it, OpenBSD doesn't have it... http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/cal/cal.c?rev=1.12&content-type=text/x-cvsweb-markup&only_with_tag=MAIN http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/cal/cal.c?re... http://svnweb.freebsd.org/base?view=revision&revision=204697 http://svnweb.freebsd.org/base?view=revision&revision=204697
Shish2k 13y ago
you can even look up `cal sept 1752`
- quchen 13y ago
  Funny, 3-13 are missing here. > cal sep 1752 September 1752 Su Mo Tu We Th Fr Sa 1 2 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Edit: Oh, colour me surprised. https://en.wikipedia.org/wiki/September_14 https://en.wikipedia.org/wiki/September_14
  - ghotli 13y ago
    This was the best thing I learned today
mturmon 13y ago
Right, you can use this to easily find the number of days in an arbitrary month in a script: % cal 2 1972 | tail +3 | wc -w 29
- parennoob 13y ago
  tail: cannot open +3 for reading: No such file or directory What am I doing wrong? This worked like a charm though: cal dec 2002 | tail -6 | wc -w (since it always takes up 6 lines, even sep 1752)
  - typicalrunt 13y ago
    It might be system-dependent. I'm on OSX and the command works properly.
  - jbarber 13y ago
    The "+n" syntax is not standards compliant, from the tail info page (on fedora 19): On older systems, the leading '-' can be replaced by '+' in the obsolete option syntax with the same meaning as in counts, and obsolete usage overrides normal usage when the two conflict. This obsolete behavior can be enabled or disabled with the '_POSIX2_VERSION' environment variable (*note Standards conformance::).

happywolf 13y ago

I recommend Quicksilver on mac, free and powerful. It lets you to find a lot of stuff (application, contact, email, etc.) and manipulate them (send, forward, print, etc.) http://qsapp.com/ http://qsapp.com/

jentulman 13y ago
Also in that vein on Mac it's worth having a look at Alfred http://alfredapp.com http://alfredapp.com
- actionscripted 13y ago
  Alread rocks. You can develop workflows for common groups of tasks and it integrates well with iTunes, 1Password, Terminal and other system actions/apps.
  - adamnemecek 13y ago
    It really does. My computer would be like 30% less usable without it. The file search is particularly amazing.
9999 13y ago
I second the Quicksilver recommendation. When you combine Quicksilver triggers (keyboard shortcut available regardless of what application you're in) with applescript, and fabric it's very powerful. I have a trigger that that restarts all of my local services using fabric and opens up new terminal windows running tails of the relevant logs. The Applescript is this if anyone wants to do something similar: tell application "Terminal" do script "tail -F /somepath/some.log" do script "tail -F /anotherpath/another.log" do script "sudo fab -f '/pathto/fabfile.py' local deploy" activate end tell Here's a detailed explanation of setting up Triggers: http://leafraker.com/2007/09/17/how-to-create-a-quicksilver-trigger/ http://leafraker.com/2007/09/17/how-to-create-a-quicksilver-...

ck2 13y ago

I want a command that counts files in a tree like du does sizes but without having to pipe find through wc which is crazy for hundred thousand files. Can't they just directly access inodes for high speed counting?

casca 13y ago
Counting number of files is probably not a sufficiently frequently used case to justify a new standard command. What part of piping concerns you?
- sp332 13y ago
  It can be really slow, even for a few tens of thousands of tiles. I remember watching Nautilus's "properties" panel slowly count up the number of files and directories.
  - jonhohle 13y ago
    I'm not sure what this has to do with piping (why would nautilus pipe the output of find when doing a directory listing?), but I doubt it's the pipe that's causing performance issues. Uncached file system access has a tendency to be slow[0]. 0 - https://gist.github.com/nahurst/3711264 https://gist.github.com/nahurst/3711264
njharman 13y ago
> Can't they just directly access inodes for high speed counting? Don't know what you mean by that exactly. But, I'd be stunned if find is actually reading the actual files rather than just reading the directory entries in directory "files". Which is what I'd call accessing inodes "directly"
bendaid 13y ago
tree does this, don't know about suppressing the actual tree, but it tells you number of files and dirs printed at the end.
- 13y ago

brown9-2 13y ago

I'm unreasonably excited about discovering xxd.

Luyt 13y ago
xxd is on OSX, but not on FreeBSD. Under FreeBSD, I always use 'hd somefile.bin'.
- D9u 13y ago
  I have xxd on my FreeBSD setup. uname -rms FreeBSD 9.1-RELEASE-p19 amd64 Although the xxd man page doesn't have the BSD label, and is from 1996, so I'm not sure if xxd only recently found its way into FBSD. This manual page documents xxd version 1.7 AUTHOR (c) 1990-1997 by Juergen Weigert
  - laumars 13y ago
    It maybe in ports, but it's not part of the base install: $ xxd xxd: Command not found. $ uname -rms FreeBSD 9.1-RELEASE amd64 edit: fixed typo where I put xdd instead of xxd
    - pessimizer 13y ago
      s/xdd/xxd/g
      - laumars 13y ago
        Sorry, that was a typo when posting on here. I did originally run it as xxd: root@primus:/usr/ports # xxd xxd: Command not found. edit: It doesn't show up in my ports either: root@primus:/usr/ports # make search name=xxd Port: textproc/xxdiff Moved: Date: 2013-07-26 Reason: Has expired: Depends on Qt 3.x Port: textproc/xxdiff-scripts Moved: Date: 2013-07-26 Reason: Has expired: Depends on Qt 3.x Obviously xxdiff is a different (if you pardon the pun) utility, but xxd should show up in that list also. edit 2: Another comment on here might explain why you're seeing xxd and I'm not: https://news.ycombinator.com/item?id=6360814 https://news.ycombinator.com/item?id=6360814 (it's not a separate utility, it's part of vim) Since I didn't install vim on that box (just use basic vi), xxd wouldn't have installed as part of it. Where as in a jail on the same box, xxd does exist: root@primus:/usr/ports # which vim xxd vim: Command not found. xxd: Command not found. root@primus:/usr/ports # jexec 1 which vim xxd /usr/local/bin/vim /usr/local/bin/xxd

casca 13y ago

TL;DR: man ascii, cal, xxd, ssh, mdfind

deleted 13y ago
[deleted]
- js2 13y ago
  It's too bad that Linux and OS X don't ship with a very useful intro man page. Here's what SunOS's used to look like: http://www.manpages.info/sunos/intro.1.html http://www.manpages.info/sunos/intro.1.html Note the list of commands. I guess you could ls /usr/bin and just start reading their man pages.
- AlecSchueler 13y ago
  `man 7 charsets` is a great read which recommends checking out the pages for console(4), console_codes(4), console_ioctl(4), ascii(7), iso_8859-1(7), unicode(7) and utf-8(7).
- casca 13y ago
  Thanks, updated.
chrisdotcode 13y ago
`mdfind' in non-Mac UNIXs is spelled `locate' (and the corresponding `updatedb' updates the indexing db, obviously (although you don't have to do this often, as it runs as a cron job)).
- Camillo 13y ago
  mdfind is incomparably more powerful than locate.
mkramlich 13y ago
and for readers age 30-40+: Summary: man ascii, cal, xxd, ssh, mdfind :-)

mpyne 13y ago

I believe xxd is actually part of vim and not a Unix command itself. It's used to support hex-editing binary files, but you can see why it's useful in other roles as well.

D9u 13y ago
The origins of xxd seem to be a bit murky, as I've seen others say the same thing about xxd being available only if vim is installed, but I've never seen a Unix system that didn't have vi(m) included. $ man xxd | grep vi Use xxd as a filter within an editor such as vim(1) to hexdump a region Use xxd as a filter within an editor such as vim(1) to recover a binary Use xxd as a filter within an editor such as vim(1) to recover one line http://grail.cba.csuohio.edu/~somos/xxd.c http://grail.cba.csuohio.edu/~somos/xxd.c From what I've been able to find, vim was released in 1991, and xxd has been around since at least 1990, whereas vi was written by Bill Joy in 1976. (according to Wikipedia) Interesting bit of trivia, perhaps someone with a good memory can clear it up for us?
- clarry 13y ago
  There are no hits for xxd on the Unix Tree (http://minnie.tuhs.org/cgi-bin/utree.pl http://minnie.tuhs.org/cgi-bin/utree.pl) so it's definitely not a traditional Unix tool. I'm sitting in front of a BSD system which includes nvi. I don't have xxd.
- Amadan 13y ago
  Well, at least on Debian it's in `vim-common` package. http://packages.debian.org/search?searchon=contents&keywords=xxd&mode=exactfilename&suite=stable&arch=any http://packages.debian.org/search?searchon=contents&keywords...
- vidarh 13y ago
  There was a time when I, annoyed at being thrown into vi(m) whenever something spawned an editor, rather than setting the EDITOR environment variable on login, removed vi and replaced it with symlinks to emacs on the servers I managed. I'm a bit more tolerant of vi now, but just because I "never" see it courtesy of suitable EDITOR entries.

rabino 13y ago

I recently discovered whatis and I love it.

peterwwillis 13y ago
Yup, it's a great way to review a lot of commands without reading every man page. Want to find all the tools on your system that use the curses library (for user-friendly interfaces)? Try this: for i in `\ls /bin/* /sbin/* /usr/bin/* /usr/sbin/* | sort -u` ; do ldd $i 2>/dev/null | grep -q curses && whatis `basename $i` ; done | grep -v 'nothing appropriate' Don't know how your current path finds a program? Use which: pwillis@zippy:~$ which df /usr/bin/df pwillis@zippy:~$ which mount /bin/mount Annoyed that /sbin and /usr/sbin aren't in your path? Use whereis: pwillis@zippy:~$ which ifconfig which: no ifconfig in (/usr/local/bin:/usr/bin:/bin:/usr/games:/usr/lib64/kde4/libexec:/usr/kerberos/bin:/usr/lib64/java/bin:/usr/lib64/java/jre/bin:/usr/lib64/qt/bin:/usr/share/texmf/bin:.) pwillis@zippy:~$ whereis ifconfig ifconfig: /sbin/ifconfig /usr/man/man8/ifconfig.8.gz /usr/share/man/man8/ifconfig.8.gz /usr/X11/man/man8/ifconfig.8.gz
- jlgreco 13y ago
  The zsh builtin 'where' is pretty good too: > where ls ls: aliased to ls --color=tty /usr/bin/ls /bin/ls
- Wicher 13y ago
  Apropos 'which', let me show 'type -a': % type -a which which is a shell builtin which is /usr/bin/which Convenient if you really want to know what you'll be calling.

hannibal5 13y ago

Unix and Common Lisp are similar in many ways: 1. Both are operating systems 2. Both are conceptually simple. 3. There is always nice command that does what you want but you somehow forget it. I can't believe that Common Lisp has just 900+ symbols but I routinely forget some of them when programming.

samatman 13y ago
Indeed! Apropos is our friend.
- qu4z-2 13y ago
  Or if apropos isn't on your system, man -k does the same thing. ... I think it works when apropos isn't installed. Possibly it just calls out to it.
Keyframe 13y ago
My favorite part is when you implement something you could've had running in a minute or so, but instead developed it for hours or days and it's still not as good as the existing solution. I often get that with C and libraries. There has to be a better way to search through existing stuff which is good and proven, not just for C of course.
- philsnow 13y ago
  Haskell has hoogle [0], that lets you search by type signature. If you already know roughly what kind of function you want, it's a lot easier to sift through a tiny handful of (or even a single) functions that match a given type signature. ISTR Forth had something similar, or maybe I'm just thinking that in Forth, words are almost uniquely defined by their stack annotations. [0] http://www.haskell.org/hoogle/ http://www.haskell.org/hoogle/

grimgrin 13y ago

Replaces the current day of the month with []: $ cal | sed "s/.*/ & /;s/ $(date +%e) / [] /" September 2013 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 [] 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

res0nat0r 13y ago
Gcal will highlight the current date for you. http://www.gnu.org/software/gcal/ http://www.gnu.org/software/gcal/
nawitus 13y ago
Hmm, the day is already highlighted for me (inverted color).
- ape4 13y ago
  You can turn that off with: cal | cat
  - aeon10 13y ago
    can you please explain how this works?
    - shabble 13y ago
      it's quite a common pattern in commandline apps to check if the stdout is a tty or not, and do things differently if so. So when you pipe it through `cat', it does the no-fancy-included output, much like the colouring that `ls' will apply interactively, but not when piped/redirected.
      - ajuc 13y ago
        It's important for grepping and sedding - the colors are escape codes and would mess with your regular expressions otherwise (you want to find foobar , but bar was blue so you got foo\e[1;34mbar\e[0m instead, and it doesn't match with foobar).

acheron 13y ago

ctrl-R in bash was mine.

claudius 13y ago
Try C-x C-e – especially helpful if you want to paste stuff from websites.
acomjean 13y ago
nice! did not know that.
reginaldjcooper 13y ago
the last time one of these came up I learned about bash's vi mode: set -o vi it's awesome

fsckin 13y ago

One of the more useful bits of ssh is not mentioned: remotely running commands. Example: ssh username@host "echo $HOSTNAME && sudo somecommand && cat somecommand.log" There's probably a better way to do this, but in a pinch I can fix a problem on dozens of machines just by altering the host string.

Kurtz79 13y ago
Still about ssh : "typing a password on your local machine once, then using a secure identity to login to several remote machines without having to retype your password by using an ssh key agent. This is awesome. " Sounds awesome, but I really didn't understand what he means. Is he talking about using a certificate (-i option) to log in ?
- oftenwrong 13y ago
  I am fairly sure they are referring to ssh-add
- laumars 13y ago
  Yeah, you have a cert set up and put the public cert in the identities file for each server you want to connect to. Then just SSH into each box with agent forwarding ('-a' I think, but you're best to check that). So you'll need to create an SSH cert first: ssh-keygen -b 4096 -t rsa -C "$USER created on $(hostname)" -f $USER.key Keep the private key, that's essential you keep this safe as it's literally your key to access any servers you do the next step for. On each server SSH, append your public key to the of /home/$USER/.ssh/authorized_keys (creating the file if it doesn't exist). It must be the public key! There'll be better guides online for this stuff so I suggest you have a read through them before blindly pasting the stuff I've posted into your terminal. But it's all quite simple stuff once you've grasped the difference between a public and private key (if you weren't already aware - you may well be)
  - larrys 13y ago
    Keeping in mind that the ssh agent is root accessible on the machine in the /tmp directory looking something like this: "ssh-ebT23030" Meaning that anyone with root access who know the other machines that someone is accessing as well as their username can access those machines. I've seen cases where in abnormally terminated sessions that file stays around instead of getting deleted when the session ends.

PuercoPop 13y ago

Protip, you can make emacs use mdfind with (setq locate-command "mdfind")[1] [1]: http://emacsredux.com/blog/2013/07/05/locate/ http://emacsredux.com/blog/2013/07/05/locate/ (setq locate-command "mdfind")

WestCoastJustin 13y ago

If anyone is interested there were several great posts on Hacker News a while about about useful UNIX commands [1, 2, 3]. I have also created several screencasts about command commands like the following [4], and one about the Filesystem Hierarchy Standard [5]. ls man pwd cd top ps df du cp mv rm mkdir rmdir less cat vi [1] https://news.ycombinator.com/item?id=6046682 https://news.ycombinator.com/item?id=6046682 [2] https://news.ycombinator.com/item?id=5022457 https://news.ycombinator.com/item?id=5022457 [3] https://news.ycombinator.com/item?id=4985393 https://news.ycombinator.com/item?id=4985393 [4] http://sysadmincasts.com/episodes/13-crash-course-on-common-commands http://sysadmincasts.com/episodes/13-crash-course-on-common-... [5] http://sysadmincasts.com/episodes/12-crash-course-on-the-filesystem-hierarchy-standard http://sysadmincasts.com/episodes/12-crash-course-on-the-fil...

jlgreco 13y ago
You should do a screencast of such infinitely useful classics as tac, tsort, and sl. (Actually, I have found use for those first two on occasion. ;)
- WestCoastJustin 13y ago
  Thanks for the suggestion! Actually, I initially wanted to do one on more advanced commands, but I thought I'd better lay a foundation first. I also want to cover using pipe (|) to chain these commands together. I'll make sure to include these.
  - nileshtrivedi 13y ago
    Don't miss xargs.
- jfb 13y ago
  tsort is one of my favorites, mostly because it's totally pointless until you need it and then it works exactly right. I love it.
  - 13y ago

simonreed 13y ago

man units

Kurtz79 13y ago

I know it's not exactly an unknown command, but I didn't know about "sort" until last week. It's freaking fast and convenient, sorts hadoop reduce results like a champ.

danieldk 13y ago
Wow, I use it (almost) daily. Especially useful with the '-u' flag or uniq.
WestCoastJustin 13y ago
One of my favorite things about unix is that you can chain commands together with pipe (|). Say for example that you have a example data file that looks like this: # Deflection Col-Force Beam-Force 0.000 0 0 0.001 104 51 0.002 202 101 0.003 298 148 0.0031 104 149 0.004 289 201 0.0041 291 209 0.005 104 250 0.010 311 260 0.020 104 240 You can chain commands together to quickly get ball park figures. I do this all the time when reviewing log data to get simple counts and look for anomalies. $ cat data.txt | grep -v '#' | awk '{print $2}' | sort | uniq -c 1 0 4 104 1 202 1 289 1 291 1 298 1 311 Let me break this down a little. I cat data.txt, which just prints the contents to the console, I use grep to remove the # header, then use awk to print the second data column, the sort the values for my next procedure, and lastly I use uniq -c, which takes the input and counts all the unique values. So you can see that "4 104" means that 4 instances of 104 were found. I use this often to look at httpd logs for IP addresses or to review sudo commands, etc. Very handy! ps. there are optimizations that could be done here, but that what is great about unix and "one function" utilities. There are many ways to the same destination!
- oftenwrong 13y ago
  Yes, there are some ways to simplify. You can get rid of the "UUOC". Instead of cat data.txt | grep -v '#' you can do: grep -v '#' data.txt or you can simplify further by moving the grepping into awk, so that the full command becomes awk '!/#/{print $2}' data.txt | sort -n | uniq -c sort -n is good to use for sorting numbers. You could probably pull the sort | uniq -c into awk somehow, but I have already demonstrated the full extent of my awk knowledge. One of these days I will get around to actually learning it.

rfatnabayeff 13y ago

There are some other useful commands for organizing the stream of datas for pretty printing: column - columnify the incoming stream into columns pr - set up incoming stream for pretty printing including columnification along or across the screen tr - sanitize input, collapse several delimiters into one

epsylon 13y ago
I use tr all the time when I want to check $PATH: echo $PATH | tr : '\n'

joshcorbin 13y ago

One of my favorites: column -t Examples include: $ df -h | column -t $ column -t -s: /etc/passwd

sebnukem2 13y ago

How can one use a Unix/Linux system without knowing SSH? How to login remotely without it?

codegeek 13y ago
you can use telnet as an alternative even though it is not secure like ssh.
jfb 13y ago
There are a large percentage of people who first encounter unix on single-interactive-user machines.
Toenex 13y ago
Wow, now I feel old. I thought everyone got started with SunOS on a Sparcstation 2? Seriously, don't most people encounter it either through installing a Linux distribution or firing up a terminal on OSX? Or is even my-first-Unix a virtual experience?
Nick_C 13y ago
There are free unix shells out there, google will find them. Don't expect anything fancy, but you will be able to try all the commands talked about here. To connect to the shell from Windows, you can use the Putty program (google for it), although the shell documentation will almost certainly tell you how to do it.

codegeek 13y ago

I love "tail -f" to see realtime updates on a file that I am writing to!!

shawkinaw 13y ago
If you don't want to clog up your terminal, try 'less' and then hit 'F' to follow.
- zipperhead 13y ago
  Better yet use 'less +F'. It's as easy to remember as tail -f. The advantage of using less here is that you can easily interrupt the tailing (ctrl-c). For example, to search. Then just hit 'F' again to restart tailing. But now your search pattern will be highlighted as the data streams by.
  - reuven 13y ago
    I have been using both "tail -f" and less for years, and never knew about this. Thank you. In three lines of text, you have made my logfile-centric life oh-so-much easier to deal with.
  - jghrng 13y ago
    I'm using Ctrl+S and Ctrl+Q for interrupting tail -f, but that's missing the search feature of course.

shawkinaw 13y ago

I really love pushd/popd.

clarry 13y ago
For OpenBSD ksh I've got this hack alias d='[ ${#DIRSTACK[@]} -gt 0 ] && for i in `jot ${#DIRSTACK[@]} 1 ${#DIRSTACK[@]} 1`; do echo ${DIRSTACK[$i-1]}; done; echo $PWD' alias popd='[ ${#DIRSTACK[@]} -gt 0 ] && { cd "${DIRSTACK[${#DIRSTACK[@]}-1]}"; unset DIRSTACK[${#DIRSTACK[@]}-1]; }; d' alias pushd='DIRSTACK[${#DIRSTACK[@]}]="$PWD"; cd'
TomAnthony 13y ago
Ok, that is pretty freaking awesome. Thanks. For others who don't know it: http://en.wikipedia.org/wiki/Pushd_and_popd http://en.wikipedia.org/wiki/Pushd_and_popd. It essentially is a stack for the working directory, so you can get back to where you were previously quickly and easily.
guard-of-terra 13y ago
zsh has AUTO_PUSHD, guess what it does.
- mafro 13y ago
  jump just made it into oh-my-zsh: http://jeroenjanssens.com/2013/08/16/quickly-navigate-your-filesystem-from-the-command-line.html http://jeroenjanssens.com/2013/08/16/quickly-navigate-your-f...

thejosh 13y ago

One of my favourite traceroute replacements, that is installed by default on Ubuntu: mtr !

salgernon 13y ago

Mac OS: pbpaste & pbcopy For instance: pbpaste | fgrep -i "`pbpaste -pboard find`" To search the copy clipboard with the find clipboard.

greyfade 13y ago
On X11 systems, there's also xclip: xclip -o -selection clipboard
- deathanatos 13y ago
  There's also xsel, which seems to be installed by default more often than xclip. … | xsel -b # Copy. xsel -b | … # Paste.
dredmorbius 13y ago
I first ran into clipboard copy/paste on Mac. Discovered "xclip" on Linux. Aliased copy to 'xc' and paste to 'xp', later added 'xpc' to paste from the secondary (application) clipboard. Makes all sorts of hackery possible.
FigBug 13y ago
In 2005 I wrote similar utilities for windows called ccopy, ccut & ppaste. I recently dug them up and put them on GitHub. https://github.com/FigBug/ccopyppaste https://github.com/FigBug/ccopyppaste

bndr 13y ago

I recently discovered ncdu (http://dev.yorhel.nl/ncdu/scr http://dev.yorhel.nl/ncdu/scr) as an alternative to du. Quite handy.

webreac 13y ago
baobab ?
- jlgreco 13y ago
  baobab is nice when you have X around, but ncdu has allowed me to get back up and running on more than one occasion.

deleted 13y ago

[deleted]

why-el 13y ago

`cd -` to go back to the previous directory. Saved me LOTS of time over the past few months.

bizarref00l 13y ago
pushd and popd are very useful too.
nja 13y ago
Wow. Thanks for this. I always cursed myself for forgetting to pushd, and now it turns out I didn't need to!
- jamie_ca 13y ago
  I recently found out that zsh lets you magically popd as well, as long as you're cding deeper into the tree. If you jump to a different subtree it will pop its internal stack until it finds a parent of the target dir.
kibwen 13y ago
And `git checkout -` to checkout the previously checked-out branch.

tptacek 13y ago

Do yourself a favor: unzip the source for ascii.7 and change the order of the tables: hex, then decimal, then octal. You'll thank me later.

jlgreco 13y ago
I kind of like how the hex and char columns are next to each other though. Makes it easier to read those two columns together quickly.

ben0x539 13y ago

bash(?) thing I wish I'd discovered years earlier: diff <(command1) <(command2) instead of command1 > file1; command2 > file2; diff file1 file2; rm file1 file2

gnaritas 13y ago
Yup, cool use of named pipes.
- tripa 13y ago
  Named pipes are filesystem objects you create with mkfifo. What's in use here is more widely known as “process substitution”.
  - gnaritas 13y ago
    Which works by creating two named pipes and feeding them to diff and pumping your processes data into them. Process substitution is built on named pipes, they are the enabling mechanism that makes it work. Try this in a terminal: echo <(true) And you'll see the anonymous named pipe created by the subshell and passed to echo.

dredmorbius 13y ago

Rather than 'man 7 ascii', there's the 'ascii' command itself. It will also provide encodings for single characters: $ ascii a ASCII 6/1 is decimal 097, hex 61, octal 141, bits 01100001: prints as `a' Official name: Miniscule a Other names: Small a, Lowercase a ASCII 0/10 is decimal 010, hex 0a, octal 012, bits 00001010: called ^J, LF, NL Official name: Line Feed Other names: Newline, \n

clarry 13y ago
Which Unix includes that command?
- dredmorbius 13y ago
  Debian GNU/Linux. It's the 'ascii' package.

tmcb 13y ago

I'm quite impressed no one mentioned 'fg', 'bg', and 'jobs'. Since I got used to them, the number of open ptys in my screen dropped drastically.

talles 13y ago

Didn't know about the ascii one.

vectorpush 13y ago

Command line globbing! For the uninitiated: Let's say my pwd is `/home/projects` and i want to edit `/home/projects/a_huge_sprawling_app/940j_394/lol/flight_controller.rb` `vim **/flight_controller.rb` opens our flight_controller.rb straight away. In terms of effort, this allows you to basically omit a `find -name` when you're in a rush to edit some damn file where the hell is it again damn we need to reevaluate this directory str... ---- Double bang to re-use your last terminal entry. One great use, taking all the pain out of forgetting your sudos `rm -rf /var/log` rm: cannot remove `/var/log': Permission denied `sudo !!` All evidence of wrongdoing is now destroyed. ----- Ok, here is an awesome one for users of the ultimate cloud IDE: Vim. In your local ~/.ssh/config: `ForwardX11 yes` `SendEnv WINDOWID` In your remote server's /etc/ssh/sshd_config: `AcceptEnv WINDOWID` In your vimrc: `set clipboard=unnamedplus` This has the effect of seamless yank and paste between local and remote vim sessions, no need for ctrl+shift+v. I love linux.

ilikepi 13y ago
FWIW, the double-star recursive globbing was added in Bash 4.0. My reading of the manual[1] suggests it is disabled by default, and can be enabled by setting the 'globstar' shell option. I believe it's enabled by default in zsh. [1]: https://www.gnu.org/software/bash/manual/bash.html#The-Shopt-Builtin https://www.gnu.org/software/bash/manual/bash.html#The-Shopt...
- vectorpush 13y ago
  Thanks. Sometimes I forget that zsh is not what everyone uses. I should add zsh to the list, hah!
- Typhon 13y ago
  I just tried it on bash 4.2.25, and it worked, even though I just learned about it. Maybe not every OS enables the same default options for Bash (I'm using Lubuntu 12.10 at the moment)
  - ilikepi 13y ago

ja27 13y ago

xargs. Can't tell you how many times I wrote quickie C programs and awk scripts to do what xargs does. http://sidvind.com/wiki/Xargs_by_example http://sidvind.com/wiki/Xargs_by_example

marcosdumay 13y ago
And find, that's a nice alternative to xargs when your args are a huge set of files.
- penland 13y ago
  Yeah, this is one of my favorite tricks as well . . .
- smagch 13y ago
  You can also use parallel instead of xargs when files are huge.

keithpeter 13y ago

history | grep <keyword> I use this for <keyword> rsync or rdesktop when I have forgotten the long list of options & directories etc I also use ctrl R <keyword> at the bash prompt but that only gives me the most recent match.

nja 13y ago
Actually, after hitting C-R <keyword>, you can keep pressing C-R to search backwards through the history. Or C-S to search forwards. More info: (bash uses emacs editing mode by default) http://www.gnu.org/software/emacs/manual/html_node/emacs/Basic-Isearch.html http://www.gnu.org/software/emacs/manual/html_node/emacs/Bas...
pbhjpbhj 13y ago
>I also use ctrl R <keyword> at the bash prompt but that only gives me the most recent match. // Are you sure, I'm using BASH on Kubuntu and I get previous matches by repeated use of ctrl+R. Actually inspired by a previous post here I made a script "hs" that searches archived history files and shows me all past match lines. Before that I did what you do now.
- keithpeter 13y ago
  Indeed it does! Excellent

general_failure 13y ago

my favorite is apt-file

Toenex 13y ago
Thank you.

dagw 13y ago

GNU Parallel. My new favorite command. I have probably written at least two or three ad-hoc, buggy, feature poor versions of that command before discovering it.

kibwen 13y ago

I recently used xxd to illustrate the concept of text encoding, and put together a little vimscript to make it easier to visualize: "Toggle hex edit mode nmap <Leader>h :call ToggleHex()<CR> let g:hex_mode_on = 0 function! ToggleHex() if g:hex_mode_on execute "%!xxd -r" let g:hex_mode_on = 0 else execute "%!xxd" let g:hex_mode_on = 1 endif endfunction Stick this in your .vimrc, type something, use \h to convert it to hex, change a value, then \h to convert it back and observe how the text has changed. Not super useful, but a neat party trick.

wpietri 13y ago

A personal favorite: the file command. Through deep magic(5), it can describe the content of a lot of files.

jbaiter 13y ago

A great one I found out about recently is 'watch'. Periodically calls a command and refreshes the screen with its stdout.

csense 13y ago

My favorite is the parallel jobs feature of xargs. For example, say you want to run a script you wrote called process-video.sh to do some processing on all the video files in a directory (extracting audio to MP3, converting format, etc.). You want to use all 8 of your cores. You could write a Makefile and run it with -j9, or you can do this: find . -name "*.flv" | xargs -n 1 -P 9 ./process-video.sh This immediately forks 9 instances of process-video.sh on the first 9 .flv files in the current directory, then starts a new instance whenever a running instance completes, so 9 instances are always in flight. (I usually set to number of cores plus one for CPU-bound tasks, hence 9 for my i7 with eight cores [1].) If you add -print0 to the find command and -0 to the xargs command, it uses null-terminated filenames (which does the right thing when filenames contain whitespace). [1] Logical cores. Most i7's have four physical cores which become eight logical cores through the magic of hyperthreading.

thret 13y ago
May I ask, what do you gain by setting it to +1 cores?
- csense 13y ago
  Even though CPU is usually the resource that limits throughput for video processing in my experience, each process will presumably do some amount of I/O as well. If you have 8 cores running 8 jobs, then whenever one of those jobs needs to do I/O you have a core sitting idle while that job waits for the disk. If you have 8 cores running 9 jobs, then all your cores will still be fully utilized when a single job is doing I/O. I think you actually want to do plus a small percentage, perhaps 5%, so ceil(1.05*N) jobs on N cores. That is, I have a gut feeling that 64 cores doing 65 jobs would still result in underutilization, 67-68 jobs would be better (provided the workload doesn't become I/O-limited with that much CPU power, and provided the number of videos to be converted is still much larger than the number of cores, so you don't run afoul of Amdahl's law). These are really just rules of thumb based on my gut feelings and mental models of how the system works; it might be fun to actually benchmark it and see if my ideas correspond to reality. (You probably want to reboot first, or a bunch of unrelated I/O, to flush the disk cache.)

social_monad 13y ago

look at: man utf8 man syscalls ... and the best hint imo to process malformed filenames per 'find -print0' is: find ... -print0 | { while read -d $'\0' fil ... # use variable fil } Regards

guard-of-terra 13y ago

xmlstarlet is a very nice obscure utility which you will love if you touch moderate amounts of XML.

gnaritas 13y ago
Yes it is!

radioxid 13y ago

xxd: it was "made you look\n"

helloTree 13y ago

My biggest improvement was about finding out about .inputrc and configuring the Bash to use VIM-keybindings which is pretty handy if you are used to the editor. Also the following mapping from ESC to pressing jf via imap jf <ESC> was very nice as I find it much more ergonomic. And to use it in Bash I have set editing-mode vi set keymap vi $if mode=vi set keymap vi-insert "jf": vi-movement-mode $endif set show-all-if-ambiguous on

aikinai 13y ago
I have my Bash set to vi keybindings, but I had no idea it was possible to remap them. Now I can finally stop hitting escape in Bash too. Thank you so much!
- helloTree 13y ago
  Yeah it is great if you can use similar commands across applications. Btw. there is also a nice history inside VIM (implemented as a separate buffer) that can be accessed using q: so you can scroll through the history with jfkl. E.g. I use that for accessing files: If you are working on a project you will probably edit similar files. If you use one "programming" vim sessions where you always edit files via :tabe you will find all the recent edited files in your history and using q:/ you can search for them pretty fast. So over time you have automatically a working environment adapted to your current project.

malkia 13y ago

I have the bad habit of trying out everything in /usr/bin (linux, OX) and in c:\windows\system32 (and further on, digging Program Files / Applications / etc.). This is how I discovered myself "fsutil" (Windows) :)

wildmXranat 13y ago

Less +F with search highlighting mentioned in comments is my new fave command. I was beholden to 'tail -f', not that it sucked.

mpu 13y ago

ssh, come on...

gajomi 13y ago

Glad to learn of cal!

cheese1756 13y ago

For the ssh section, I'd also include sshfs. It's super handy when you want to use local (or GUI) applications for a file on a server, or just to mount your home computer as a drive while you are away. I personally use it to mount my backup server's drive on my Chromebook, effectively giving myself a 500GB hard drive whenever I need it.

DavidWanjiru 13y ago

I'm feeling rather good about myself because although I'm a complete novice at linux, I knew the cal command. Yay me! I read about it here: http://linuxcommand.org/tlcl.php http://linuxcommand.org/tlcl.php

Kiro 13y ago

When do you need man ascii and xxd? I'm a front-end developer so I have no clue about these low-level things but I'm interested and would like to learn.

F30 13y ago
There are a lot of usecases for bot of them, mostly debugging I's say. So I'm gonna tell you what I used them for in the past months: The ASCII table might be useful when you're dealing with encoding problems and want to know e.g. what exactly happened in a broken URL encoding. I used both of them for binary exploitation in the security class I took at university: When utilizing buffer overflows, it's important to get every byte at the right place of your payload. Sometimes, you also need to treat it as a string and that's where ASCII comes in. These are only two specific usecases. As I said, there are a lot more. Development of binary file parsers could be another.
cmhamill 13y ago
This is a very silly use, but I use `ascii` when I can't distinguish between 0 and O in a font ? `ascii 0`, for example, clears that right up.

doktorn 13y ago

strings - find the printable strings in a object, or other binary, file. Good command to know of if you want to snoop around in binary files.

ngcazz 13y ago

Brace expansion to create a backup: cp {file,.bak} reduces to cp file file.bak

qu4z-2 13y ago
Brace expansion is great! My understanding, though, is that you'd need to write: cp file{,.bak} Was it a typo, or is there some feature/special-case I don't know about?
- ngcazz 13y ago
  Oh, you're right. Thanks. (Can't edit the original comment anymore)

ruok0101 13y ago

One little known, but very handy tip: At any password prompt, if you knowingly mis-keyed your password, rather than pressing return, getting the "your an idiot" message (often after an annoying 1 - 2 second delay), and having to retype the command, you can actually press ctrl-U, which is the "Clear the line" bash command, and start over typing your password again.

rockybutler 13y ago
For those of us using emacs-style bash keybinds. For vi-style keybinds, it's something more like "esc dd". PS if you are a Vim user, you owe it to yourself to try the vim-style keybinds. They make a world of difference.

mncolinlee 13y ago

Along with the SSH recommendations, I would include some of the great helpers for ssh agent like keychain. https://wiki.gentoo.org/wiki/Keychain https://wiki.gentoo.org/wiki/Keychain