26 ms·

The Art of Command Line

466 points by zalzal 11y ago

pella 11y ago
similar for data science: http://datascienceatthecommandline.com/ http://datascienceatthecommandline.com/ https://github.com/jeroenjanssens/data-science-at-the-command-line https://github.com/jeroenjanssens/data-science-at-the-comman...
michaelvkpdx 11y ago
I learned a dozen new things in the first 100 lines or so, that will save me hours, and I've been living in bash for 20 years. Thanks for this!
cbd1984 11y ago
There's no reason to learn vi if you know Emacs. If you claim that Emacs isn't installed everywhere, guess what: Neither is vi. If you want an "installed everywhere" editor, learn ed. If you're willing to take an editor with you (or otherwise make sure a specific editor is everywhere you are), there's no reason it has to be vi.
- kec 11y ago
  vi is a posix requirement, emacs is not. If the system you're working on has ed, it'll have some version of vi or vim as well.
  - shabble 11y ago
    and not having to know ed is an excellent reason to learn vi :) obCompulsoryEdJoke: https://www.gnu.org/fun/jokes/ed-msg.html https://www.gnu.org/fun/jokes/ed-msg.html
    - CPLX 11y ago
      That was pretty damm amusing.
    - endgame 11y ago
      a I learned ed for a joke and found myself actually liking some features so much that I started porting them to emacs. . w
      - bch 11y ago
        I've used ed(1) to do complex(ish) programmatic edits to files that I think would have been extremely painful otherwise. An interesting editor worth spending at least a small amount of brainpower on.
  - emidln 11y ago
    I don't think this is true. I've certainly been in a bunch of hairy situations over the years where I had nothing fancier than ed available to me because I was in some sort of recovery mode. The single user mode that FreeBSD comes with is lacking vi if, for example, you don't have / mounted yet. Thankfully ed is in the ramdisk.
  - cbd1984 11y ago
    Not in Ubuntu.
  - cogburnd02 11y ago
    V7/x86 [1] has both vi and ed, but ed is easier to get to. "Of course, we are not absolutely compelled to use ed, since V7/x86 comes with the full-screen editor vi (as well as the related line editor ex), but getting access to vi in this context would involve mounting the /usr filesystem and changing various environment variables and settings to allow editing in full-screen mode. Some basic ed skills are particularly useful in the V7 world, and most of all when one is involved on system administration tasks." [2] [1] http://www.nordier.com/v7x86/ http://www.nordier.com/v7x86/ [2] http://www.nordier.com/v7x86/doc/v7x86intro.pdf http://www.nordier.com/v7x86/doc/v7x86intro.pdf
- umanwizard 11y ago
  I can't remember ever encountered a unix-like system without "vi" installed. I've used (and set up) machines running FreeBSD, DragonflyBSD, OSX, Ubuntu, Debian, RHEL, ... Plenty of those didn't have emacs, especially on first boot, when you're most likely to need to modify network configuration files in order to get online to install new packages in the first place... Edit: By "vi" I mean that somewhere in the path there is a program called vi. Of course, usually this is nvi, vim, evil, etc; I don't mean the original UNIX vi binary.
  - cbd1984 11y ago
    > I can't remember ever encountered a unix-like system without "vi" installed. Default Ubuntu install.
    - umanwizard 11y ago
      Wow, seriously? I stand corrected -- still much more likely to find than emacs.
      - dagw 11y ago
        Ubuntu (and Debian) comes with vi (not vim) by default. It's just not the default editor.
    - artursapek 11y ago
      Ubuntu 14.04, the default Ubuntu on EC2, definitely comes with vi.
      - usef- 11y ago
        Yeah, I'm guessing cbd1984 is confusing it with 'vim'.
        cbd1984 11y ago
        Of course not.
- baby 11y ago
  I much prefer emacs and at least emacs shortcuts are by default in most shell (and everywhere in osx!) but no, every servers I install (debian, ubuntu, osx...) comes with vi but not emacs.
- cbd1984 11y ago
  Anyone considering using vi should be warned away by this practice of "disagreement by downvoting". If they do it here, imagine what they would do to a bug report!
scrollaway 11y ago
> To disable slow i18n routines and use traditional byte-based sort order, use export LC_ALL=C (in fact, consider putting this in your ~/.bashrc). Do not. Having a non-utf8 locale means you won't be able to handle utf-8 sanely ("that's why it's faster") and it will break at the most inexplicable times. Any non-latin1 character appearing in your prompt or command line with this will mess its spacing up for example. Do not do not do not. Hell I even check for it in my .zshrc: https://github.com/jleclanche/dotfiles/blob/master/.zshrc#L378-L381 https://github.com/jleclanche/dotfiles/blob/master/.zshrc#L3... Good post otherwise.
- binarycrusader 11y ago
  I have to reiterate this; don't do it -- sort order changes and you'll run into other mysterious issues. It's not worth the "performance improvement".
  - cle 11y ago
    In some cases, it certainly is worth the improvement. Performance improvements translate into $$$. Knowing that "LC_ALL=C can improve performance but cause UTF problems" is awesome information to base a design on. A blanket statement like "never use LC_ALL=C" is not.
    - binarycrusader 11y ago
      Short of a bug or other pathological issue, if you resort to messing with locale just to avoid a performance problem, you're likely to hide other issues that will cause problems whenever another user attempts to do the same thing and it breaks. Again, not worth it -- if only for performance sake, generally speaking.
  - unhammer 11y ago
    There are good reasons to use LC_ALL=C for certain commands, especially if you want sorting that's stable across different systems. Sometimes you want "aa" to be before "b" even though the Norwegian UTF-8 locale puts it after "å" (which comes after "…zæø"): $ echo $'a\naa\nb\nå'| LC_ALL=C sort a aa b å $ echo $'a\naa\nb\nå'| LC_ALL=nn_NO.UTF-8 sort a b å aa Even worse, some characters that are byte-different collate the same: $ echo $'∨\n∧'|LC_ALL=C sort -u ∧ ∨ $ echo $'∨\n∧'|sort -u ∨ Handling Unicode sanely requires understanding some Unicode.
agumonkey 11y ago
Just throwing this http://mywiki.wooledge.org/BashGuide http://mywiki.wooledge.org/BashGuide (it's listed 2 indirections deep so will probably be missed) Every time you have an issue with the shell, go there.
MachinaX 11y ago
The Linux Command Line (free pdf): http://linuxcommand.org/tlcl.php http://linuxcommand.org/tlcl.php
- tomaac 11y ago
  Hosted on Sourceforge which is now blocked almost everywhere.
  - MachinaX 11y ago
    You're exactly right tomaac. Thank you for your comment. The reason: http://arstechnica.com/information-technology/2015/05/sourceforge-grabs-gimp-for-windows-account-wraps-installer-in-bundle-pushing-adware/ http://arstechnica.com/information-technology/2015/05/source...
hamburglar 11y ago
The author should prepare to catch some hell for advocating 'ForwardAgent=yes' in ssh configs.
- xenophonf 11y ago
  I had meant to take another look at ProxyCommand (http://heipei.github.io/2015/02/26/SSH-Agent-Forwarding-considered-harmful/ http://heipei.github.io/2015/02/26/SSH-Agent-Forwarding-cons...), so thanks for the reminder!
widdershins 11y ago
Seems like a good introduction. I'm an amateur who's been using IDEs up until now. I'm just starting to become dimly aware of the power at the command line, and slowly learning in a haphazard way. This could be very helpful.
jwcrux 11y ago
> Fluency on the command line is a skill that is in some ways archaic... I don't see this to be the case at all. Plus, having this be the opening line may cause people to equate archaic=I don't need this. I'd suggest (would pull request but afk) that you remove "is a skill that is in some ways archaic".
- _jal 11y ago
  Agree. I've been hearing people saying the command line is archaic since Windows 95, and probably before that. Yet somehow I (any everyone I work with, and have worked with over the past 25 years) still works in a shell daily. I don't understand how someone on the technical end of running large internet services could avoid it. Sure, GUI tools have crept in here and there. And I understand that some toolchains mandate all-singing, all-dancing IDEs. I don't work in those areas (the closest I've come was doing Java work for about five years and I still lived in vi, because Eclipse makes me homicidal). And I still see our iOS developers using `find' and `sed'. Which isn't to say GUIs can't improve on the command line for some things. For example, the MacOS tool "A Better Finder Rename" (despite its inability to rename itself something less clunky) is a great tool for mass-renaming, e.g., photos. I have written such critters before as shell/perl/python scripts, but pulling EXIF data and renaming files programmatically is ticky and potentially destructive enough that I tend to write it once and then not modify it unless it breaks. The GUI preview and regex capture display makes this much less annoying.
  - Nemcue 11y ago
    It's a fun experience trying to introduce command lines to people who have lived in IDE-land all their lives. "Command Line?! HAHAHAH"
  - lloyd-christmas 11y ago
    Comparing your first sentence and that of the author, you're missing what I see as the key word: "skill". I use command line everyday. Probably most developers do as well. But I definitely don't have "skill" in it. I think the point is that most developers can get by with nothing more than the basics (whatever "basics" are in the context of their work). The command line isn't archaic, but command line as a fluency might be.
bdamm 11y ago
If you learn bash, and you learn vi, then the next most glorious addition is: set -o vi Then you have vi keys in your shell. And it is marvelous.
- jonahx 11y ago
  Learning about this was a revelation for me. I can't live without it now.
- javajosh 11y ago
  Can you share a little more info about this? I tried it but found it rather confusing. Also, for more readline maneuvers: http://unix.stackexchange.com/questions/21788/how-to-delete-part-of-a-path-in-an-interactive-shell/ http://unix.stackexchange.com/questions/21788/how-to-delete-...
  - brown-dragon 11y ago
    `set -o vi` allows us to use `vi` bindings instead of the default `emacs` bindings. The way to think about it is that the prompt starts out in `vi insert mode` so whatever we type in will just be inserted at the prompt. However, we can hit the`Escape` key and move into `vi normal mode` where the keys are interpreted as normal `vi` commands. So, for example, once we hit `esc` we can navigate around the line using `h` and `l` and navigate history (previous lines) using `k` and `j`. Commands I find very useful are: `jk` : for history navigation `bw`: for moving in the current line. These are word movement commands that are faster to navigate with. `cw` : similarly, changing a particular word in the current line is also great. This works particularly well with history - retrieve a previous command and change a word or option `dw` : for deleting the current word `/` : is a fantastic command for searching history. `/cd` will bring up our last change directory command and just hitting `n` will cycle through all our change directory commands that are in history. `AI` : to insert at the beginning/end of line ...and so on. If you have `vi` muscle memory it's a fantastic option for working on the shell. (edit: Formatting)
    - coldpie 11y ago
barely_stubbell 11y ago
I'm glad the author took time to mention sl - every sysadmin should make sure that its installed on all the boxes they manage.
- pyre 11y ago
  Good ole' steam locomotive command...
craneca0 11y ago
The new csysdig curses CLI [1] unifies and replaces a bunch of the system debugging tools listed here. [1] https://github.com/draios/sysdig/wiki/Csysdig%20Overview https://github.com/draios/sysdig/wiki/Csysdig%20Overview
Wonnk13 11y ago
pretty good writeup. Learning nohup was a life changing experience for me.
- unhammer 11y ago
  there's also disown. It's a bash builtin, so you'll have to type "help disown" ("man disown" gets you nothing). E.g. to run touchegg and close your terminal, keeping touchegg running: touchegg & disown; exit
  - scrollaway 11y ago
    I use an alias for it: function launch { type $1 >/dev/null || { print "$1 not found" && return 1 } $@ &>/dev/null &| } alias launch="launch "
    - unhammer 11y ago
      why the alias as well as the function?
      - scrollaway 11y ago
        https://wiki.archlinux.org/index.php/Sudo#Passing_aliases https://wiki.archlinux.org/index.php/Sudo#Passing_aliases
        unhammer 11y ago
        ah, thanks for that tip :-)
karmicthreat 11y ago
So this is tangentially related by why do people seem to push Vi? Nano is a perfectly serviceable editor. If I need to do extensive editing I always end up loading it into Atom, Sublime or some other editor anyway. I've never really had to use a console editor for more than 10s of lines.
- girvo 11y ago
  Nano is fine for 10s of lines, but Vim is great for much much more than that. It's also nice to have the same keybindings for every console app (gosh I love readline), and it's repeat commands make editing text an absolute breeze. Why open another editor on my host if I can edit the file in my console on a remote server?
- elchief 11y ago
  nano is super easy to use, I agree. I tried vim and gave up. Will try again.
  - baghira 11y ago
    I find it a bit irksome that when discussing command line editors, the dichotomy appears to be be "nano vs vim". There are other CLI editors. I'm quite partial to ne (the Nice Editor), which has more manageable shortcuts than emacs, a command line (it's still a non-modal editor, to be clear), macros and syntax highlighting. Also jed is not bad. While vim (and nvim) have advantages, and certainly benefit from being ubiquitous and having ther shortcuts replicated in other programs, users should shop around, even on the command line.
- mbillie1 11y ago
  vi keybindings are ubiquitous, so familiarity with the editor is beneficial outside the editor. Just offhand, both ncdu and tig (which I have used in the past hour) respond to vi keystrokes. It's not as hard as it first seems to learn, either. Well worth the time spent, vi is worlds more powerful than nano.
- kjs3 11y ago
  "I don't understand why everyone doesn't do things exactly like I do using only tools I like" is a really stupid way to look at the world.
cmrx64 11y ago
> StrictHostKeyChecking=no And welcome to MITM haven. This is awful advice if you care about the first S in SSH.
- Karunamon 11y ago
  Depends on your circumstances, really. If you're in an environment with lots of VM's floating around, being created, destroyed, and so on, dealing with SSH's paranoia (why can't I just dismiss a warning about a host/key mismatch instead of having to edit .known_hosts?) quickly becomes an exercise in frustration for dubious security benefit. (If the enemy is on your LAN and able to manipulate your DNS, you've already lost) On your home box, sure. But let's not pretend there's not a reason for the option to exist.
  - richardwhiuk 11y ago
    In which case, just use telnet. Seriously.
  - DDub 11y ago
    I use an alias to select when to, and not to, use host keys; alias ssht='ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' ssht floaty.vm # does not use host key checking ssh my.bastion.host # validates host key Really I ought to get SSHFP records populated when my vm's are created...
  - jamespo 11y ago
    In these environments you can use server certificates https://www.digitalocean.com/community/tutorials/how-to-create-an-ssh-ca-to-validate-hosts-and-clients-with-ubuntu https://www.digitalocean.com/community/tutorials/how-to-crea...
- AYBABTME 11y ago
  What would be good is an interactive confirmation when a host fingerprint has changed, instead of a flatout abortion and having to edit your `known_host` file. I have to SSH into hundreds/thousands of VM that reuse the same IPs/FQDN sometimes, and the strict host checking is a major PITA. I haven't turned it off tho, I prefer the pain to MITM.
ryenus 11y ago
cat hosts | xargs -I{} ssh root@{} hostname glad to know this one :-) echo y | xargs -Ix echo x how tricky!
- jnpatel 11y ago
  I agree, this is one of my favorite tricks. I like to think of xargs as a `map` function over stdin lines. As the article mentions, especially useful when combined with the -P flag for paralellism. cat urlpaths.txt | xargs -I{} -P wget domain.com/{} Another handy trick, is to use `sh -c` or `bash -c` to include multiple commands with xargs. cat dirs.txt | xargs -I{} sh -c 'cd {}; touch file;'
cbd1984 11y ago
There's no reason to learn vi.
- cbd1984 11y ago
  Anyone considering using vi should be warned away by this practice of "disagreement by downvoting". If they do it here, imagine what they would do to a bug report!
webnrrd2k 11y ago
If you're into the command line, I'd recommend getting a physical copy of Unix Power Tools [1] and spending some time with it. This is a nice article, but Unix Power Tools is better in almost every way for learning the basics (and more) of the Unix command line. This article mentions a few more modern tools, but Unix Power Tools has a far better explanation of what's going on. [1] http://www.amazon.com/Unix-Power-Tools-Third-Edition/dp/0596003307 http://www.amazon.com/Unix-Power-Tools-Third-Edition/dp/0596...
mediocrejoker 11y ago
> To locate a file by name in the current directory, find -iname something . (or similar). To find a file anywhere by name, use locate something (but bear in mind updatedb my not have indexed recently created files). I think this will glob unless you escape the asterisks. Also on my system (debian 8) I need to put the directory to search first, or not at all: find . -iname \something\ find -iname \something\ edit: hackernews ate all my asterisks
- myth_buster 11y ago
  edit: hackernews ate all my asterisks Well that solved the mystery.
- AdieuToLogic 11y ago
  > I think this will glob unless you escape the asterisks. The majority of times, this is the case. At least one exception is when the shell pattern does not match one or more files in the directory which find(1) is executed. However, this can be "surprising" so it's best practice to either use escapes or ensure interpolation is disabled via single quotes (in Bourne shell syntax). > Also on my system (debian 8) I need to put the directory to search first... FWIW, you can specify more than one directory if desired. The primaries will be applied to the results of each.
  - wnoise 11y ago
    > The majority of times, this is the case. At least one exception is when the shell pattern does not match one or more files in the directory Thankfully zsh has the NULLGLOB option which will do the right thing and expand to an empty list, of the zero matching files.
- hobarrera 11y ago
  This depends a lot on the terminal, and my advise is to always suround the glob with quotes: find . -iname "*file*" zsh will cry that nothing matches the glob if you don't. bash will expand it if something matches, or pass it as-is to the client if something does - which will give you unexpected results.
mozumder 11y ago
Man so I guess tcsh/csh lost the shell wars?
- AdieuToLogic 11y ago
  Nah, just the hearts and minds.
seri 11y ago
Unix tools are very powerful, but they are not as intuitive as I would like. It would be nice if we can have an httpie for process management, one for text manipulation, another for system stats, and so on.
- AdieuToLogic 11y ago
  Powerful tools are rarely instinctive. Quite often, one person's obvious is another's opaque.
MarcScott 11y ago
I'd never even thought about commenting a line I was half way through writing, when suddenly realising I'd forgotten the correct arguments. I normally end up opening another terminal and using man there.
- unhammer 11y ago
  I do this sometimes when I want to run some command first from earlier in my history, then the commented out command also stays in history so I can continue with it after I've re-run the earlier command first.
AdieuToLogic 11y ago
Much, much more can be found at: http://www.commandlinefu.com/commands/browse http://www.commandlinefu.com/commands/browse
Ardebou 11y ago
Before that I was not so familiar with cmd. After that I was.
stinos 11y ago
Fluency on the command line is a skill now often neglected or considered archaic By whom? (honest question: even the most GUI oriented people I know reckon the power of command line skills)
- weavie 11y ago
  I work in a .Net house. When we had to move to front end development which required using grunt shock from the command line there was plenty of dissent at how we were being forced back into the "dark ages".
natch 11y ago
Another one for obscure but useful: jot # print 10 values from 1 to 10, with leading zeros: for n in `jot -w "%04d" 10 1 10`; do echo $n; done
- anc84 11y ago
  Just use Bash's builtin things: for n in {0001..0010}; do echo ${n}; done
  - willthames 11y ago
    Or seq -f %04g 1 10
    - natch 11y ago
      Sure seq is OK, but jot does everything seq does and more, so I prefer jot. # generate 10 random numbers between 1 and 1 million jot -r 10 1 1000000
      - anc84 11y ago
        I meant to go all "just use coreutils, there is shuf available already, no need for some third-party tool" but then I realised that jot is an integral part of BSD. Rock on! :) For us Linuxers: shuf -i 1-1000000 -n 10
  - stiGGG 11y ago
    Nice, but this needs an actual version of bash, with that oldie shipped with OSX Yosemite the result is not padded with zeros.
  - natch 11y ago
    Cool, thanks.
joelthelion 11y ago
No mention of autojump? :-p
- ryenus 11y ago
  I googled about autojump, then found fasd and decided to go with it. https://github.com/clvv/fasd https://github.com/clvv/fasd
gitaarik 11y ago
One really useful shortcut I learned in bash is Ctrl-/ which is a sort of undo for bash when composing a command. Can't really explain how it works, you should try it out yourself. Also really handy is Ctrl-Y, which pastes the last cut characters by Ctrl-W, Ctrl-U or Ctrl-K.
- nanny 11y ago
  C-/ and C-y are from readline which mimicked them (and many more) from Emacs.
hobarrera 11y ago
Fun fact: alt+# is impossible on a Spanish keyboard, because alt+3 = #. So you actually need to hold alt to type the character in the first place.

466 points by zalzal 11y ago

pella 11y ago

similar for data science: http://datascienceatthecommandline.com/ http://datascienceatthecommandline.com/ https://github.com/jeroenjanssens/data-science-at-the-command-line https://github.com/jeroenjanssens/data-science-at-the-comman...

michaelvkpdx 11y ago

I learned a dozen new things in the first 100 lines or so, that will save me hours, and I've been living in bash for 20 years. Thanks for this!

cbd1984 11y ago

There's no reason to learn vi if you know Emacs. If you claim that Emacs isn't installed everywhere, guess what: Neither is vi. If you want an "installed everywhere" editor, learn ed. If you're willing to take an editor with you (or otherwise make sure a specific editor is everywhere you are), there's no reason it has to be vi.

kec 11y ago
vi is a posix requirement, emacs is not. If the system you're working on has ed, it'll have some version of vi or vim as well.
- shabble 11y ago
  and not having to know ed is an excellent reason to learn vi :) obCompulsoryEdJoke: https://www.gnu.org/fun/jokes/ed-msg.html https://www.gnu.org/fun/jokes/ed-msg.html
  - CPLX 11y ago
    That was pretty damm amusing.
  - endgame 11y ago
    a I learned ed for a joke and found myself actually liking some features so much that I started porting them to emacs. . w
    - bch 11y ago
      I've used ed(1) to do complex(ish) programmatic edits to files that I think would have been extremely painful otherwise. An interesting editor worth spending at least a small amount of brainpower on.
- emidln 11y ago
  I don't think this is true. I've certainly been in a bunch of hairy situations over the years where I had nothing fancier than ed available to me because I was in some sort of recovery mode. The single user mode that FreeBSD comes with is lacking vi if, for example, you don't have / mounted yet. Thankfully ed is in the ramdisk.
- cbd1984 11y ago
  Not in Ubuntu.
- cogburnd02 11y ago
  V7/x86 [1] has both vi and ed, but ed is easier to get to. "Of course, we are not absolutely compelled to use ed, since V7/x86 comes with the full-screen editor vi (as well as the related line editor ex), but getting access to vi in this context would involve mounting the /usr filesystem and changing various environment variables and settings to allow editing in full-screen mode. Some basic ed skills are particularly useful in the V7 world, and most of all when one is involved on system administration tasks." [2] [1] http://www.nordier.com/v7x86/ http://www.nordier.com/v7x86/ [2] http://www.nordier.com/v7x86/doc/v7x86intro.pdf http://www.nordier.com/v7x86/doc/v7x86intro.pdf
umanwizard 11y ago
I can't remember ever encountered a unix-like system without "vi" installed. I've used (and set up) machines running FreeBSD, DragonflyBSD, OSX, Ubuntu, Debian, RHEL, ... Plenty of those didn't have emacs, especially on first boot, when you're most likely to need to modify network configuration files in order to get online to install new packages in the first place... Edit: By "vi" I mean that somewhere in the path there is a program called vi. Of course, usually this is nvi, vim, evil, etc; I don't mean the original UNIX vi binary.
- cbd1984 11y ago
  > I can't remember ever encountered a unix-like system without "vi" installed. Default Ubuntu install.
  - umanwizard 11y ago
    Wow, seriously? I stand corrected -- still much more likely to find than emacs.
    - dagw 11y ago
      Ubuntu (and Debian) comes with vi (not vim) by default. It's just not the default editor.
  - artursapek 11y ago
    Ubuntu 14.04, the default Ubuntu on EC2, definitely comes with vi.
    - usef- 11y ago
      Yeah, I'm guessing cbd1984 is confusing it with 'vim'.
      - cbd1984 11y ago
        Of course not.
baby 11y ago
I much prefer emacs and at least emacs shortcuts are by default in most shell (and everywhere in osx!) but no, every servers I install (debian, ubuntu, osx...) comes with vi but not emacs.
cbd1984 11y ago
Anyone considering using vi should be warned away by this practice of "disagreement by downvoting". If they do it here, imagine what they would do to a bug report!

scrollaway 11y ago

> To disable slow i18n routines and use traditional byte-based sort order, use export LC_ALL=C (in fact, consider putting this in your ~/.bashrc). Do not. Having a non-utf8 locale means you won't be able to handle utf-8 sanely ("that's why it's faster") and it will break at the most inexplicable times. Any non-latin1 character appearing in your prompt or command line with this will mess its spacing up for example. Do not do not do not. Hell I even check for it in my .zshrc: https://github.com/jleclanche/dotfiles/blob/master/.zshrc#L378-L381 https://github.com/jleclanche/dotfiles/blob/master/.zshrc#L3... Good post otherwise.

binarycrusader 11y ago
I have to reiterate this; don't do it -- sort order changes and you'll run into other mysterious issues. It's not worth the "performance improvement".
- cle 11y ago
  In some cases, it certainly is worth the improvement. Performance improvements translate into $$$. Knowing that "LC_ALL=C can improve performance but cause UTF problems" is awesome information to base a design on. A blanket statement like "never use LC_ALL=C" is not.
  - binarycrusader 11y ago
    Short of a bug or other pathological issue, if you resort to messing with locale just to avoid a performance problem, you're likely to hide other issues that will cause problems whenever another user attempts to do the same thing and it breaks. Again, not worth it -- if only for performance sake, generally speaking.
- unhammer 11y ago
  There are good reasons to use LC_ALL=C for certain commands, especially if you want sorting that's stable across different systems. Sometimes you want "aa" to be before "b" even though the Norwegian UTF-8 locale puts it after "å" (which comes after "…zæø"): $ echo $'a\naa\nb\nå'| LC_ALL=C sort a aa b å $ echo $'a\naa\nb\nå'| LC_ALL=nn_NO.UTF-8 sort a b å aa Even worse, some characters that are byte-different collate the same: $ echo $'∨\n∧'|LC_ALL=C sort -u ∧ ∨ $ echo $'∨\n∧'|sort -u ∨ Handling Unicode sanely requires understanding some Unicode.

agumonkey 11y ago

Just throwing this http://mywiki.wooledge.org/BashGuide http://mywiki.wooledge.org/BashGuide (it's listed 2 indirections deep so will probably be missed) Every time you have an issue with the shell, go there.

MachinaX 11y ago

The Linux Command Line (free pdf): http://linuxcommand.org/tlcl.php http://linuxcommand.org/tlcl.php

tomaac 11y ago
Hosted on Sourceforge which is now blocked almost everywhere.
- MachinaX 11y ago
  You're exactly right tomaac. Thank you for your comment. The reason: http://arstechnica.com/information-technology/2015/05/sourceforge-grabs-gimp-for-windows-account-wraps-installer-in-bundle-pushing-adware/ http://arstechnica.com/information-technology/2015/05/source...

hamburglar 11y ago

The author should prepare to catch some hell for advocating 'ForwardAgent=yes' in ssh configs.

xenophonf 11y ago
I had meant to take another look at ProxyCommand (http://heipei.github.io/2015/02/26/SSH-Agent-Forwarding-considered-harmful/ http://heipei.github.io/2015/02/26/SSH-Agent-Forwarding-cons...), so thanks for the reminder!

widdershins 11y ago

Seems like a good introduction. I'm an amateur who's been using IDEs up until now. I'm just starting to become dimly aware of the power at the command line, and slowly learning in a haphazard way. This could be very helpful.

jwcrux 11y ago

> Fluency on the command line is a skill that is in some ways archaic... I don't see this to be the case at all. Plus, having this be the opening line may cause people to equate archaic=I don't need this. I'd suggest (would pull request but afk) that you remove "is a skill that is in some ways archaic".

_jal 11y ago
Agree. I've been hearing people saying the command line is archaic since Windows 95, and probably before that. Yet somehow I (any everyone I work with, and have worked with over the past 25 years) still works in a shell daily. I don't understand how someone on the technical end of running large internet services could avoid it. Sure, GUI tools have crept in here and there. And I understand that some toolchains mandate all-singing, all-dancing IDEs. I don't work in those areas (the closest I've come was doing Java work for about five years and I still lived in vi, because Eclipse makes me homicidal). And I still see our iOS developers using `find' and `sed'. Which isn't to say GUIs can't improve on the command line for some things. For example, the MacOS tool "A Better Finder Rename" (despite its inability to rename itself something less clunky) is a great tool for mass-renaming, e.g., photos. I have written such critters before as shell/perl/python scripts, but pulling EXIF data and renaming files programmatically is ticky and potentially destructive enough that I tend to write it once and then not modify it unless it breaks. The GUI preview and regex capture display makes this much less annoying.
- Nemcue 11y ago
  It's a fun experience trying to introduce command lines to people who have lived in IDE-land all their lives. "Command Line?! HAHAHAH"
- lloyd-christmas 11y ago
  Comparing your first sentence and that of the author, you're missing what I see as the key word: "skill". I use command line everyday. Probably most developers do as well. But I definitely don't have "skill" in it. I think the point is that most developers can get by with nothing more than the basics (whatever "basics" are in the context of their work). The command line isn't archaic, but command line as a fluency might be.

bdamm 11y ago

If you learn bash, and you learn vi, then the next most glorious addition is: set -o vi Then you have vi keys in your shell. And it is marvelous.

jonahx 11y ago
Learning about this was a revelation for me. I can't live without it now.
javajosh 11y ago
Can you share a little more info about this? I tried it but found it rather confusing. Also, for more readline maneuvers: http://unix.stackexchange.com/questions/21788/how-to-delete-part-of-a-path-in-an-interactive-shell/ http://unix.stackexchange.com/questions/21788/how-to-delete-...
- brown-dragon 11y ago
  `set -o vi` allows us to use `vi` bindings instead of the default `emacs` bindings. The way to think about it is that the prompt starts out in `vi insert mode` so whatever we type in will just be inserted at the prompt. However, we can hit the`Escape` key and move into `vi normal mode` where the keys are interpreted as normal `vi` commands. So, for example, once we hit `esc` we can navigate around the line using `h` and `l` and navigate history (previous lines) using `k` and `j`. Commands I find very useful are: `jk` : for history navigation `bw`: for moving in the current line. These are word movement commands that are faster to navigate with. `cw` : similarly, changing a particular word in the current line is also great. This works particularly well with history - retrieve a previous command and change a word or option `dw` : for deleting the current word `/` : is a fantastic command for searching history. `/cd` will bring up our last change directory command and just hitting `n` will cycle through all our change directory commands that are in history. `AI` : to insert at the beginning/end of line ...and so on. If you have `vi` muscle memory it's a fantastic option for working on the shell. (edit: Formatting)
  - coldpie 11y ago

barely_stubbell 11y ago

I'm glad the author took time to mention sl - every sysadmin should make sure that its installed on all the boxes they manage.

pyre 11y ago
Good ole' steam locomotive command...

craneca0 11y ago

The new csysdig curses CLI [1] unifies and replaces a bunch of the system debugging tools listed here. [1] https://github.com/draios/sysdig/wiki/Csysdig%20Overview https://github.com/draios/sysdig/wiki/Csysdig%20Overview

Wonnk13 11y ago

pretty good writeup. Learning nohup was a life changing experience for me.

unhammer 11y ago
there's also disown. It's a bash builtin, so you'll have to type "help disown" ("man disown" gets you nothing). E.g. to run touchegg and close your terminal, keeping touchegg running: touchegg & disown; exit
- scrollaway 11y ago
  I use an alias for it: function launch { type $1 >/dev/null || { print "$1 not found" && return 1 } $@ &>/dev/null &| } alias launch="launch "
  - unhammer 11y ago
    why the alias as well as the function?
    - scrollaway 11y ago
      https://wiki.archlinux.org/index.php/Sudo#Passing_aliases https://wiki.archlinux.org/index.php/Sudo#Passing_aliases
      - unhammer 11y ago
        ah, thanks for that tip :-)

karmicthreat 11y ago

So this is tangentially related by why do people seem to push Vi? Nano is a perfectly serviceable editor. If I need to do extensive editing I always end up loading it into Atom, Sublime or some other editor anyway. I've never really had to use a console editor for more than 10s of lines.

girvo 11y ago
Nano is fine for 10s of lines, but Vim is great for much much more than that. It's also nice to have the same keybindings for every console app (gosh I love readline), and it's repeat commands make editing text an absolute breeze. Why open another editor on my host if I can edit the file in my console on a remote server?
elchief 11y ago
nano is super easy to use, I agree. I tried vim and gave up. Will try again.
- baghira 11y ago
  I find it a bit irksome that when discussing command line editors, the dichotomy appears to be be "nano vs vim". There are other CLI editors. I'm quite partial to ne (the Nice Editor), which has more manageable shortcuts than emacs, a command line (it's still a non-modal editor, to be clear), macros and syntax highlighting. Also jed is not bad. While vim (and nvim) have advantages, and certainly benefit from being ubiquitous and having ther shortcuts replicated in other programs, users should shop around, even on the command line.
mbillie1 11y ago
vi keybindings are ubiquitous, so familiarity with the editor is beneficial outside the editor. Just offhand, both ncdu and tig (which I have used in the past hour) respond to vi keystrokes. It's not as hard as it first seems to learn, either. Well worth the time spent, vi is worlds more powerful than nano.
kjs3 11y ago
"I don't understand why everyone doesn't do things exactly like I do using only tools I like" is a really stupid way to look at the world.

cmrx64 11y ago

> StrictHostKeyChecking=no And welcome to MITM haven. This is awful advice if you care about the first S in SSH.

Karunamon 11y ago
Depends on your circumstances, really. If you're in an environment with lots of VM's floating around, being created, destroyed, and so on, dealing with SSH's paranoia (why can't I just dismiss a warning about a host/key mismatch instead of having to edit .known_hosts?) quickly becomes an exercise in frustration for dubious security benefit. (If the enemy is on your LAN and able to manipulate your DNS, you've already lost) On your home box, sure. But let's not pretend there's not a reason for the option to exist.
- richardwhiuk 11y ago
  In which case, just use telnet. Seriously.
- DDub 11y ago
  I use an alias to select when to, and not to, use host keys; alias ssht='ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' ssht floaty.vm # does not use host key checking ssh my.bastion.host # validates host key Really I ought to get SSHFP records populated when my vm's are created...
- jamespo 11y ago
  In these environments you can use server certificates https://www.digitalocean.com/community/tutorials/how-to-create-an-ssh-ca-to-validate-hosts-and-clients-with-ubuntu https://www.digitalocean.com/community/tutorials/how-to-crea...
AYBABTME 11y ago
What would be good is an interactive confirmation when a host fingerprint has changed, instead of a flatout abortion and having to edit your `known_host` file. I have to SSH into hundreds/thousands of VM that reuse the same IPs/FQDN sometimes, and the strict host checking is a major PITA. I haven't turned it off tho, I prefer the pain to MITM.

ryenus 11y ago

cat hosts | xargs -I{} ssh root@{} hostname glad to know this one :-) echo y | xargs -Ix echo x how tricky!

jnpatel 11y ago
I agree, this is one of my favorite tricks. I like to think of xargs as a `map` function over stdin lines. As the article mentions, especially useful when combined with the -P flag for paralellism. cat urlpaths.txt | xargs -I{} -P wget domain.com/{} Another handy trick, is to use `sh -c` or `bash -c` to include multiple commands with xargs. cat dirs.txt | xargs -I{} sh -c 'cd {}; touch file;'

cbd1984 11y ago

There's no reason to learn vi.

cbd1984 11y ago
Anyone considering using vi should be warned away by this practice of "disagreement by downvoting". If they do it here, imagine what they would do to a bug report!

webnrrd2k 11y ago

If you're into the command line, I'd recommend getting a physical copy of Unix Power Tools [1] and spending some time with it. This is a nice article, but Unix Power Tools is better in almost every way for learning the basics (and more) of the Unix command line. This article mentions a few more modern tools, but Unix Power Tools has a far better explanation of what's going on. [1] http://www.amazon.com/Unix-Power-Tools-Third-Edition/dp/0596003307 http://www.amazon.com/Unix-Power-Tools-Third-Edition/dp/0596...

mediocrejoker 11y ago

> To locate a file by name in the current directory, find -iname something . (or similar). To find a file anywhere by name, use locate something (but bear in mind updatedb my not have indexed recently created files). I think this will glob unless you escape the asterisks. Also on my system (debian 8) I need to put the directory to search first, or not at all: find . -iname \something\ find -iname \something\ edit: hackernews ate all my asterisks

myth_buster 11y ago
edit: hackernews ate all my asterisks Well that solved the mystery.
AdieuToLogic 11y ago
> I think this will glob unless you escape the asterisks. The majority of times, this is the case. At least one exception is when the shell pattern does not match one or more files in the directory which find(1) is executed. However, this can be "surprising" so it's best practice to either use escapes or ensure interpolation is disabled via single quotes (in Bourne shell syntax). > Also on my system (debian 8) I need to put the directory to search first... FWIW, you can specify more than one directory if desired. The primaries will be applied to the results of each.
- wnoise 11y ago
  > The majority of times, this is the case. At least one exception is when the shell pattern does not match one or more files in the directory Thankfully zsh has the NULLGLOB option which will do the right thing and expand to an empty list, of the zero matching files.
hobarrera 11y ago
This depends a lot on the terminal, and my advise is to always suround the glob with quotes: find . -iname "*file*" zsh will cry that nothing matches the glob if you don't. bash will expand it if something matches, or pass it as-is to the client if something does - which will give you unexpected results.

mozumder 11y ago

Man so I guess tcsh/csh lost the shell wars?

AdieuToLogic 11y ago
Nah, just the hearts and minds.

seri 11y ago

Unix tools are very powerful, but they are not as intuitive as I would like. It would be nice if we can have an httpie for process management, one for text manipulation, another for system stats, and so on.

AdieuToLogic 11y ago
Powerful tools are rarely instinctive. Quite often, one person's obvious is another's opaque.

MarcScott 11y ago

I'd never even thought about commenting a line I was half way through writing, when suddenly realising I'd forgotten the correct arguments. I normally end up opening another terminal and using man there.

unhammer 11y ago
I do this sometimes when I want to run some command first from earlier in my history, then the commented out command also stays in history so I can continue with it after I've re-run the earlier command first.

AdieuToLogic 11y ago

Much, much more can be found at: http://www.commandlinefu.com/commands/browse http://www.commandlinefu.com/commands/browse

Ardebou 11y ago

Before that I was not so familiar with cmd. After that I was.

stinos 11y ago

Fluency on the command line is a skill now often neglected or considered archaic By whom? (honest question: even the most GUI oriented people I know reckon the power of command line skills)

weavie 11y ago
I work in a .Net house. When we had to move to front end development which required using grunt shock from the command line there was plenty of dissent at how we were being forced back into the "dark ages".

natch 11y ago

Another one for obscure but useful: jot # print 10 values from 1 to 10, with leading zeros: for n in `jot -w "%04d" 10 1 10`; do echo $n; done

anc84 11y ago
Just use Bash's builtin things: for n in {0001..0010}; do echo ${n}; done
- willthames 11y ago
  Or seq -f %04g 1 10
  - natch 11y ago
    Sure seq is OK, but jot does everything seq does and more, so I prefer jot. # generate 10 random numbers between 1 and 1 million jot -r 10 1 1000000
    - anc84 11y ago
      I meant to go all "just use coreutils, there is shuf available already, no need for some third-party tool" but then I realised that jot is an integral part of BSD. Rock on! :) For us Linuxers: shuf -i 1-1000000 -n 10
- stiGGG 11y ago
  Nice, but this needs an actual version of bash, with that oldie shipped with OSX Yosemite the result is not padded with zeros.
- natch 11y ago
  Cool, thanks.

joelthelion 11y ago

No mention of autojump? :-p

ryenus 11y ago
I googled about autojump, then found fasd and decided to go with it. https://github.com/clvv/fasd https://github.com/clvv/fasd

gitaarik 11y ago

One really useful shortcut I learned in bash is Ctrl-/ which is a sort of undo for bash when composing a command. Can't really explain how it works, you should try it out yourself. Also really handy is Ctrl-Y, which pastes the last cut characters by Ctrl-W, Ctrl-U or Ctrl-K.

nanny 11y ago
C-/ and C-y are from readline which mimicked them (and many more) from Emacs.

hobarrera 11y ago

Fun fact: alt+# is impossible on a Spanish keyboard, because alt+3 = #. So you actually need to hold alt to type the character in the first place.