7 ms·

Linux shell tips and tricks

95 points by Ashuu 13y ago

clarry 13y ago
So, a random assortment of commands, most of which have nothing to do with Linux. I'm sure somebody will find something useful here though.
meacco 13y ago
Can be useful, but I found this more interesting on same website: https://news.ycombinator.com/item?id=6789252 https://news.ycombinator.com/item?id=6789252
onosendai 13y ago
The quick 'n dirty recipe to test the disk write speed isn't ideal, since there's the buffer cache which significantly skews the results. A much better way would be: > sync && time sh -c "dd if=/dev/zero of=foo bs=1M count=10000 && sync" Then just divide 10000 (or whatever value you choose) by the number of seconds elapsed and you should get a much closer approximation of the sequential write speed, taking into account completely flushing the buffer to disk.
- emilw 13y ago
  Even better would be to not use dd, it isn't intended for testing/benchmarking write speeds, and use http://www.iozone.org/ http://www.iozone.org/
  - ehsanu1 13y ago
    Or fio. https://www.linux.com/learn/tutorials/442451-inspecting-disk-io-performance-with-fio/ https://www.linux.com/learn/tutorials/442451-inspecting-disk...
  - onosendai 13y ago
    This is a collection of quick shell recipes, so in that context it's perfectly acceptable. Besides, there are some environments where you can't just install packages willy-nilly, even if they're present on the repositories.
- vram22 13y ago
  I had read some time ago that sync doesn't actually flush the buffer cache, it just schedules it to be written (and that may happen a little later). Could be wrong though, or the information could be outdated in recent Linux versions (I read that about some UNIX version, IIRC).
  - vram22 13y ago
    Edit: by "a little later", I meant: not necessarily synchronously. So the sync command may return before the buffer cache is actually flushed.
lucb1e 13y ago
> Extract tar.gz in new directory: tar zxvf package.tar.gz -C new_dir For quick reference, since it seems to be hard[1], these are the flags you'll most need in tar: c = create x = extract v = verbose z = gzipped; compressed. f = file So for example `tar xf example.tar` will extract that tar file. Or `tar cf example.tar .` will pack all files in the current directory to example.tar. For compression and verbosity, it would be `tar zvcf example.tar.gz .`. The .gz at the end is optional, but now others can see what kind of archive it is. Extract .tar.gz files with `tar xzf example.tar.gz`. For bonus points, here is a poor man's version of scp/rsync: # sending computer tar czf - . | openssl aes-128-cbc -k 'secret' | nc -l 1 # receiving computer nc 192.168.1.102 1 | openssl aes-128-cbc -d -k 'secret' | tar xf - [1] Obligatory xkcd https://xkcd.com/1168/ https://xkcd.com/1168/ Edit: replaced all asterisks with a period (.) since it would become italics. Tar is recursive by default, so it should work the same.
- dmd 13y ago
  And the 'z' flag is unnecessary; gnu tar is smart enough to figure that out on its own.
  - lucb1e 13y ago
    Yesterday I got an error 'error: tar archive is compressed. Aborting' or something along those lines. When reading a file it might detect it because of the filename, but with stdin you need to specify the flag it seems.
- parennoob 13y ago
  Just use dtrx (do the right extraction) via `pip install dtrx`. Makes life much easier.
  - voltagex_ 13y ago
    GNU tar seems to have gained the a switch which will detect the right decompressor. e.g. tar axf linux.tar.gz tar axf linux.tar.bz2 tar axf linux.tar.xz will all work, assuming you have gzip, bzip2 and xz installed.
olog-hai 13y ago
This is incorrect and misleading right off the bat. Ctrl+Z does not "send process to background." You will almost always need to use the bg command for that after suspending the running program.
- voltagex_ 13y ago
  Hey cool, I never knew there was bg to go with fg. Ctrl Z is stop/suspend, unless the application has handling for that signal (e.g. lftp) Now off to learn about disown, dtatch and nohup
  - teddyh 13y ago
    SIGTSTP (which Ctrl-Z sends) is not the same as SIGSTOP (which always stops/suspends processes). There is always more to learn.
    - voltagex_ 13y ago
      That's very useful too - http://stackoverflow.com/questions/11886812/whats-the-difference-between-sigstop-and-sigtstp http://stackoverflow.com/questions/11886812/whats-the-differ... and then looking for a way to send everything via the keyboard brought me to http://superuser.com/questions/288772/shell-sigkill-keybinding http://superuser.com/questions/288772/shell-sigkill-keybindi...
  - jlgaddis 13y ago
    > Hey cool, I never knew there was bg to go with fg. `jobs` might be useful to you as well, in that case.
- njharman 13y ago
  Came here to say that and to add. Long list of commands with no explanation or details is minimally useful. I challenge you to read a man page a week. Use that command every day that week. Find away to use it with two or three other commands you know. Breakdown tasks into units the size of commands you know. Fill in missing pieces with bash or scripting language of choice.
hpaavola 13y ago
"Error establishing a database connection" :( Here's a Google Cache link: http://webcache.googleusercontent.com/search?q=cache:5z9gUWvIH40J:www.techbar.me/linux-shell-tips/+&cd=1&hl=en&ct=clnk&gl=fi http://webcache.googleusercontent.com/search?q=cache:5z9gUWv... Content is typical blog style content and even comments come from Disqus, so ther should not be any reason to connect to a database with every request.
- yeukhon 13y ago
  "Copyright © 2013 - Alen Komljen - Hosted on OpenShift " Maybe author is using a free edition which has limit?
weland 13y ago
> I’m using Linux shell on daily basis, but I often forgot some useful command or shell tip. I hate to be overly pedantic, but what shell would that be? There is no such thing as the Linux shell.
- emilw 13y ago
  I don't think your being overly pedantic. The differences between sh/bash/csh/tcsh/ksh/zsh alone are enough to make your head spin.
- dredmorbius 13y ago
  Well, the default Linux shell is /bin/bash on most systems. I'd say more about the article itself and what it should and/or does say on this, but at present, I suspect the author is looking up "Webserver and Database Scalability and Availability Tips & Tricks": "Error establishing a database connection"
  - profquail 13y ago
    A number of Linux distributions have moved to the BSD-licensed dash[1] for running their system initialization scripts, as it's noticeably faster than bash (resulting in faster boot times). Ubuntu made this change back in 2006 [2][3], and Debian switched to dash for Debian Squeeze. Bash is still the default user shell on those systems, though. [1]: https://en.wikipedia.org/wiki/Debian_Almquist_shell https://en.wikipedia.org/wiki/Debian_Almquist_shell [2]: http://lwn.net/Articles/343924/ http://lwn.net/Articles/343924/ [3]: https://wiki.ubuntu.com/DashAsBinSh https://wiki.ubuntu.com/DashAsBinSh
  - weland 13y ago
    I'm well-aware that bash is a (if not the) popular choice on desktops and servers, but my terse and somewhat snarky answer was motivated by two other factors: 1. Not only do distributions like Debian or Ubuntu use dash, but various other shells are becoming more relevant to programmers due to their inclusion in special-purpose systems, such as devices running with a Busybox-userspace (which includes pretty much every Android phone, for instance). 2. If the article claims to have any educative value, it should definitely mention that this is something about a particular shell. Assuming the author isn't lying about his use of nix systems (since the diversity of shells is common knowledge to anyone seriously using Unices), the other logical choice is that saying bash instead of Linux isn't as cool in terms of SEO. The WWW is cluttered enough with every marketing drone hiring cheap labour on freelancer.com to fill it with bad spinoffs just to promote their semi-scam business, making any kind of legitimate information so hard to find you'd think it's 1998 again for some topics; it itches me on a personal* level when I see programmers doing that. We should know better.
mrmaloke 13y ago
A friend of mine made a presentation on tips and tricks on bash (and more). I learned some really helpful shortcuts from it. http://joschi.github.io/shell-for-developers-presentation/#contents http://joschi.github.io/shell-for-developers-presentation/#c...
- derwiki 13y ago
  Decent breadth coverage, but I'm still confused why people still use screen over tmux.
rtpg 13y ago
Nice list of stuff, especially for people relatively new to command line stuff there's a cool twitter account called "Command Line Magic" (https://twitter.com/climagic https://twitter.com/climagic) that does some fun stuff, and is pretty shell-agnostic. Also, for those willing to, I'd strongly suggest checking out fish. it's not bash-compatible, but there's a lot of good usability aspects to it (especially compared to vanilla bash), I love it. Out of curiosity though, this command : tar zxvf package.tar.gz -C new_dir it's always intrigued me that the tar command (I imagine it is mainly used to "unzip" things) does not make it very simple to do this. I've started to memorize the 4-letter combo but for a while I had to google it everytime (man pages are not useful for learning things for a lot of unix cl tools).
- rich90usa 13y ago
  The mnemonic that I've ran across that has been helpful for this is feigning a horrible German accent and saying "eXtract Ze File!"
  - elteto 13y ago
    That's funny! i just remember it as eXtract gZip File, no accents necessary!
- elteto 13y ago
  tar is not a compression program per se, but a file archiving tool. It is used because it preserves file system information such as permissions, owner and group, which would otherwise be lost if you used zip/gzip/bzip alone. Each letter has a meaning, x is for extract, f is file, v is verbose (print details to screen) and f is for file. So the cryptic four letters are really easy to rememeber: eXtract the Zip File Verbosely. Change the x for a c (Create) and you can make a new file.
- clarry 13y ago
  I was going to ask what is wrong with the man page (because in my experience they do a mighty fine job of explaining most traditional unix tools). But then I checked the manuals for two variants of tar (gnu and obsd), and I can see how they might appear less than helpful for a new user who has never dealt with gzip before. I am surprised the xz(v)f invocation isn't in the examples. I could've sworn it's there...
noselasd 13y ago
CTRL-x CTRL-e to bring up editor in bash, often helpful
- jlgaddis 13y ago
  Eh? That does nothing for me.
- dTal 13y ago
  bash: emacs: command not found As a vim user I feel as if I just got trolled by a piece of software: Cool website: Hey type this emacs-y key sequence into bash, it's super useful! Unsuspecting chump: Ooh okay! <tippy-tap> Bash: Hahahaha lolno, install emacs luser! Admittedly $EDITOR is unset and it works if I set it, but why the hell is Emacs the default?
WizzleKake 13y ago
Any discussion of Linux shell tricks/tips is incomplete without a mention of the readline shortcuts: http://www.bigsmoke.us/readline/shortcuts http://www.bigsmoke.us/readline/shortcuts
- derwiki 13y ago
  I just told myself that learning all of these would be time well spent. Thanks for posting!
  - tomsthumb 13y ago
    Knowing all of those means you basically get movement in emacs for free.
yeukhon 13y ago
I also recommend http://explainshell.com/ http://explainshell.com/. It does its best to break down what a shell command like tar vxf file.gz does.
nu2ycombinator 13y ago
Here is the cached version of this page http://webcache.googleusercontent.com/search?q=cache:5z9gUWvIH40J:www.techbar.me/linux-shell-tips/+&cd=1&hl=en&ct=clnk&gl=us http://webcache.googleusercontent.com/search?q=cache:5z9gUWv...

95 points by Ashuu 13y ago

So, a random assortment of commands, most of which have nothing to do with Linux. I'm sure somebody will find something useful here though.

meacco 13y ago

Can be useful, but I found this more interesting on same website: https://news.ycombinator.com/item?id=6789252 https://news.ycombinator.com/item?id=6789252

onosendai 13y ago

The quick 'n dirty recipe to test the disk write speed isn't ideal, since there's the buffer cache which significantly skews the results. A much better way would be: > sync && time sh -c "dd if=/dev/zero of=foo bs=1M count=10000 && sync" Then just divide 10000 (or whatever value you choose) by the number of seconds elapsed and you should get a much closer approximation of the sequential write speed, taking into account completely flushing the buffer to disk.

emilw 13y ago
Even better would be to not use dd, it isn't intended for testing/benchmarking write speeds, and use http://www.iozone.org/ http://www.iozone.org/
- ehsanu1 13y ago
  Or fio. https://www.linux.com/learn/tutorials/442451-inspecting-disk-io-performance-with-fio/ https://www.linux.com/learn/tutorials/442451-inspecting-disk...
- onosendai 13y ago
  This is a collection of quick shell recipes, so in that context it's perfectly acceptable. Besides, there are some environments where you can't just install packages willy-nilly, even if they're present on the repositories.
vram22 13y ago
I had read some time ago that sync doesn't actually flush the buffer cache, it just schedules it to be written (and that may happen a little later). Could be wrong though, or the information could be outdated in recent Linux versions (I read that about some UNIX version, IIRC).
- vram22 13y ago
  Edit: by "a little later", I meant: not necessarily synchronously. So the sync command may return before the buffer cache is actually flushed.

lucb1e 13y ago

> Extract tar.gz in new directory: tar zxvf package.tar.gz -C new_dir For quick reference, since it seems to be hard[1], these are the flags you'll most need in tar: c = create x = extract v = verbose z = gzipped; compressed. f = file So for example `tar xf example.tar` will extract that tar file. Or `tar cf example.tar .` will pack all files in the current directory to example.tar. For compression and verbosity, it would be `tar zvcf example.tar.gz .`. The .gz at the end is optional, but now others can see what kind of archive it is. Extract .tar.gz files with `tar xzf example.tar.gz`. For bonus points, here is a poor man's version of scp/rsync: # sending computer tar czf - . | openssl aes-128-cbc -k 'secret' | nc -l 1 # receiving computer nc 192.168.1.102 1 | openssl aes-128-cbc -d -k 'secret' | tar xf - [1] Obligatory xkcd https://xkcd.com/1168/ https://xkcd.com/1168/ Edit: replaced all asterisks with a period (.) since it would become italics. Tar is recursive by default, so it should work the same.

dmd 13y ago
And the 'z' flag is unnecessary; gnu tar is smart enough to figure that out on its own.
- lucb1e 13y ago
  Yesterday I got an error 'error: tar archive is compressed. Aborting' or something along those lines. When reading a file it might detect it because of the filename, but with stdin you need to specify the flag it seems.
parennoob 13y ago
Just use dtrx (do the right extraction) via `pip install dtrx`. Makes life much easier.
- voltagex_ 13y ago
  GNU tar seems to have gained the a switch which will detect the right decompressor. e.g. tar axf linux.tar.gz tar axf linux.tar.bz2 tar axf linux.tar.xz will all work, assuming you have gzip, bzip2 and xz installed.

olog-hai 13y ago

This is incorrect and misleading right off the bat. Ctrl+Z does not "send process to background." You will almost always need to use the bg command for that after suspending the running program.

voltagex_ 13y ago
Hey cool, I never knew there was bg to go with fg. Ctrl Z is stop/suspend, unless the application has handling for that signal (e.g. lftp) Now off to learn about disown, dtatch and nohup
- teddyh 13y ago
  SIGTSTP (which Ctrl-Z sends) is not the same as SIGSTOP (which always stops/suspends processes). There is always more to learn.
  - voltagex_ 13y ago
    That's very useful too - http://stackoverflow.com/questions/11886812/whats-the-difference-between-sigstop-and-sigtstp http://stackoverflow.com/questions/11886812/whats-the-differ... and then looking for a way to send everything via the keyboard brought me to http://superuser.com/questions/288772/shell-sigkill-keybinding http://superuser.com/questions/288772/shell-sigkill-keybindi...
- jlgaddis 13y ago
  > Hey cool, I never knew there was bg to go with fg. `jobs` might be useful to you as well, in that case.
njharman 13y ago
Came here to say that and to add. Long list of commands with no explanation or details is minimally useful. I challenge you to read a man page a week. Use that command every day that week. Find away to use it with two or three other commands you know. Breakdown tasks into units the size of commands you know. Fill in missing pieces with bash or scripting language of choice.

hpaavola 13y ago

"Error establishing a database connection" :( Here's a Google Cache link: http://webcache.googleusercontent.com/search?q=cache:5z9gUWvIH40J:www.techbar.me/linux-shell-tips/+&cd=1&hl=en&ct=clnk&gl=fi http://webcache.googleusercontent.com/search?q=cache:5z9gUWv... Content is typical blog style content and even comments come from Disqus, so ther should not be any reason to connect to a database with every request.

yeukhon 13y ago
"Copyright © 2013 - Alen Komljen - Hosted on OpenShift " Maybe author is using a free edition which has limit?

weland 13y ago

> I’m using Linux shell on daily basis, but I often forgot some useful command or shell tip. I hate to be overly pedantic, but what shell would that be? There is no such thing as the Linux shell.

emilw 13y ago
I don't think your being overly pedantic. The differences between sh/bash/csh/tcsh/ksh/zsh alone are enough to make your head spin.
dredmorbius 13y ago
Well, the default Linux shell is /bin/bash on most systems. I'd say more about the article itself and what it should and/or does say on this, but at present, I suspect the author is looking up "Webserver and Database Scalability and Availability Tips & Tricks": "Error establishing a database connection"
- profquail 13y ago
  A number of Linux distributions have moved to the BSD-licensed dash[1] for running their system initialization scripts, as it's noticeably faster than bash (resulting in faster boot times). Ubuntu made this change back in 2006 [2][3], and Debian switched to dash for Debian Squeeze. Bash is still the default user shell on those systems, though. [1]: https://en.wikipedia.org/wiki/Debian_Almquist_shell https://en.wikipedia.org/wiki/Debian_Almquist_shell [2]: http://lwn.net/Articles/343924/ http://lwn.net/Articles/343924/ [3]: https://wiki.ubuntu.com/DashAsBinSh https://wiki.ubuntu.com/DashAsBinSh
- weland 13y ago
  I'm well-aware that bash is a (if not the) popular choice on desktops and servers, but my terse and somewhat snarky answer was motivated by two other factors: 1. Not only do distributions like Debian or Ubuntu use dash, but various other shells are becoming more relevant to programmers due to their inclusion in special-purpose systems, such as devices running with a Busybox-userspace (which includes pretty much every Android phone, for instance). 2. If the article claims to have any educative value, it should definitely mention that this is something about a particular shell. Assuming the author isn't lying about his use of nix systems (since the diversity of shells is common knowledge to anyone seriously using Unices), the other logical choice is that saying bash instead of Linux isn't as cool in terms of SEO. The WWW is cluttered enough with every marketing drone hiring cheap labour on freelancer.com to fill it with bad spinoffs just to promote their semi-scam business, making any kind of legitimate information so hard to find you'd think it's 1998 again for some topics; it itches me on a personal* level when I see programmers doing that. We should know better.

mrmaloke 13y ago

A friend of mine made a presentation on tips and tricks on bash (and more). I learned some really helpful shortcuts from it. http://joschi.github.io/shell-for-developers-presentation/#contents http://joschi.github.io/shell-for-developers-presentation/#c...

derwiki 13y ago
Decent breadth coverage, but I'm still confused why people still use screen over tmux.

rtpg 13y ago

Nice list of stuff, especially for people relatively new to command line stuff there's a cool twitter account called "Command Line Magic" (https://twitter.com/climagic https://twitter.com/climagic) that does some fun stuff, and is pretty shell-agnostic. Also, for those willing to, I'd strongly suggest checking out fish. it's not bash-compatible, but there's a lot of good usability aspects to it (especially compared to vanilla bash), I love it. Out of curiosity though, this command : tar zxvf package.tar.gz -C new_dir it's always intrigued me that the tar command (I imagine it is mainly used to "unzip" things) does not make it very simple to do this. I've started to memorize the 4-letter combo but for a while I had to google it everytime (man pages are not useful for learning things for a lot of unix cl tools).

rich90usa 13y ago
The mnemonic that I've ran across that has been helpful for this is feigning a horrible German accent and saying "eXtract Ze File!"
- elteto 13y ago
  That's funny! i just remember it as eXtract gZip File, no accents necessary!
elteto 13y ago
tar is not a compression program per se, but a file archiving tool. It is used because it preserves file system information such as permissions, owner and group, which would otherwise be lost if you used zip/gzip/bzip alone. Each letter has a meaning, x is for extract, f is file, v is verbose (print details to screen) and f is for file. So the cryptic four letters are really easy to rememeber: eXtract the Zip File Verbosely. Change the x for a c (Create) and you can make a new file.
clarry 13y ago
I was going to ask what is wrong with the man page (because in my experience they do a mighty fine job of explaining most traditional unix tools). But then I checked the manuals for two variants of tar (gnu and obsd), and I can see how they might appear less than helpful for a new user who has never dealt with gzip before. I am surprised the xz(v)f invocation isn't in the examples. I could've sworn it's there...

noselasd 13y ago

CTRL-x CTRL-e to bring up editor in bash, often helpful

jlgaddis 13y ago
Eh? That does nothing for me.
dTal 13y ago
bash: emacs: command not found As a vim user I feel as if I just got trolled by a piece of software: Cool website: Hey type this emacs-y key sequence into bash, it's super useful! Unsuspecting chump: Ooh okay! <tippy-tap> Bash: Hahahaha lolno, install emacs luser! Admittedly $EDITOR is unset and it works if I set it, but why the hell is Emacs the default?

WizzleKake 13y ago

Any discussion of Linux shell tricks/tips is incomplete without a mention of the readline shortcuts: http://www.bigsmoke.us/readline/shortcuts http://www.bigsmoke.us/readline/shortcuts

derwiki 13y ago
I just told myself that learning all of these would be time well spent. Thanks for posting!
- tomsthumb 13y ago
  Knowing all of those means you basically get movement in emacs for free.

yeukhon 13y ago

I also recommend http://explainshell.com/ http://explainshell.com/. It does its best to break down what a shell command like tar vxf file.gz does.

nu2ycombinator 13y ago

Here is the cached version of this page http://webcache.googleusercontent.com/search?q=cache:5z9gUWvIH40J:www.techbar.me/linux-shell-tips/+&cd=1&hl=en&ct=clnk&gl=us http://webcache.googleusercontent.com/search?q=cache:5z9gUWv...