10 ms·
Cool, but obscure Unix tools
- deleted 15y ago[deleted]
- hassy 15y agoLooks like he's running them on OSX which means they're likely to work on BSDs too.
- zerosanity 15y agoHow is vim an obscure tool? I'm pretty sure quite a few people use vim daily. After looking over this list, I suspect it's also true for many listed programs.
- larelli 15y agoI knew most of them but not all. However as he targets OSX users, I suspect many of the programs are unknown to them.
- 3am 15y agoAh - thanks clarifying that he targets OSX people. I didn't think that dstat, xargs, curl, vim, screen, rsync, or ack were obscure; but it makes sense if it the audience is OSX users.
- mitjak 15y agoWhy?
- 3am 15y agoNot a dig at OSX users. I have a macbook in addition to a thinkpad running a less common distro in front of me right now. I'm thinking of it in a Bayesian way: if a person is a Linux user or an OSX user, I expect them to know all/most of those tools. However, given that a person (limited to Linux or OSX users) doesn't know about those tools, I expect them to be an OSX user.
- deleted 15y ago[deleted]
- dedward 15y agoI don't think OSX is specifically that important - I've met plenty of "new" linux users who have no idea htese tools exist either - linux has a gui too, you know. Converseley, there are plenty of long-term unix experts out there who also find that a mac makes a nice workstation that still satisifies their need for unix tools plus other stuff.
- 3am 15y agoPlease see my earlier response to mitjak, i think it addresses both of your points.
- jcromartie 15y agoI don't think he's targeting OS X. He just has an OS X skin on his UI. Some of these tools absolutely will not run on OS X.
- msbarnett 15y agoOut of curiosity, which? Edit: looking at the site again, at least one of those screenshots is from OS X. htop lists /sbin/launchd and several processes from /Library/Frameworks.
- lloeki 15y agoNice catch, I thought the same as GP because of powertop.
- kkovacs 15y agoThere really is no "target" audience of this post. :) It just started out as a list of "recommended" tools, then people started suggesting stuff -- some of which I knew, some of which I don't, some of which run on every unix, some only on Linux. And it was easiest to take the screenshots from my OS X laptop. For some tools, like powertop, I had SSH-d into various Linux or FreeBSD boxes. Some of these are uncommon enough that they are not ported to every package manager system, too. :(
- deleted 15y ago[deleted]
- alnayyir 15y agovim is actually more popular than Emacs (FOR NOW), so I don't really know where he's coming from on this one.
- fsniper 15y agoquite a few people uses vim daily?? My life is spent inside vim :) -- ups sorry got the wrong impression.. You should use sarcasm tag. Shame on you!
- lysol 15y agoIf rsync is obscure, then I'm a dumpster wizard. That's a wizard that lives in an overturned dumpster.
- deleted 15y ago[deleted]
- brcrth 15y agohtop, tmux/screen, xargs, vim/emacs, rsync, rtorrent, ack are far common to everyone I know (direct and indirectly) that uses the command line.
- elehack 15y agoYep, lots of them are known to some subset of people. However, if people find one or two really useful ones that they didn't know about previously, the list has done its job. For me, the major winner was dstat. Did not know about that tool - I've been struggling with the output of iostat. I'll also be experimenting with tmux as a screen replacement/alternative.
- pdovy 15y agoYeah, most of these weren't anything new, but I hadn't heard about ack. Very useful.
- hubb 15y agosudo apt-get install ack got me: ack - Kanji code converter :(
- kevindication 15y agoOn debian (and derivatives) the package is called "ack-grep"
- selectnull 15y agoI always install it by downloading standalone version from http://betterthangrep.com/ http://betterthangrep.com/ There is a command on the homepage you can copy paste to shell and execute. Ack is simply too often used and I don't want to type ack-grep all the time.
- pyre 15y ago"man alias" ?
- tybris 15y agoI recently discovered nl and felt sad about the amount of time I had wasted figuring out how to add line numbers to files.
- chanux 15y agocat -n but nl is shorter :)
- uriel 15y agocat -n is discussed in Pike and Kernighan's paper on Unix style: http://harmful.cat-v.org/cat-v/ http://harmful.cat-v.org/cat-v/
- sigil 15y agoThey're both annoying. For use in pipelines that is, as they seem more geared towards typesetting. Which I've honestly never had a use for. Consider this input file: $ cat > test.txt one two three five Let's try to number every line: $ nl test.txt 1 one 2 two 3 three 4 five Ok, the leading spaces are annoying (why not use a single tab?), and are also produced with "cat -n". And why is not numbering blank lines the default? How can you prefix every line, even non-blank ones, with the line number followed by a single tab? Turns out the answer is: $ nl -w1 -n ln -ba -s"`printf '\t'`" test.txt Horrible defaults. HORRIBLE.
- jshb 15y agoWhy does cat need the ability to add line numbers anyway? It seems outside the scope of it's purpose. Next it'll need ability to translate Russian to English. If I ever needed line numbering, I would output into a file and open it in a far more capable text editor/viewer that was designed for doing that well.
- AlecSchueler 15y agoIndeed - `nl` should be preferable to `cat -n` for that reason even if for no others. People have been actually saying this since at least 1983 (http://harmful.cat-v.org/cat-v/ http://harmful.cat-v.org/cat-v/). I'm often surprised that cat is anywhere near as popular as it is. I think I've used it for actual file concatenation maybe two times in the past three years.
- oyving 15y agoI really like pipe viewer (pv). I wish it as more common in base installs of Linux. http://www.catonmat.net/blog/unix-utilities-pipe-viewer/ http://www.catonmat.net/blog/unix-utilities-pipe-viewer/
- jz 15y agoAlthough not as featureful as pv, try hitting CTRL+t under OS X or any of the BSDs during a long running process. It provides status of the current running process without having to prefix the command with pv. I blogged about this a while back: http://jz.posterous.com/bsd-tip-of-the-day-ctrl-t http://jz.posterous.com/bsd-tip-of-the-day-ctrl-t
- pearle 15y agoGreat post! It introduced me to a few tools I wasn't aware of previously. The main OS on my laptop is Ubuntu so this is very appreciated.
- malux85 15y agoMost of these aren't obscure, but it was a good read, and I didn't know about slurm. So over on one of the test machines I apt-get install cowsay ... One of the other devs here is going go get a surprise next time he logs into one of the webservers ... ;)
- gnosis 15y agoabcde - CD to mp3 ripper apg - random password generator base64 - better than uuencode boxes - draw any kind of boxes around your text bsdiff - binary differ bspatch - binary patcher bvi - binary vi (yet another hex editor) ccx2 - console xmms2 client clive - flash video downloader dvipdfmx - dvi to pdf converter enfuse - poor man's HDR get_flash_videos - yet another flash video downloader glark - advanced grep indent - code beautifier lshw - list hardware configuration mcurl - multiple part downloader using curl mktemp - safely create temporary files and directories msort - sort records in complex ways netbrake - bandwidth limiter od - octal dump par - paragraph reformatter par2 - archive verification and repair tool ped - sed done right with perl pinfo - color info reader pipe.vim - make vim part of a unix pipe and allow it to edit the pipe contents pv - Pipe Viewer: a tool for monitoring the progress of data through a pipe pydf - pretty df (disk space viewer) qmv - use your favorite editor to rename files (part of renameutils) qodem - modem program that can do serial, telnet, ssh, zmodem, kermit, etc rdiff-backup - like rsync, but can do incremental backups recode - like dos2unix and unix2dos, but with many more encodings recordmydesktop - make screencast videos remark - great logfile colorizer (part of regex-markup) rkhunter - find rootkit infections rlwrap - add readline editing support to any command safecopy - data recovery tool (better than dd) sponge - soak up stdin and write to a file (for things like pipeline editing) sux - su while transferring X credentials unbuffer - force flushing of stdout upx - executable compressor utimer - countdown timer and stopwatch vared - edit shell variables (part of zsh) watch - run a command multiple times and display the output (with differences highlighted) xdotool - simulate keyboard and mouse activity xxd - hex dump zargs - a version of xargs that makes the find command redundant (part of zsh) zed - very small and fast vi-like editor (part of zsh) zrun - automatically uncompress arguments to command
- deleted 15y ago[deleted]
- cygwin98 15y agotsort -- perform topological sort I bet very few people here are aware of its existence, even it has been part of Unix since Version 7. I recently discovered it and have used it to solve some project Euler problems.
- IvarTJ 15y agoYou don’t need stdio.h to use puts. I was in the process of making something similar before I found what I wanted – rlwrap. It provides readline line editing capabilities to command line applications that don’t support them, such as netcat.
- deleted 15y ago[deleted]
- eru 15y agoHave a look at `join', it joins lines of two files on a common field. Together with `cut' and `grep' you can use text-files as relational databases.
- alnayyir 15y ago>Have a look at `join', it joins lines of two files on a common field. Together with `awk' and `sed' you can use text-files as relational databases. Fixed it for you.
- eru 15y agoawk and sed are useful. But as a principle one should use the weakest tool that gets the job done. That principle also applies to grep regular expressions vs Perl regexes. Regular languages are quite weak in the right way, and thus can be recognized in linear time. Perl's regexes on the other hand, are so powerful, they can even tell prime numbers from composite numbers with their back tracking. Choosing a weaker tool also serves as documention---about which features not to worry about. (Of course you shouldn't try to use a tool that's weaker than what you need. For example, trying to solve some problem crying for recursion or iteration in a spreadsheet will lead to more harm than good.)
- SeanLuke 15y ago> nethack & slash'em > Still the most complex game on the planet. Dwarf Fortress.
- jokermatt999 15y agoDefinitely agreed, but is there anything more complex than DF? As a sidenote, to anyone who doesn't know the story of how Dwarf Fortress is being made, you should check it out. It's one guy, working full time on it, and managing to support himself and his brother through monthly donations. It's released completely free of charge, so this is just solely from people donating. That alone would be a pretty cool story, but when you look at the immense, mindboggling, ridiculously ambitious scale of the the planned finished game, it's incredible. He's basically trying to build a completely "generic fantasy world simulator", with procedurally generated world, history, etc. The amount of detail he goes into in this is amazing, and he has so much more planned. If you ever need to be inspired that one person can make a living doing what they love, or that you can actually implement features that are ambitious beyond most AAA developers' wildest dreams, look to Toady One. As a bonus, it's a pretty fun game!
- getsat 15y agoAlso, check out Boatmurdered[1], a "Let's Play" of Dwarf Fortress. It is pee-your-pants funny and contains some strong language. The sequel, Headshoots[2], is also funny but not quite the same calibre as Boatmurdered. FYI, the name of your fortress is randomly generated. [1] http://lparchive.org/Dwarf-Fortress-Boatmurdered/Introduction/ http://lparchive.org/Dwarf-Fortress-Boatmurdered/Introductio... [2] http://lparchive.org/Dwarf-Fortress-Headshoots/ http://lparchive.org/Dwarf-Fortress-Headshoots/
- sharmajai 15y agoI recently discovered 'bc'. It stands for 'basic calculator' or more precisely from the man page - 'arbitrary precision arithmetic language'. It is all but a basic calculator, with better floating point precision capabilities than Java/Python.
- personalcompute 15y agosee also: expr
- guard-of-terra 15y agoAnother cool one is xmlstarlet. It's like grep and sed and some on top, but for XML files. Especially, xmlstarlet sel lets you select whatever data you want from XML files, being a command-line XSL templates generator. Don't know anything else useful for XML, so it fills a vacuum.
- jefffoster 15y agoxmllint is another hugely useful xml tool.
- ericmoritz 15y agoAnd all these are replaced by Emacs :p
- cema 15y agoActually, speaking seriously (for a change), I would like to see an agrep for Emacs (approximate search).
- thdn 15y agowhy nmap is not listed?
- lewispb 15y agoMy operating system doesn't have a package manager ;)
- krzysz00 15y agoGet a better operating system, or install Cygwin
- jefffoster 15y agoHere's a few of my favourites. agrep - approximate grep based on edit distance). GNU Global - source tagging system that integrates with the shell (less -tfunc displays the function given in the shell). xmllint - xml validator, pretty printer and schema validator.
- deleted 15y ago[deleted]
- joelthelion 15y agoApparently he hasn't discovered autojump yet :-D
- nl 15y agoautojump looks pretty cool. I assume your name and that of the github implementation isn't a coincidence?
- joelthelion 15y agoIt's not :)
- SandB0x 15y agosl ==== ________ ___________ _D _| |_______/ \__I_I_____===__|_________| |(_)--- | H\________/ | | =|___ ___| _________________ / | | H | | | | ||_| |_|| _| \_____A | | | H |__--------------------| [___] | =| | | ________|___H__/__|_____/[][]~\_______| | -| | |/ | |-----------I_____I [][] [] D |=======|____|________________________|_ __/ =| o |=-~~\ /~~\ /~~\ /~~\ ____Y___________|__|__________________________|_ |/-=|___|| || || || |_____/~\___/ |_D__D__D_| |_D__D__D_| \_/ \__/ \__/ \__/ \__/ \_/ \_/ \_/ \_/ \_/
- rdtsc 15y agotip: try different combinations of options (sl -la, sl -DF, ...)
- fsniper 15y agoonce upon a time I wrote a patch to sl, for s/car/cat/ typo.. you can get it from my site: http://www.yalazi.org/2008/11/08/slssl-ve-scatcar-yamasi/#more-186 http://www.yalazi.org/2008/11/08/slssl-ve-scatcar-yamasi/#mo....
- nickolai 15y agoIts amusing to see nethack and cowsay listed as a full-fledged unix 'tool's.
- selectnull 15y agoYeah, no 'sl' though... :)
- deleted 15y ago[deleted]
- deleted 15y ago[deleted]
- aba_sababa 15y ago______________________________ < cowsay is the epitome of a unix tool. > ------------------------------------------------ \ _ \ (_) \ ^__^ / \ \ (OO)\_____/_\ \ (__)\ ) / ||----w (( || ||>>
- zbowling 15y agocowsay is used on craiglist's 404 page.
- oinksoft 15y ago`ncdu` is a godsend. It's like `du` but usable for troubleshooting. None of the following from this list are obscure: screen, vim, rsync, xargs, curl I'm tempted to put ack in that list.
- pstadler 15y agomytop - a `top` clone for MySQL
- morganpyne 15y agoYou're probably better off with innotop http://sourceforge.net/projects/innotop/files/ http://sourceforge.net/projects/innotop/files/
- dolinsky 15y agoThe new home for innotop is http://code.google.com/p/innotop/ http://code.google.com/p/innotop/, and for anyone looking for other amazing FOSS tools for MySQL (especially Percona builds with XtraDB)- http://www.percona.com/docs/wiki/percona-xtrabackup:start http://www.percona.com/docs/wiki/percona-xtrabackup:start http://www.maatkit.org/ http://www.maatkit.org/ You can do some amazing things with MySQL, Postgres, and Memcached with some of the maatkit tools.
- morganpyne 15y agoAh yep, thanks for that; just Googled it in a hurry and pasted in the first link that came up. Can't agree enough on the Maatkit tools, and I'm also in the process of moving a production database from MySQL to Percona after hearing such good things about it. Xtrabackup for consistent backups of innodb/xtradb live systems + incremental (delta) backups are also very useful.
- dolinsky 15y agoDepending on the version of MySQL that you're upgrading from and if you're upgrading in place vs starting fresh on a new system you won't be able to use Xtrabackup to handle the mysql.proc table due to backwards incompatibilities. I had an awful go of this upgrading from MySQL 5.0.51 to Percona 5.1.56 and wound up having to massage the stored procs and functions in one by one. Once we were migrated over though the xtrabackup tool is great for incremental backups.
- kkovacs 15y agoHi guys, Kristof here, creator of this particular list. First, thanks for the kind words! :) I'll maybe add more tools when I have some time to make more screenshots. It's surprisingly more time consuming than it seems :) Also, I'm a bit of two minds with the whole list -- many people think that some of these tools are already not "obscure" enough, while others suggest adding even more trivial ones like ifconfig or grep. I'll have to think about this a bit :) Once again, thanks for your feedback! KKovacs
- freedrull 15y agoLibcaca and its various programs: http://caca.zoy.org/wiki/libcaca http://caca.zoy.org/wiki/libcaca Also, there is neercs, a terminal multiplexer that uses libcaca: http://caca.zoy.org/wiki/neercs http://caca.zoy.org/wiki/neercs
- newman314 15y agoHere's one that I find does not get a lot of mention but is quite useful when you need it. "tee"
- chow 15y agoMy obscure favorites: yafc: The best command-line FTP client that nobody's ever heard of. Local caching, tab completion, bookmarking, SFTP, and other generally awesome stuff. clex: Full-screen file manager for command-line junkies. Configurable directory display, smart name completion, enhances the command line without seeking to replace it.
- gnosis 15y agoHow does yafc compare to lftp, and how does clex compare to midnight commander?
- evangineer 15y ago+1 on comparing yafc with lftp
- chow 15y agoI switched to yafc from lftp years ago and never looked back, so I can't really compare it to newer versions of lftp. IIRC the biggest wins at the time were context-sensitive tab completion, including support for remote paths, and automatic bookmarking with saved passwords (optional, of course). FTP sucks, yafc makes it tolerable. clex's approach to file management is by complementing the command line; you can generally use it as a pure CLI and it stays out of your way unless until you need to find or select a path. Every action you perform remains on the command line, clex merely makes it easier to quickly build commands. Compared to clex, Midnight Commander (like Norton Commander before it) feels like an attempt at a graphical file manager for the terminal, where the command line is almost an afterthought.
- IProgrammer 15y agoFor those who want to learn how to write their own UNIX tools, and specifically, how to write tools that work well with other UNIX tools, such as the shell and friends, this article may help - Developing a Linux command-line utility: http://www.ibm.com/developerworks/linux/library/l-clutil/ http://www.ibm.com/developerworks/linux/library/l-clutil/
- younata 15y agoEven though these aren't really obscure, but they're worth mentioning as some of the best unix "tools" I've seen: Irssi - irc client Mutt - email client These are mentioned mostly because he included the newsbeuter rss client (which is amazing, I highly recommend it).
- rizumu 15y agoI've discovered a lot of new apps from the post and comments. Give ncmpcpp, an excellent ncurses mpd client, a try and say goodbye to GUI music players. http://unkart.ovh.org/ncmpcpp/screenshots.php http://unkart.ovh.org/ncmpcpp/screenshots.php Also in line with tmux, checkout teamocil and tmuxinator on github.
- ashish_0x90 15y agoGuake/yakuake - A top-down terminal based on the tilda terminal from the game Quake. Redshift/f.lux - Redshift adjusts the color temperature of your screen according to your surroundings.
- kree10 15y agolftp is cool, though I wish FTP would die (as telnet sensibly did over a decade ago) so I will no longer need it.
- evangineer 15y agolftp is an old favorite of mine that i've used many a time. Its built-in shell with history, support for sftp as well as various flavours of ftp, bookmarks are features I've made use of.
- evangineer 15y agoIs it worth mentioning pushd/popd here?