12 ms·
Devops/Sysadmin Cheatsheet
- patja 14y agoThanks for this nice list. Small nitpick/question: why put the commands in inputs? Do you edit them on that page? Or is it just to format them? I tried to do a copy/paste on the page contents to my personal offline notebook but the most important bits, the commands, didn't paste.
- sudara 14y agoAck! I'm sorry! I started out with "pre" elements, but they were causing problems with copying and pasting (it looked odd, and it added a dreadful newline so the command would execute on paste!) The plan is to have a pdf shortly; that will most definitely be copy and pasteable.
- sudara 14y agoStill working on it, but: https://www.dropbox.com/s/a0rx9c28euwokaz/cheatsheet-pdf-version-1.pdf https://www.dropbox.com/s/a0rx9c28euwokaz/cheatsheet-pdf-ver...
- sudara 14y agoThis is just a small curated collection from myself and old friends — mainly rails devops. I'd be interested in hearing from other rails devs/ops what you use on the command line on a daily basis, what cool thing you know about that no one else does, etc! I'll be adding more as we go....
- DougN7 14y agoIt would be nice if the title mentioned it was rails focused.
- wlll 14y agoI didn't mention Rails in the title because most of the commands aren't Rails specific.
- zackkitzmiller 14y agoIt's not. It's a poor title. Nothing there is Rails specific.
- sudara 14y agoSorry about that! I didn't submit the link otherwise I would have mentioned it. Though...the only way it's really "rails focused" is that the commands are collected over time and sourced from my fellow rails developer/sysadmins. A lot of these are basic/general linux commands.
- eggsby 14y agoFor getting around linux I've found no better helpers than manpages and the cheatsheets by Peteris Krumins[1]. I notice this isn't offered as a pdf or image, is the idea that someone will come and visit the site when their server is in trouble? That said, I didn't know about some stuff here like scriptreplay -- thanks! [1]: http://www.catonmat.net/projects/cheat-sheets/ http://www.catonmat.net/projects/cheat-sheets/
- nicholassmith 14y agoI'd second a quick pdf version of it as well, I'm sure more than a few of us have a directory with cheatsheets in somewhere on their drive as a 'just in case'.
- crymer11 14y agohttp://pdfcrowd.com/ http://pdfcrowd.com/ Note: not affialated with them; just one of the top Google results. Also, I'd take a screenshot (and then export to pdf if an image file wasn't acceptable) since the pdf conversion via pdfcrowd splits it into multiple pages.
- sudara 14y agoHere's a work in progress, still needs some loooovvvee: https://www.dropbox.com/s/a0rx9c28euwokaz/cheatsheet-pdf-version-1.pdf https://www.dropbox.com/s/a0rx9c28euwokaz/cheatsheet-pdf-ver...
- nicholassmith 14y agoNice work!
- deleted 14y ago[deleted]
- mben 14y agoCheck also the Unix Toolbox: http://cb.vu/unixtoolbox.xhtml http://cb.vu/unixtoolbox.xhtml I've got a copy of the pdf in my dropbox just in case.
- adamgravitis 14y agoThat's not devops - that's system administration. Not complaining; it's still a well-presented list.
- peterwwillis 14y agoSeriously! The term is becoming useless when it's just a synonym for 'sysadmin' or 'random linux cruft'
- sudara 14y agoThanks! You are completely right. Just basic sysadmin stuff. It's titled that way simply a) I thought I'd see if I could hang with the term and b) the "source" of these commands (including myself) are from developers who specialize in rails ops work (so we do write app code, but specialize on the server side)
- cwh 14y agoI would warn anyone that read this to understand what each command does before actually running anything on a machine. Things like "sudo !!" are INCREDIBLY dangerous, and I would never put that on a cheat sheet.
- ericcholis 14y agoTruthfully, I'm more comfortable using the up arrow and re-running it manually. At least I can inspect what the command was before running it. Obviously, this doesn't help when using Fabric or scripting, but you shouldn't need "sudo !!" in those scenarios.
- mhurron 14y agoAnyone running commands blindly without knowing what they do deserves what they get. That goes for potentially dangerous commands or not. Or put in a slightly nicer way - Blindly running commands is going to turn into a learning experience, don't do it on a production machine.
- jiggy2011 14y agoI suppose if you type it at any random point it might be. It's useful more when typed something that you forgot to sudo.
- darkarmani 14y agoIf you are running sudo, the long-term payoff is probably higher if you never use sudo !!. The very small chance of a huge disaster is still a lot higher than up arrow. Unless you are on a laggy connection and up arrow delays and you push it twice and end up running the wrong command! If you have lots of jitter and lag, i suggest using Mosh which I've found to be amazingly awesome when using it on Amtrak's spotty free wifi (3G uplink).
- ineedtosleep 14y agoI've never had any incidents with `sudo !!`, however, I should point out that zsh (maybe other shells as well) replaces the `!!` with the full command and requires another press of Enter. $ rm -rf /root rm: cannot remove `root': Permission denied $ sudo !! # Does not run this, but returns the line below. $ sudo rm -rf /root # Then you say, "Wait I dont want to do this."
- fduran 14y agoA little gotcha: "iptables -L" doesn't list the NAT table rules; do "iptables -L; iptables -t nat -L" instead.
- js2 14y agoI also almost always add '-n' to iptables commands to disable reverse name resolution, which can sometimes take a while. Similarly when calling netstat.
- Florin_Andrei 14y agoI use -n with almost any network-related command. Probably overreacting, but I hate to just sit there staring at a black rectangle.
- lflux 14y agoI always use 'iptables -L -n -v --line-numbers' - verbose will show you stuff like which interface the rule is set to, and line numbers can be useful when adding a rule before another with 'iptables -I'
- thyrsus 14y agoMe too. Even more important is that a host on multiple networks may have the same PTR value for each address, and thus one can lose information in the reverse lookup. "host w.x.y.z" ( or Solaris "nslookup -type=ptr z.y.x.w.in-addr.arpa" ) is available if you don't recognize the IP address.
- X-Istence 14y agodig -x <ip> Will also correctly do a PTR query for the reverse entry for that IP. Also functions correctly with IPv6, so you don't need to remember to split the IPv6 address up into a lot of dots :P
- js2 14y agoSome random thoughts: - Instead of 'while true;' you can use the shorter 'while :;'. ':' is a null command. - On OS X, dtruss is kinda, sorta like strace (it's a wrapper around dtrace, and the dtruss name comes from Solaris). - For basic host to IP resolution, I prefer ping as it calls gethostbyname(), like most other programs will do. host/dig are suitable for querying/testing DNS independent of how the local box is configured as they call the resolver library directly. For example, host bypasses nsswitch.conf. (Separately, OS X's stub resolver has some neat tricks. e.g., using /etc/resolver/<domain_name> to route DNS requests for a particular domain to a specific name server.) - I had never heard of http://michael.toren.net/code/tcptraceroute/ http://michael.toren.net/code/tcptraceroute/ before. I don't think I've ever encountered a situation where the issue was outbound ICMP/UDP packets being blocked, but rather it's the return ICMP Time Exceeded packets. - 'find . -size +100M' is shorter than 'find ./ -size +100000000c -print'. - ctime is NOT when a file was created, though this is a common misconception. Unix does not record when a file is created. Rather, ctime is the last time the file's status (i.e. inode information aka metadata) was changed. This differs from mtime which records the last time the file's data was changed. ctime is a superset of mtime. atime is the last time a file was accessed, unless the filesystem is mounted with noatime (common for NFS file ssystems). atime/mtime can be set to arbitrary values (permissions allowing) via the utime() system call. - Modern find supports -delete instead of '-exec rm {} \;'. In any case, my muscle memory still defaults to '-print0 | xargs -0 rm'. - 'dd if=/dev/zero of=file.txt count=1024 bs=102' seems like an odd way to do that. I think that 'bs=1024 count=102' might be more efficient depending upon buffering?
- sudara 14y agoTHANK YOU for the insight and effort. Enlightening. Updated the find, MUCH better. Will play with the others and modify the sheet as I go!
- cparedes 14y agoYou can use TCP/UDP/GRE for your traceroute packets with the -P switch: traceroute -P udp <hostname> This is super useful if you're probing a network that's hostile to ICMP packets - it's a lot more reliable than ICMP in many scenarios, from my experience.
- 14y ago
- ghjm 14y agoOkay, I'll admit it. I don't actually know what devops is. I know what developers do and what sysadmins do. Is devops just a buzzword for one person who can do both? Or does it mean something other than that?
- falcolas 14y agoI don't really know how to state what we do aside from "Writing the glue/lube between the application and the hardware." Deployment automation, monitors, backups, cleanup...
- deleted 14y ago[deleted]
- darkarmani 14y agoI will admit that I have a fuzzy understanding as well. My take on it is a holistic view and integration between development and operations instead of formal or informal walls between the two. Wikipedia: http://en.wikipedia.org/wiki/Devops http://en.wikipedia.org/wiki/Devops In small companies it usually always exists by accident. I think the "hype" is more around large companies that have huge barriers and sometimes friction between development and IT or sustaining Operations.
- ajdecon 14y agoAt the moment, it's mostly an overloaded buzzword. :) The best definition I've found is that it's a culture of collaboration and integration between development and operations, rather than a role for one specific person. Ben Rockwood from Joyent has a good conceptual talk at http://www.youtube.com/watch?v=h5E--QSBVBY http://www.youtube.com/watch?v=h5E--QSBVBY , "DevOps Demysitified".
- sudara 14y agoI'm the author of this cheatsheet and I have no clue really. Just wanted to try the word out, see if it felt right. But yes, my impression is that it's normally a developer who "trends towards doing systems administration" – so someone who knows the codebase, but can provision and wrangle servers. So, someone like me (I write full stack apps, but my "speciality" is the server end of things).
- falcolas 14y agoGreat list. A few more (related or building on those there) Quick mysql "why is my database under so much load suddenly" that doesn't rely on the slow query log: pt-query-digest --processlist h=<host> --interval=0.01 --print > /tmp/queries.out pt-query-digest /tmp/queries.out Tracking down what files exactly are filling up the disk. Go to /, and follow the big numbers: du -smc * Combine strace (for the file handle no), lsof (for the file handle to ip/port) and netstat (to find the process on the remote host) to track RPC calls that are hung. For 'ps auxww', throw on an f to get a visual representation of child processes. And finally, ssh tunnels (which can be chained together) to get past bastion servers easily: Host <alias> HostName <ip> ProxyCommand ssh -q <bastion host, can be another alias in .ssh/config> "nc -w 3600 %h %p"
- AaronBBrown 14y agoI prefer using tcpdump instead of processlist when dealing with pt-query-digest. You get everything instead of just what happens to show up during the interval specified. This is, of course, assuming that you don't allow localhost socket connections.
- falcolas 14y agoVery true - extremely fast queries can slip between the processlist checks. The problem I've encountered with tcpdump is that you can cause packet loss by running it. It's not common, but it happens enough that I've become gun shy about using it on production systems. The pt-query-digest function will cause additional load on the DB, but it won't interrupt communications.
- X-Istence 14y agoWhat OS will drop packets while tcpdump is running?
- falcolas 14y agoLinux. :) If tcpdump can't keep up with the incoming traffic, the kernel will drop the packets in its buffer (or rather overwrite them with new packets). Throw in TCP's flow and congestion control protocols, and dropped packets can have disastrous effects on your database. Google has many references you might find useful on this subject.
- roozbeh18 14y agoChecks the speed of the network. if you are looking to good connectivity on your VPS. wget cachefly.cachefly.net/100mb.test -O /dev/null check the write speed of the disk. Mostly used to check what kind of a write speed you get on the machine. dd if=/dev/zero of=iotest bs=64k count=16k conv=fdatasync && rm -rf iotest
- njharman 14y ago[Edit oh gads I see now the commands are in text boxes and the full command is not visible. It is "ps aux | head -1 && ps aux | sort -k 4 -nr | head" which maybe correct is even more ridiculous.] > ps aux | head -1 && ps aux | sort Is a wasteful construct and doesn't even do what is claimed "List the top 10 memory hogs". Depending on what you consider "memory" something like this is shorter and correct. ps aux --sort=-resident|head -11 Other errors: They list same command "du -hs" for twice. I believe they meant "df -h" for "overview of all disks". Although, it's correctly overview of mounted file systems.
- sudara 14y agoThanks! I've updated that with your version, much MUCH more concise. (Also fixed the df thing, definitely copy paste error!)
- julian37 14y agoSmall typo, "Overview of all disks" should be df -h not du -sh
- sudara 14y agoThanks for noting. Last minute copy/paste fail :(
- 3amOpsGuy 14y agoThere's a few typos in this, in case it helps: ps auxww -> ps auxww -H (H is hierarchy) Faster than lsof, and only displays files - although most things in unix are files! lsof -p -> ls -l /proc/$PID/fd Run something forever watch command Overview of all disks du should be df. Find files over 100mb find . -size +100M Low hanging fruit for size ls -al | sort -nk5 Files created (modified) within the past 7 days: find . -mtime -7 Find files older than 14 days: find .gz -mtime +14 -type f This will break when you have more archived files in the directory than the shell's glob char can support. Use: find . -mtime +14 -type f -name '*.gz' and it will run quicker too. TCP Sockets in use, "netstat -antp" will be faster and also lists the process id. EDIT: formatting
- sudara 14y agoThanks for catching the ps aux and for the refinements! Deploying new version now...
- activepeanut 14y agoInstead of: ls -al | sort -nk5 I suggest: ls -larS
- 3amOpsGuy 14y agoI upvoted you the other day and it seems coming back today i have the upvote option again? Weird. Either way, have an upvote!
- joshdotsmith 14y agoJust a UX suggestion, but I would highlight the text on the input focus for easy copy/paste.
- sudara 14y agoGood point. I considered doing this, but then it ends up being a "surprise" to users when they quickly go to triple-click (select all) and the interaction is different than expected. What I find really useful is you can immediately type some characters to filter, and then tab to the command you want. It's automatically selected in that case. I'll likely stick a "copy" button on hover shortly — had some initial problems with z-indexes (it uses flash) but this solves the "one click to copy" issue without affecting the default and expected interaction.
- jthomp 14y agoNice list
- sudara 14y agoThank you!
- AaronBBrown 14y agoHere's one I like for testing bandwidth between two machines: host1$ while : ; do nc -l 6666 > /dev/null; done host2$ pv /dev/zero | nc host1 6666 156MiB 0:00:17 [9.46MiB/s] [ <=> ]
- imbriaco 14y agoYou should check out iperf as well.
- AaronBBrown 14y agoLooks like a useful tool. Thanks!
- thaumaturgy 14y agodu -h --max-depth=1 -x I use this frequently to keep an eye on disk used by user directories. It returns the disk space used by each directory in the current directory.
- falcolas 14y agoSo does: du -sh *
- thaumaturgy 14y agoCorrect with one important caveat: because * is a shell expansion, it will not process any directories starting with a ".". e.g., imap maildir directories.
- X-Istence 14y agodu -sh .* * Problem solved.
- thaumaturgy 14y agoYou might be right. I have some recollection of .* stabbing me in the back once before in this command, and I know I put -x (do not traverse file systems) in there for a reason -- but I don't have any notes on why, so I can't say anything smart about it.
- andrewaylett 14y agoThe trouble with that is that it matches '.' and '..', which you probably don't want. I don't tend to have a single character after the dot in a name, so I usually use '.??*'.
- falcolas 14y agoThat's fair, but for the proposed use, checking the size of home directories, .* directories are of little concern. Honestly, when I have to get fine grained enough to dig into disk usage using 'du', I'm rarely concerned with hidden directories (and I typically append a 'c' to the command as well, so if I do have to pay attention to . directories, I will notice the discrepancy between reported sizes).
- jlkinsel 14y agoThis is a UNIX cheatsheet. Nothing devops about it. And I concur with the comments about "sudo !!" - !! in general I've totally removed from my UNIX vocabulary. At least do something like !?string[?]
- X-Istence 14y agoDevops/Sysadmin Cheatsheet for Linux Most of the commands listed rely on various GNU extensions to the various utilities or are only applicable to Linux. FreeBSD: strace -> dtrace/ktrace/ lsof -> sockstat/fstat watch -> no idea sudo -> Almost never installed by default on FreeBSD ps aux --sort=-resident|head -11 -> --sort is not valid ... And the list goes on ...
- vacri 14y agoIf you're going down that path, add windows servers to your list and find more things missing.
- X-Istence 14y agoI don't know a lot of people running Ruby services on top of Windows ...
- smoofles 14y agoI don’t know a lot of people running Ruby services on FreeBSD, either. I’m pretty sure I know one or two who, at least, develop on Windows. But, yeah, noone’s stopping you to make your FreeBSD version of the cheatsheet, what with the open source spirit and all… :P
- tszming 14y agoBe careful when you think you really understand the meaning of atime/mtime of findutils. E.g. man find -atime n File was last accessed n*24 hours ago. When find figures out how many 24-hour periods ago the file was last accessed, any fractional part is ignored, so to match -atime +1, a file has to have been accessed at least two days ago.
- reinhardt 14y agoMeta question: where do you actually learn this stuff? I am a developer with an intermediate desktop level command line knowledge learnt mostly on the job but would appreciate a crash course on system-level command line, diagnostics, devops, etc.
- eLobato 14y agoIn my experience, you pretty much learn this on the fly. You don't know how to do X, you google it, if you do X often enough, you'll eventually learn. After a while you end up mastering most of the unix suite (ls, grep, awk, du, etc..) so instead of googling stuff you write it. Unlike fundamentals (think books like SICP, Introduction to Algorithms, K&R, the Dragon book, et al), this is just a collection of useful commands that do not bring barely any collateral learning aside from learning the 'UNIX' way. If you really want to put up the effort to learn this, and you don't have any projects or anything that requires this knowledge, I've noticed lots of tech offices have a copy of UNIX in a nutshell always have a copy around. I've checked it out myself and it's pretty useful (I already 'know'), not sure how good is it for learning this stuff from the ground up.
- sudara 14y agoThis is a great reply. "On the fly" is definitely my experience too. Needing to do something. Doing things slowly, inefficiently, with googling until you find or get told the more efficient way of doing things. Also, talking with/demanding explanations/debating with geekier than thou friends has been really helpful with the bigger concepts.
- exDM69 14y agoLearning by doing is probably the way most people learn their unixcraft. It also helps if you have a friend or a team member who is a lot more experienced than you, a go-to guy with infinite patience to your stupid questions. I had the privilege of working with a badass russian unix hacker for some time, he taught me how to do black magic with find, grep, sort and uniq.
- 14y ago
- royblow 14y agothis is a computer operator cheatsheat. there are no systems administration tasks being implemented here, just basic unix.