7 ms·

SSH Tricks

172 points by fideloper 12y ago

kyrra 12y ago
The ~/.ssh/config file was something I discovered when my my company was going through some changes and I had 2 different usernames for accessing internal systems. Host hosta hostb hostc User usera Host * User userb Since my local system username was different than many of the remote systems I could wildcard to a different default username then had a list of servers that would use my other username. The bad thing is, as the blog post shows, "Host" above is really just an alias. So if I have an entry like: Host hosta.mycompany.com User usera and then try to do "ssh hosta", even if hosta resolves to hosta.mycompany.com, it won't match the config entry, as config entry data is all used prior to DNS lookup. EDIT: thanks for all the suggestions below.
- cdjk 12y ago
  You should be able to do something like: Host hosta* hostb* hostc* User usera Assuming you don't have two different machines named hosta with different domain names, it should do what you want.
- cmsj 12y ago
  so, I would say that the easiest solution here is tab completion. Suitably modern bash-completion should expand entries in ~/.ssh/config (as well as ~/.ssh/known_hosts if its not hashed). Host hosta.mycompany.com User usera and then type "ssh hosta<tab>" :)
- mpapi 12y ago
  You can do: Host hosta hostb hostc HostName %h.mycompany.com User usera I believe this isn't a super-new feature -- every ssh version I've run into client-side in the last couple years has supported it.
cmsj 12y ago
The chmod commands listed can be chained into a single one: chmod u=rw,go-rwx /path/to/lol
dsirijus 12y ago
Sometimes, if we have a lot of SSH keys in our ~/.ssh directory No, that's not how it's supposed to work. Ideally, one key per machine per user.
- cmsj 12y ago
  Correct, but sometimes it's unavoidable, particularly when talking to dumb management devices :(
- fideloper 12y ago
  Hmmm interesting - do you recommend I use the same key for github, my personal server, some work servers, etc?
  - emilsedgh 12y ago
    Why not?
    - hamburglar 12y ago
      Personally, I like to have different keys that I treat with different levels of care/paranoia. I'm not particularly worried about leaving my github key 'added' to my ssh agent 24/7, but I don't do that with my work production key.
  - cmsj 12y ago
    Yes, but use an RSA key. It doesn't really matter how many you have, you still need to protect them. Encrypt your laptop, lock your screen when you get up for a break, etc, etc.
  - robmccoll 12y ago
    Personally, I use one key pair per source user/device pair and then comment in the authorized keys where each public key is from. To me, it is more likely that a user/machine pair will be compromised than a key. This makes cutting a compromised box or account off easy. :-)
- 12y ago
cmsj 12y ago
see also my post from a couple of years ago: http://www.tenshu.net/2012/02/sysadmin-talks-openssh-tips-and-tricks.html http://www.tenshu.net/2012/02/sysadmin-talks-openssh-tips-an...
bfwi 12y ago
Any list of 'SSH tricks' should contain: 'ssh -D 2001 user@host.com'. This creates a SOCKS proxy on localhost:2001 that goes through host.com. For example, I use a digital ocean instance hosted in the US to tunnel through, so that I can watch hulu (I'm from Europe).
- cmsj 12y ago
  There are also lots of legitimate uses for a SOCKS proxy, such as sshing to a useful machine in a corp network and using something like FoxyProxy to direct *.intranet sites via the SOCKS proxy. It's not all about fricken video piracy :)
  - mey 12y ago
    https://en.wikipedia.org/wiki/Jump_Server https://en.wikipedia.org/wiki/Jump_Server
  - jonemo 12y ago
    Also commonly used in academia when you are not on campus but need access to IP-restricted journals. Or to freak out your lab mates and start the robot from home.
peterwwillis 12y ago
Does anyone read man pages anymore? This is all well documented in the ssh, sshd, and ssh_config man pages...
- cmsj 12y ago
  FWIW, I think it can be helpful to discover features of tools in smaller chunks, in the context of a blog post. man pages can be very dry and it's not always easy to think of all the possible ways some particular flag could help you. I'm not necessarily defending this particular post, just making a general point about the more esoteric features of something like ssh :)
- fideloper 12y ago
  I guess not, since people decided to upvote! Personally my favorite part is the logical the tie-in to Ansible, which I want to expand upon in a near-future edition.
- Zikes 12y ago
  You ever read the man pages for less? Actually apparently it doesn't have a man page, I had to use less --help. Anyways, it turns out there's a ton of functionality I never thought to look for. I only ever see "| less" and that's all it ever was to me. There's nothing more I really expected out of it than that, really. But that's just how it goes, for the most part. You learn the basics about how to use a given tool, enough to serve the purpose you originally sought it out for, and then that's it, everything else is just noise. less lets me easily scroll through whatever output I pipe to it, ssh gets me onto another server, what more would I need or expect? But then an article like this comes along and prompts me to look more closely at something I had been taking for granted, showing it to be much more versatile than I thought.
  - mooism2 12y ago
    Less has a manpage. http://manpages.ubuntu.com/manpages/trusty/en/man1/less.1.html http://manpages.ubuntu.com/manpages/trusty/en/man1/less.1.ht...
Argorak 12y ago
The list misses one of the best. Connect through a jump host directly to the target server using ProxyCommand and nc: http://undeadly.org/cgi?action=article&sid=20070925181947 http://undeadly.org/cgi?action=article&sid=20070925181947 Combines well with aliases - prefix all hosts with a common name and use "Host prefix-*" to setup the ProxyCommand.
- mahmoudimus 12y ago
  You do not need nc anymore. -W flag intelligently forwards traffic and interpolates correctly
  - stock_toaster 12y ago
    Requires openssh > 6.0 to work consistently with controlpersist. I still have problems on Centos 6.x boxes, so I still use nc in proxycommand for those hosts. https://news.ycombinator.com/item?id=4678117 https://news.ycombinator.com/item?id=4678117
    - qzervaas 12y ago
      I have the following in my ~/.ssh/config, which for those who don't know, maintains the connection even after you disconnect so you can reconnect really quickly: ControlMaster auto ControlPath /tmp/%r@%h:%p ControlPersist yes Having said that, sometimes I need to remove the entry from /tmp to reconnect if my network settings have changed.
      - stock_toaster 12y ago
        > Having said that, sometimes I need to remove the entry > from /tmp to reconnect if my network settings have changed. You can also just send the exit command to the conn. ssh somehost -O exit In addition, I put my controlpath in my .ssh dir. Keeps it out of the global /tmp dir. shrug controlpath ~/.ssh/cp-%r:%h:%p
        12y ago
baldfat 12y ago
Dynamic port forwarding with ssh is a life saver! Not on this web page??? -D #### (Whatever port not being used) and then I just use a proxy extension on my web client and instant privacy.
welder 12y ago
Also check out proxychains for tunneling through multiple ssh servers, for example to ssh into an intranet machine via the internet: https://github.com/haad/proxychains https://github.com/haad/proxychains
- kneth 12y ago
  Often (non-IT) companies' firewalls do not allow anything but HTTP and HTTPS traffic and you have to go through proxies. That implies that you cannot get to the outside using SSH. In my days as a freelance consultant, I used Corkscrew (http://www.agroman.net/corkscrew/ http://www.agroman.net/corkscrew/) to get SSH access.
JoshTheGeek 12y ago
There's been a bunch of articles about ssh that all say more or less the same things recently.
- 1amzave 12y ago
  ...and which essentially amount to "hey, look what's in the man page!".
aaronharnly 12y ago
At the urging of the tech lead at the time, my team has pursued the ultimate ssh trick: not using it. i.e. we intentionally do not enable (human) ssh access to the production hosts. In an autoscaling AWS world, logging into an individual machine by hand is the last thing you want to be doing. So we are learning the (sometime difficult!) lessons of how to rely only on what our logging, tracing, monitoring, and deployment automation (including snapshotting) can afford us. I suspect sooner or later we will break down and swap in a login-enabled image to diagnose some sticky problem, but -- as much as I resented the idea when he presented it -- it's an interesting discipline. Anyone else living by that principle?
- fideloper 12y ago
  I like the idea of it, given how expendable servers are. I'd love to hear any instances where you have SSH'ed in, or what kind of bugs would prompt you to want to. Maybe because you didn't have some monitoring on a specific aspect of the server, or an application bug where you needed to use strace - something like that?
- michaelbuckbee 12y ago
  Until I read your comment, I hadn't really thought about it, but this type of automation is at least mostly enforced by PaaS like Heroku.
- X-Istence 12y ago
  This is the same principle I have started to use with Docker. It runs a single application, stuff logs, but besides that there is no way to get into it, no ssh, nothing.
- js2 12y ago
  This is mostly how we ran things at Rockmelt. 50+ VMs.
- ProAm 12y ago
  This is great, I think at first glance it works at scale, but I think a lot of shops not in the 'startup world', older IT shops still working out of the 90s and 2000's will find that hard to do. When you have 5 monster work horse servers instead of 50 VMs where you can spin up 50 more within 30 minutes it's hard to get away from direct server access for the devops team.
lucb1e 12y ago
One interesting bit is that ssh aliases also work for scp. For example I have ssh keys setup on a VPS and to copy a file I can type scp vps:~/www/back<tab> and it'll autocomplete to scp vps:/home/vhosting/c/vhost12345/www/backup-2014-07-02.tar.gz Or if there are multiple matches for file or directory names, it'll list them like bash normally would. This seamless integration is so awesome, I can highly recommend ssh keys (and cygwin for Windows users).
- notfoss 12y ago
  SSH aliases work for SFTP as well.
geerlingguy 12y ago
I liked the mention of Ansible (which thankfully abstracts away the need to log into a server via SSH altogether), but the author left out the fact that you can easily use any ansible module (250+ right now, more added all the time[1]) to manage your servers ad-hoc. Or use the same syntax to build a playbook that you can run to manage infrastructure with the `ansible-playbook` command. Since Ansible uses SSH as it's transport (in most cases—you can do it other ways), if you can connect to a server via SSH (and who can't?), you can have it completely managed/version-controlled pretty simply. [1] http://docs.ansible.com/list_of_all_modules.html http://docs.ansible.com/list_of_all_modules.html
sciurus 12y ago
I'd be inclined to call this "basic usage of ssh", rather than "ssh tricks". For some more interesting tips see http://www.jedi.be/blog/2010/08/27/ssh-tricks-the-usual-and-beyond/ http://www.jedi.be/blog/2010/08/27/ssh-tricks-the-usual-and-...
robmccoll 12y ago
sshuttle (https://github.com/apenwarr/sshuttle https://github.com/apenwarr/sshuttle) is one of my favorite SSH tricks. It behaves like a VPN more than other ssh-based proxies that I've used.
citrik 12y ago
If the author is reading this... You have a typo on your example config file, your line for aws says "How aws" instead of "Host aws".
therealidiot 12y ago
Nobody ever seems to show off the '-w' flag for OpenSSH in these articles. It creates a virtual network interface and allows for "real" tunneling, which is pretty cool.

172 points by fideloper 12y ago

kyrra 12y ago

The ~/.ssh/config file was something I discovered when my my company was going through some changes and I had 2 different usernames for accessing internal systems. Host hosta hostb hostc User usera Host * User userb Since my local system username was different than many of the remote systems I could wildcard to a different default username then had a list of servers that would use my other username. The bad thing is, as the blog post shows, "Host" above is really just an alias. So if I have an entry like: Host hosta.mycompany.com User usera and then try to do "ssh hosta", even if hosta resolves to hosta.mycompany.com, it won't match the config entry, as config entry data is all used prior to DNS lookup. EDIT: thanks for all the suggestions below.

cdjk 12y ago
You should be able to do something like: Host hosta* hostb* hostc* User usera Assuming you don't have two different machines named hosta with different domain names, it should do what you want.
cmsj 12y ago
so, I would say that the easiest solution here is tab completion. Suitably modern bash-completion should expand entries in ~/.ssh/config (as well as ~/.ssh/known_hosts if its not hashed). Host hosta.mycompany.com User usera and then type "ssh hosta<tab>" :)
mpapi 12y ago
You can do: Host hosta hostb hostc HostName %h.mycompany.com User usera I believe this isn't a super-new feature -- every ssh version I've run into client-side in the last couple years has supported it.

cmsj 12y ago

The chmod commands listed can be chained into a single one: chmod u=rw,go-rwx /path/to/lol

dsirijus 12y ago

Sometimes, if we have a lot of SSH keys in our ~/.ssh directory No, that's not how it's supposed to work. Ideally, one key per machine per user.

cmsj 12y ago
Correct, but sometimes it's unavoidable, particularly when talking to dumb management devices :(
fideloper 12y ago
Hmmm interesting - do you recommend I use the same key for github, my personal server, some work servers, etc?
- emilsedgh 12y ago
  Why not?
  - hamburglar 12y ago
    Personally, I like to have different keys that I treat with different levels of care/paranoia. I'm not particularly worried about leaving my github key 'added' to my ssh agent 24/7, but I don't do that with my work production key.
- cmsj 12y ago
  Yes, but use an RSA key. It doesn't really matter how many you have, you still need to protect them. Encrypt your laptop, lock your screen when you get up for a break, etc, etc.
- robmccoll 12y ago
  Personally, I use one key pair per source user/device pair and then comment in the authorized keys where each public key is from. To me, it is more likely that a user/machine pair will be compromised than a key. This makes cutting a compromised box or account off easy. :-)
12y ago

cmsj 12y ago

see also my post from a couple of years ago: http://www.tenshu.net/2012/02/sysadmin-talks-openssh-tips-and-tricks.html http://www.tenshu.net/2012/02/sysadmin-talks-openssh-tips-an...

bfwi 12y ago

Any list of 'SSH tricks' should contain: 'ssh -D 2001 user@host.com'. This creates a SOCKS proxy on localhost:2001 that goes through host.com. For example, I use a digital ocean instance hosted in the US to tunnel through, so that I can watch hulu (I'm from Europe).

cmsj 12y ago
There are also lots of legitimate uses for a SOCKS proxy, such as sshing to a useful machine in a corp network and using something like FoxyProxy to direct *.intranet sites via the SOCKS proxy. It's not all about fricken video piracy :)
- mey 12y ago
  https://en.wikipedia.org/wiki/Jump_Server https://en.wikipedia.org/wiki/Jump_Server
- jonemo 12y ago
  Also commonly used in academia when you are not on campus but need access to IP-restricted journals. Or to freak out your lab mates and start the robot from home.

peterwwillis 12y ago

Does anyone read man pages anymore? This is all well documented in the ssh, sshd, and ssh_config man pages...

cmsj 12y ago
FWIW, I think it can be helpful to discover features of tools in smaller chunks, in the context of a blog post. man pages can be very dry and it's not always easy to think of all the possible ways some particular flag could help you. I'm not necessarily defending this particular post, just making a general point about the more esoteric features of something like ssh :)
fideloper 12y ago
I guess not, since people decided to upvote! Personally my favorite part is the logical the tie-in to Ansible, which I want to expand upon in a near-future edition.
Zikes 12y ago
You ever read the man pages for less? Actually apparently it doesn't have a man page, I had to use less --help. Anyways, it turns out there's a ton of functionality I never thought to look for. I only ever see "| less" and that's all it ever was to me. There's nothing more I really expected out of it than that, really. But that's just how it goes, for the most part. You learn the basics about how to use a given tool, enough to serve the purpose you originally sought it out for, and then that's it, everything else is just noise. less lets me easily scroll through whatever output I pipe to it, ssh gets me onto another server, what more would I need or expect? But then an article like this comes along and prompts me to look more closely at something I had been taking for granted, showing it to be much more versatile than I thought.
- mooism2 12y ago
  Less has a manpage. http://manpages.ubuntu.com/manpages/trusty/en/man1/less.1.html http://manpages.ubuntu.com/manpages/trusty/en/man1/less.1.ht...

Argorak 12y ago

The list misses one of the best. Connect through a jump host directly to the target server using ProxyCommand and nc: http://undeadly.org/cgi?action=article&sid=20070925181947 http://undeadly.org/cgi?action=article&sid=20070925181947 Combines well with aliases - prefix all hosts with a common name and use "Host prefix-*" to setup the ProxyCommand.

mahmoudimus 12y ago
You do not need nc anymore. -W flag intelligently forwards traffic and interpolates correctly
- stock_toaster 12y ago
  Requires openssh > 6.0 to work consistently with controlpersist. I still have problems on Centos 6.x boxes, so I still use nc in proxycommand for those hosts. https://news.ycombinator.com/item?id=4678117 https://news.ycombinator.com/item?id=4678117
  - qzervaas 12y ago
    I have the following in my ~/.ssh/config, which for those who don't know, maintains the connection even after you disconnect so you can reconnect really quickly: ControlMaster auto ControlPath /tmp/%r@%h:%p ControlPersist yes Having said that, sometimes I need to remove the entry from /tmp to reconnect if my network settings have changed.
    - stock_toaster 12y ago
      > Having said that, sometimes I need to remove the entry > from /tmp to reconnect if my network settings have changed. You can also just send the exit command to the conn. ssh somehost -O exit In addition, I put my controlpath in my .ssh dir. Keeps it out of the global /tmp dir. shrug controlpath ~/.ssh/cp-%r:%h:%p
      - 12y ago

baldfat 12y ago

Dynamic port forwarding with ssh is a life saver! Not on this web page??? -D #### (Whatever port not being used) and then I just use a proxy extension on my web client and instant privacy.

welder 12y ago

Also check out proxychains for tunneling through multiple ssh servers, for example to ssh into an intranet machine via the internet: https://github.com/haad/proxychains https://github.com/haad/proxychains

kneth 12y ago
Often (non-IT) companies' firewalls do not allow anything but HTTP and HTTPS traffic and you have to go through proxies. That implies that you cannot get to the outside using SSH. In my days as a freelance consultant, I used Corkscrew (http://www.agroman.net/corkscrew/ http://www.agroman.net/corkscrew/) to get SSH access.

JoshTheGeek 12y ago

There's been a bunch of articles about ssh that all say more or less the same things recently.

1amzave 12y ago
...and which essentially amount to "hey, look what's in the man page!".

aaronharnly 12y ago

At the urging of the tech lead at the time, my team has pursued the ultimate ssh trick: not using it. i.e. we intentionally do not enable (human) ssh access to the production hosts. In an autoscaling AWS world, logging into an individual machine by hand is the last thing you want to be doing. So we are learning the (sometime difficult!) lessons of how to rely only on what our logging, tracing, monitoring, and deployment automation (including snapshotting) can afford us. I suspect sooner or later we will break down and swap in a login-enabled image to diagnose some sticky problem, but -- as much as I resented the idea when he presented it -- it's an interesting discipline. Anyone else living by that principle?

fideloper 12y ago
I like the idea of it, given how expendable servers are. I'd love to hear any instances where you have SSH'ed in, or what kind of bugs would prompt you to want to. Maybe because you didn't have some monitoring on a specific aspect of the server, or an application bug where you needed to use strace - something like that?
michaelbuckbee 12y ago
Until I read your comment, I hadn't really thought about it, but this type of automation is at least mostly enforced by PaaS like Heroku.
X-Istence 12y ago
This is the same principle I have started to use with Docker. It runs a single application, stuff logs, but besides that there is no way to get into it, no ssh, nothing.
js2 12y ago
This is mostly how we ran things at Rockmelt. 50+ VMs.
ProAm 12y ago
This is great, I think at first glance it works at scale, but I think a lot of shops not in the 'startup world', older IT shops still working out of the 90s and 2000's will find that hard to do. When you have 5 monster work horse servers instead of 50 VMs where you can spin up 50 more within 30 minutes it's hard to get away from direct server access for the devops team.

lucb1e 12y ago

One interesting bit is that ssh aliases also work for scp. For example I have ssh keys setup on a VPS and to copy a file I can type scp vps:~/www/back<tab> and it'll autocomplete to scp vps:/home/vhosting/c/vhost12345/www/backup-2014-07-02.tar.gz Or if there are multiple matches for file or directory names, it'll list them like bash normally would. This seamless integration is so awesome, I can highly recommend ssh keys (and cygwin for Windows users).

notfoss 12y ago
SSH aliases work for SFTP as well.

geerlingguy 12y ago

I liked the mention of Ansible (which thankfully abstracts away the need to log into a server via SSH altogether), but the author left out the fact that you can easily use any ansible module (250+ right now, more added all the time[1]) to manage your servers ad-hoc. Or use the same syntax to build a playbook that you can run to manage infrastructure with the `ansible-playbook` command. Since Ansible uses SSH as it's transport (in most cases—you can do it other ways), if you can connect to a server via SSH (and who can't?), you can have it completely managed/version-controlled pretty simply. [1] http://docs.ansible.com/list_of_all_modules.html http://docs.ansible.com/list_of_all_modules.html

sciurus 12y ago

I'd be inclined to call this "basic usage of ssh", rather than "ssh tricks". For some more interesting tips see http://www.jedi.be/blog/2010/08/27/ssh-tricks-the-usual-and-beyond/ http://www.jedi.be/blog/2010/08/27/ssh-tricks-the-usual-and-...

robmccoll 12y ago

sshuttle (https://github.com/apenwarr/sshuttle https://github.com/apenwarr/sshuttle) is one of my favorite SSH tricks. It behaves like a VPN more than other ssh-based proxies that I've used.

citrik 12y ago

If the author is reading this... You have a typo on your example config file, your line for aws says "How aws" instead of "Host aws".

therealidiot 12y ago

Nobody ever seems to show off the '-w' flag for OpenSSH in these articles. It creates a virtual network interface and allows for "real" tunneling, which is pretty cool.