7 ms·

> What are your favorite SSH tips & tricks? $ ssh -J user1@host1 user_final@host_final or $ ssh -J user1@host1,user2@host2 user_final@host_final Not

> What are your favorite SSH tips & tricks? $ ssh -J user1@host1 user_final@host_final or $ ssh -J user1@host1,user2@host2 user_final@host_final Not many people know it, you don't need to launch a SSH within a SSH session - SSH has built-in support of using one SSH server as a proxy to another SSH server. Useful for <del>hacking servers</del> accessing servers behind a firewall, or using your own server as a proxy to bypass a bottleneck in the network.

nprescott 7y ago
This is a good one. I've used this in the past in order to get onto IPv6-only networks as well. In my case I don't have IPv6 enabled on my home internet (thanks Verizon!) and I had a tiny virtual machine with Vultr, which at their lowest price point aren't offering IPv4 address space any more. Using a jump through another machine with both 4 and 6 address space saved me from having to cough up more money solely for a IPv4 address.
- _wldu 7y ago
  Use teredo at home. apt install miredo (on most debian based systems) https://en.wikipedia.org/wiki/Teredo_tunneling https://en.wikipedia.org/wiki/Teredo_tunneling
  - segfaultbuserr 7y ago
    You can also use Tor as IPv6 proxy in a pure IPv4 network (or as IPv4 proxy in a pure IPv6 network), recent versions of Tor can work under pure IPv6, gaining privacy and connectivity simultaneously. The speed is not actually too bad for web browsing, although not ideal for SSH. But still comes handy sometimes, I'm used it before to clone packages from GitHub on IPv6-only servers.
    - Polylactic_acid 7y ago
      How does github not have an AAAA record in 2020?? The faster people move to gitlab the better. ipv6 servers are not at all rare. Useful for individual use since you can save $1/month by dropping a useless for personal use feature.
      - segfaultbuserr 7y ago
        > How does github not have an AAAA record in 2020? Yeah, it's hopeless, and it goes as if [0] they explicitly decided not to support it, incredibly frustrating! > The faster people move to gitlab the better. A lot projects will always host their master repository on GitHub for better or worse... [0] Not meant to be an accusation, I don't have any evidence.
        mmalone 7y ago
        Last time I launched a new website it took less than a month for someone to let us know that we'd forgotten to configure a AAAA and our site was inaccessible for them. And that was at new website traffic volume. So yea, GitHub definitely knows about AAAA records and has intentionally decided not to have one. The question is: why? They must have a reason. Maybe even a good one. I'm curious.
        segfaultbuserr 7y ago
        I do find some rumors on Reddit [0], which says IPv6 is a low priority project that gives its way to other projects. [0] https://old.reddit.com/r/ipv6/comments/ec8i7y/github_still_doesnt_support_ipv6/fbc166r/ https://old.reddit.com/r/ipv6/comments/ec8i7y/github_still_d...
        rob-olmos 7y ago
        Interesting. Any likely reasons why not having IPv6/AAAA make the site inaccessible to them? Sounds like most of the internet would be inaccessible to them as well?
        7y ago
- mmalone 7y ago
  You can also use the `ProxyJump` directive in your `~/.ssh/config`, which is the same as `-J` on the command line. So, for example: Host host_final ProxyJump user1@host1 will do the same thing as `-J user1@host`, but will allow you to just type: ssh user_final@host_final If you're using an older SSH you can do this with a `ProxyCommand` (requires netcat on the jump box, but that's pretty standard): Host host_final ProxyCommand ssh user1@host1 nc %h %p There are a bunch of variations on this technique, but these are the most common configs. Super easy transparent bastioning. You can get really fancy with this stuff, particularly with `ProxyCommand`. We use it to trigger auto-login for our "Single sign-on for SSH" product at smallstep. When you have a `ProxyCommand` configured, instead of opening its own socket, OpenSSH just execs your proxy command and expects stdin & stdout to end up connected to a socket to the remote server. It doesn't care how that happens or what else happens before you get there. So we (ab)use this as a hook to check if you have a valid SSH certificate in your `ssh-agent` and, if you don't, trigger a single sign-on flow. It's nifty. If you've never read the man pages for `ssh_config` and `sshd_config`, I highly recommend it. It's not that long and there's a lot of good stuff in there.
  - ryukafalz 7y ago
    >If you're using an older SSH you can do this with a `ProxyCommand` (requires netcat on the jump box, but that's pretty standard) Even without ProxyJump, you can do something similar with ProxyCommand without netcat: Host final HostName host.example.com ProxyCommand ssh -W %h:%p user@jumpbox.example.com (Granted, this is still more recent, but I think ProxyJump was introduced later than this.)
davb 7y ago
And in your ~/.ssh/config that's the ProxyJump directive. Adding a proper configuration for the bastion/jump host and for the target host means you can just to "ssh target". In my case, I usually do "ssh target -t tmux -2 att" to attach to my tmux session, then when I detach it will close the SSH connection (and all of my tunnels).
- mmalone 7y ago
  That `tmux` bit is clever. Wonder if you could do that in a `ForceCommand` or something like that so you don’t need to type that part either?
  - mod 7y ago
    (Not OP) I personally don't type my advanced commands, I alias them. So I would use "ssh-target" as my alias there.
    - chrisweekly 7y ago
      Same -- and I comment them heavily, so when (like today per this thread) I learn more about said advanced options, I'm "forced" to update my "docs". Always be capturing your knowledge! :)
  - xelxebar 7y ago
    A combination of `RemoteCommand` and `RequestTTY` should do the trick: Host <host> ... RequestTTY yes RemoteCommand tty The problem is that this will mess up your one-off command invocations of ssh, git ssh connections, etc: $ ssh <host> ls Cannot execute command-line and remote command. However, we can use `Match` blocks to get around that! There are a thousand ways to skin this cat, but one way is to use an environment variable, here `t` for "tmux". Put this at the end of your config: Match exec "test ${t:-0} = 1" RequestTTY yes RemoteCommand tmux has-session && tmux attach-session || tmux and the following Just Work, reattaching tmux sessions as necessary: $ t=1 ssh <host> $ ssh <host> ls
lathiat 7y ago
A good trick to combine with this is ControlSockets.. you can have multiple SSH/SCP connections over the same actual SSH connections. And starting a new SSH over the existing connection is much faster than the initial connection particularly if you are on higher latency (e.g. from Australia at 250-400ms) ``` Host xyz HostName 1.2.3.4 ControlMaster auto ControlPath ~/.ssh/sockets/%r@%h-%p ControlPersist 600 ``` Then if you SSH to the same host multiple times it will re-use the connection and it persists for 10 minutes after you disconnect. If you are using a ProxyJump like the above post this can speed up the initial ProxyJump connection.. or you can just use it with normal SSH (which is what I do) and when i want to open multiple tabs to the same machine its significantly faster.
- mmalone 7y ago
  I've always wondered what this is useful for. So it's purely for performance? Why is it faster to establish the connection? Does it re-use the authentication from the existing ControlMaster too, so you skip the handshake? Seems like it could be dangerous if you're not careful. I guess that's why you can configure it to use `ssh-askpass` for conformation (which, btw, is missing on macOS these days). The other neat-looking directive that I've never tried is `ProxyUseFdpass`, which tells OpenSSH to expect a file descriptor back from your `ProxyCommand` instead of using stdin/stdout. I'm not sure why it exists, but it feels like it could be a performance optimization. Particularly for `scp`. But I've never actually run into a performance problem using `ProxyCommand` so shrug?
  - kbenson 7y ago
    I think it's purely for performance, but it does make a big difference. There's a lot of round trips to set up an SSH connection, and depending on the RTT to the server, that can feel like a real delay if there's another host in the connection. If you use a Proxy host for jumping, and it's got a better RTT to most servers you connect to, reusing the existing connection to it might actually make it feel like it connects faster than a direct connection from your workstation would. e.g. $ ping -c 2 foo PING foo (a.b.c.d) 56(84) bytes of data. 64 bytes from foo (a.b.c.d): icmp_seq=1 ttl=55 time=3.56 ms 64 bytes from foo (a.b.c.d): icmp_seq=2 ttl=55 time=3.62 ms --- foo ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 3ms rtt min/avg/max/mdev = 3.564/3.594/3.624/0.030 ms $ time ssh root@foo "echo foo" foo real 0m0.650s user 0m0.062s sys 0m0.006s Now if I add the following to .ssh/config: Host foo controlmaster auto controlpath ~/.ssh/ssh-%r@%h:%p You can see how much faster it gets if I already have a connection to that server open: $ time ssh root@foo "echo foo" foo real 0m0.032s user 0m0.003s sys 0m0.004s
InvaderFizz 7y ago
Another lesser known tunneling trick is that SSH will happily act as a SOCKS5 Proxy. I've been using this trick for going on 20 years now. Just do: ssh -D9090 user@remote Then, in Firefox, set it to use a SOCK5 proxy of localhost:9090 and "Proxy DNS when using SOCKS v5". Now, when you use Firefox it is as if you are using Firefox on the machine you are SSH'd into(including DNS resolution!). This is really handy for things like accessing otherwise unreachable resources or other internal resources externally. I use it for accessing iDRAC/IPMI/ESXi (you can also tell Java to use the proxy so VMRC works as well). It is also handy to be able to put all your web traffic as originating from a remote VPS with no advanced setup required.
- segfaultbuserr 7y ago
  > Now, when you use Firefox it is as if you are using Firefox on the machine you are SSH'd into(including DNS resolution!). Don't forget to check the "Proxy DNS when using SOCKS v5" option in Firefox.
- orbitingpluto 7y ago
  I used to do this all the time over port 53. My closest coffee shop would allow people to access Wi-Fi only if you gave them full access to your Facebook account. DNS was the only port open to the outside world.
  - s_gourichon 7y ago
    > My closest coffee shop would allow people to access Wi-Fi only if you gave them full access to your Facebook account. What the???
    - mercora 7y ago
      I've seen this on Ubiquity hardware as an option too. Apparently it requires you to "check-in" via facebook to use it, whatever that means exactly. There is also an option to login via facebook without this though.
itsjloh 7y ago
Jump hosts are fantastic! With a little bit of sed magic you can also setup wildcard jump hosts, eg: Host *-via-dc1 ProxyCommand ssh you@jump-01.example.com nc $(echo %h | sed 's/-via-dc1$//') %p I wrote about it ages ago here https://jloh.co/posts/dynamic-ssh-jump-hosts/ https://jloh.co/posts/dynamic-ssh-jump-hosts/
- Piskvorrr 7y ago
  Whoa! Now that's mighty useful, thanks! (Note that this works even without the ProxyJump option, but requires a sane netcat on the jumphost; older busyboxes only had a netcat)
- davb 7y ago
  That's really smart! I'd maybe even reverse it, having: Host my-jump-host Hostname my-jump-host.mydomain.example User myuser Host my-jump-host-* User myuser ProxyCommand ssh my-jump-host nc $(echo %h | sed 's/^my-jump-host-//') %p This way, autocomplete in bash would work. Typing "ssh m<tab>", assuming m* doesn't match anything else, would suggest "ssh my-jump-host" (which I often ssh directly to anyway). Then just add "-myotherhost" so the whole sequence becomes "ssh m<tab><bs>-myotherhost". Not a huge saving but I'm lazy and like autocomplete in the shell.
- davidcollantes 7y ago
  Let's say all machines are named in the form "vm12345," but the jump server itself is also named in the same form. How would you go to include all vm in a wildcard, but the jump server itself?
siver_john 7y ago
Thank you for being worth the whole reason I clicked on this link. I could have used this forever ago when I was trying to figure out how to do this to bypass a firewall (now I don't need to but this will be useful in the future).

nprescott 7y ago

This is a good one. I've used this in the past in order to get onto IPv6-only networks as well. In my case I don't have IPv6 enabled on my home internet (thanks Verizon!) and I had a tiny virtual machine with Vultr, which at their lowest price point aren't offering IPv4 address space any more. Using a jump through another machine with both 4 and 6 address space saved me from having to cough up more money solely for a IPv4 address.

_wldu 7y ago
Use teredo at home. apt install miredo (on most debian based systems) https://en.wikipedia.org/wiki/Teredo_tunneling https://en.wikipedia.org/wiki/Teredo_tunneling
- segfaultbuserr 7y ago
  You can also use Tor as IPv6 proxy in a pure IPv4 network (or as IPv4 proxy in a pure IPv6 network), recent versions of Tor can work under pure IPv6, gaining privacy and connectivity simultaneously. The speed is not actually too bad for web browsing, although not ideal for SSH. But still comes handy sometimes, I'm used it before to clone packages from GitHub on IPv6-only servers.
  - Polylactic_acid 7y ago
    How does github not have an AAAA record in 2020?? The faster people move to gitlab the better. ipv6 servers are not at all rare. Useful for individual use since you can save $1/month by dropping a useless for personal use feature.
    - segfaultbuserr 7y ago
      > How does github not have an AAAA record in 2020? Yeah, it's hopeless, and it goes as if [0] they explicitly decided not to support it, incredibly frustrating! > The faster people move to gitlab the better. A lot projects will always host their master repository on GitHub for better or worse... [0] Not meant to be an accusation, I don't have any evidence.
      - mmalone 7y ago
        Last time I launched a new website it took less than a month for someone to let us know that we'd forgotten to configure a AAAA and our site was inaccessible for them. And that was at new website traffic volume. So yea, GitHub definitely knows about AAAA records and has intentionally decided not to have one. The question is: why? They must have a reason. Maybe even a good one. I'm curious.
        segfaultbuserr 7y ago
        I do find some rumors on Reddit [0], which says IPv6 is a low priority project that gives its way to other projects. [0] https://old.reddit.com/r/ipv6/comments/ec8i7y/github_still_doesnt_support_ipv6/fbc166r/ https://old.reddit.com/r/ipv6/comments/ec8i7y/github_still_d...
        rob-olmos 7y ago
        Interesting. Any likely reasons why not having IPv6/AAAA make the site inaccessible to them? Sounds like most of the internet would be inaccessible to them as well?
        7y ago
mmalone 7y ago
You can also use the `ProxyJump` directive in your `~/.ssh/config`, which is the same as `-J` on the command line. So, for example: Host host_final ProxyJump user1@host1 will do the same thing as `-J user1@host`, but will allow you to just type: ssh user_final@host_final If you're using an older SSH you can do this with a `ProxyCommand` (requires netcat on the jump box, but that's pretty standard): Host host_final ProxyCommand ssh user1@host1 nc %h %p There are a bunch of variations on this technique, but these are the most common configs. Super easy transparent bastioning. You can get really fancy with this stuff, particularly with `ProxyCommand`. We use it to trigger auto-login for our "Single sign-on for SSH" product at smallstep. When you have a `ProxyCommand` configured, instead of opening its own socket, OpenSSH just execs your proxy command and expects stdin & stdout to end up connected to a socket to the remote server. It doesn't care how that happens or what else happens before you get there. So we (ab)use this as a hook to check if you have a valid SSH certificate in your `ssh-agent` and, if you don't, trigger a single sign-on flow. It's nifty. If you've never read the man pages for `ssh_config` and `sshd_config`, I highly recommend it. It's not that long and there's a lot of good stuff in there.
- ryukafalz 7y ago
  >If you're using an older SSH you can do this with a `ProxyCommand` (requires netcat on the jump box, but that's pretty standard) Even without ProxyJump, you can do something similar with ProxyCommand without netcat: Host final HostName host.example.com ProxyCommand ssh -W %h:%p user@jumpbox.example.com (Granted, this is still more recent, but I think ProxyJump was introduced later than this.)

davb 7y ago

And in your ~/.ssh/config that's the ProxyJump directive. Adding a proper configuration for the bastion/jump host and for the target host means you can just to "ssh target". In my case, I usually do "ssh target -t tmux -2 att" to attach to my tmux session, then when I detach it will close the SSH connection (and all of my tunnels).

mmalone 7y ago
That `tmux` bit is clever. Wonder if you could do that in a `ForceCommand` or something like that so you don’t need to type that part either?
- mod 7y ago
  (Not OP) I personally don't type my advanced commands, I alias them. So I would use "ssh-target" as my alias there.
  - chrisweekly 7y ago
    Same -- and I comment them heavily, so when (like today per this thread) I learn more about said advanced options, I'm "forced" to update my "docs". Always be capturing your knowledge! :)
- xelxebar 7y ago
  A combination of `RemoteCommand` and `RequestTTY` should do the trick: Host <host> ... RequestTTY yes RemoteCommand tty The problem is that this will mess up your one-off command invocations of ssh, git ssh connections, etc: $ ssh <host> ls Cannot execute command-line and remote command. However, we can use `Match` blocks to get around that! There are a thousand ways to skin this cat, but one way is to use an environment variable, here `t` for "tmux". Put this at the end of your config: Match exec "test ${t:-0} = 1" RequestTTY yes RemoteCommand tmux has-session && tmux attach-session || tmux and the following Just Work, reattaching tmux sessions as necessary: $ t=1 ssh <host> $ ssh <host> ls

lathiat 7y ago

A good trick to combine with this is ControlSockets.. you can have multiple SSH/SCP connections over the same actual SSH connections. And starting a new SSH over the existing connection is much faster than the initial connection particularly if you are on higher latency (e.g. from Australia at 250-400ms) ``` Host xyz HostName 1.2.3.4 ControlMaster auto ControlPath ~/.ssh/sockets/%r@%h-%p ControlPersist 600 ``` Then if you SSH to the same host multiple times it will re-use the connection and it persists for 10 minutes after you disconnect. If you are using a ProxyJump like the above post this can speed up the initial ProxyJump connection.. or you can just use it with normal SSH (which is what I do) and when i want to open multiple tabs to the same machine its significantly faster.

mmalone 7y ago
I've always wondered what this is useful for. So it's purely for performance? Why is it faster to establish the connection? Does it re-use the authentication from the existing ControlMaster too, so you skip the handshake? Seems like it could be dangerous if you're not careful. I guess that's why you can configure it to use `ssh-askpass` for conformation (which, btw, is missing on macOS these days). The other neat-looking directive that I've never tried is `ProxyUseFdpass`, which tells OpenSSH to expect a file descriptor back from your `ProxyCommand` instead of using stdin/stdout. I'm not sure why it exists, but it feels like it could be a performance optimization. Particularly for `scp`. But I've never actually run into a performance problem using `ProxyCommand` so shrug?
- kbenson 7y ago
  I think it's purely for performance, but it does make a big difference. There's a lot of round trips to set up an SSH connection, and depending on the RTT to the server, that can feel like a real delay if there's another host in the connection. If you use a Proxy host for jumping, and it's got a better RTT to most servers you connect to, reusing the existing connection to it might actually make it feel like it connects faster than a direct connection from your workstation would. e.g. $ ping -c 2 foo PING foo (a.b.c.d) 56(84) bytes of data. 64 bytes from foo (a.b.c.d): icmp_seq=1 ttl=55 time=3.56 ms 64 bytes from foo (a.b.c.d): icmp_seq=2 ttl=55 time=3.62 ms --- foo ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 3ms rtt min/avg/max/mdev = 3.564/3.594/3.624/0.030 ms $ time ssh root@foo "echo foo" foo real 0m0.650s user 0m0.062s sys 0m0.006s Now if I add the following to .ssh/config: Host foo controlmaster auto controlpath ~/.ssh/ssh-%r@%h:%p You can see how much faster it gets if I already have a connection to that server open: $ time ssh root@foo "echo foo" foo real 0m0.032s user 0m0.003s sys 0m0.004s

InvaderFizz 7y ago

Another lesser known tunneling trick is that SSH will happily act as a SOCKS5 Proxy. I've been using this trick for going on 20 years now. Just do: ssh -D9090 user@remote Then, in Firefox, set it to use a SOCK5 proxy of localhost:9090 and "Proxy DNS when using SOCKS v5". Now, when you use Firefox it is as if you are using Firefox on the machine you are SSH'd into(including DNS resolution!). This is really handy for things like accessing otherwise unreachable resources or other internal resources externally. I use it for accessing iDRAC/IPMI/ESXi (you can also tell Java to use the proxy so VMRC works as well). It is also handy to be able to put all your web traffic as originating from a remote VPS with no advanced setup required.

segfaultbuserr 7y ago
> Now, when you use Firefox it is as if you are using Firefox on the machine you are SSH'd into(including DNS resolution!). Don't forget to check the "Proxy DNS when using SOCKS v5" option in Firefox.
orbitingpluto 7y ago
I used to do this all the time over port 53. My closest coffee shop would allow people to access Wi-Fi only if you gave them full access to your Facebook account. DNS was the only port open to the outside world.
- s_gourichon 7y ago
  > My closest coffee shop would allow people to access Wi-Fi only if you gave them full access to your Facebook account. What the???
  - mercora 7y ago
    I've seen this on Ubiquity hardware as an option too. Apparently it requires you to "check-in" via facebook to use it, whatever that means exactly. There is also an option to login via facebook without this though.

itsjloh 7y ago

Jump hosts are fantastic! With a little bit of sed magic you can also setup wildcard jump hosts, eg: Host *-via-dc1 ProxyCommand ssh you@jump-01.example.com nc $(echo %h | sed 's/-via-dc1$//') %p I wrote about it ages ago here https://jloh.co/posts/dynamic-ssh-jump-hosts/ https://jloh.co/posts/dynamic-ssh-jump-hosts/

Piskvorrr 7y ago
Whoa! Now that's mighty useful, thanks! (Note that this works even without the ProxyJump option, but requires a sane netcat on the jumphost; older busyboxes only had a netcat)
davb 7y ago
That's really smart! I'd maybe even reverse it, having: Host my-jump-host Hostname my-jump-host.mydomain.example User myuser Host my-jump-host-* User myuser ProxyCommand ssh my-jump-host nc $(echo %h | sed 's/^my-jump-host-//') %p This way, autocomplete in bash would work. Typing "ssh m<tab>", assuming m* doesn't match anything else, would suggest "ssh my-jump-host" (which I often ssh directly to anyway). Then just add "-myotherhost" so the whole sequence becomes "ssh m<tab><bs>-myotherhost". Not a huge saving but I'm lazy and like autocomplete in the shell.
davidcollantes 7y ago
Let's say all machines are named in the form "vm12345," but the jump server itself is also named in the same form. How would you go to include all vm in a wildcard, but the jump server itself?

siver_john 7y ago

Thank you for being worth the whole reason I clicked on this link. I could have used this forever ago when I was trying to figure out how to do this to bypass a firewall (now I don't need to but this will be useful in the future).