5 ms·

The pitfalls of using ssh-agent, or how to use an agent safely

87 points by lelf 12y ago

nhaehnle 12y ago
I would replace recommendation #5 with: use a different key on each physical machine. My home desktop has a different private key than my private laptop, and at work I have yet another SSH private key. This makes more sense to me than separating keys by "purpose", since it aligns better with the threat model (machine gets compromised).
- bdarnell 12y ago
  You can do both: use a different key for each purpose on each physical machine. This addresses two different threat models: each machine has a different set of keys to address compromise of that machine, and you use a different key for each purpose so that forwarding your agent to a compromised machine doesn't compromise your other keys.
  - akerl_ 12y ago
    Partial compromise of a machine's private keys is not a realistic threat profile for the vast majority of users. Unless you're using a different agent per key, which very few people do, your same agent will give out any of your keys that it has stored, regardless of which key you actually used to hit the remote machine. If somebody is in a position to get some of your keys off a machine, they're almost certainly in a position to get all of them. As such, having multiple private keys per system doesn't provide much additional security, it only increases confusion. One of the only threats that separate keys does protect against is if you suspect multiple remotes would be compromised and being able to link your public key to both would be dangerous. But in that case, you also need to be doing so many other things to ensure your single source machine cannot be linked to both remote systems.
    - bdarnell 12y ago
      Agreed; it doesn't really make sense to use multiple keys if you're going to put them all in one agent. That's why the original post recommends using multiple agents at the same time it recommends multiple keys (and the author provides a script to make it easier to manage multiple agents)
Florin_Andrei 12y ago
I'm deploying YubiKey NEO tokens for employees here. It has the form factor of a very small, flattened-out USB stick (almost like a cardboard contour), but it's a smartcard and an OTP generator. The smartcard can generate a pair of public/private SSH keys, and you could then just push the public key wherever. The private key remains on the token, and never leaves it no matter what. You need gpg-agent running in ssh-agent emulation to have ssh use the private key on the token. It requires you to enter a user PIN before it authenticates connections, but the PIN can be cached by gpg-agent for a configurable duration. Enter the wrong user PIN three times in a row and it locks up (can be unlocked via the admin PIN). Basically, everything you know about smartcards applies here - except this is very small and portable. You can't reveal the private SSH key even if you're very sloppy. The OTP can be setup with things such as VPN or other places where you need to authenticate with user/pass. There's an open source authentication server that you can deploy wherever, or you could use the manufacturer's cloud auth. It can also do NFC and FIDO U2F (it's a superset of Google's cheap FIDO U2F Security Key). In a word - awesome.
- gcb0 12y ago
  what about other users with root access where you are running *-agent?
  - Florin_Andrei 12y ago
    One thing is clear: this is a smartcard, the private key never leaves the token, period. gpg-agent acts only as a middleman between ssh and the token, that's all. It's impossible to capture or extract the private key (unless you're the NSA and you have physical possession of the token, I guess). Now, are there ways to attack the authentication via tapping as root into the gpg-agent process? I don't think so, but frankly I'm not that familiar with the sausage-making parts of it. I'd like to learn more about this topic.
    - MichaelGG 12y ago
      > impossible to capture or extract the private key They should get FIPS 140-2 Level 4 certification, then. Other devices that claimed to be so resistant haven't really fared that well when someone actually bothered to look. It might be out if there range of average software hackers, but perhaps easily handled by someone with hardware capabilities. Even the Chrysalis Luna CA3 (I think that model) was breached by a university team. And a sole researcher breached a popular TPM, finding no special protection. So I'm not sure it's justified to say it requires an NSA level adversary. The bar is probably much lower. On personal stuff, I use drive level encryption, then Bitlocker with TPM, then Athena SCS smartcard for logon+EFS, and Bitlocker on a vhd for the VM actually running stuff. Only the last layer, the passphrase on the vhd, is something I count on. The rest is just to make it easy to brick the device and prevent a quick handed adversary from quickly installing a rootkit. In particular, I have very little confidence in Crucial's SSD's drive encryption. For all I know, the key is stored unencrypted on the first sector. But it's totally free, as the device runs AES to balance the 1s and 0s anyways.
IgorPartola 12y ago
My big worry with ssh-agent forwarding is that someone may get root on a random server I connect to and then be able to use my forwarded connection to access another machine. To avoid this I created https://github.com/ipartola/mac-ssh-confirm https://github.com/ipartola/mac-ssh-confirm. Basically, every time you go to use your ssh-agent you get a GUI confirmation dialog with OK/Cancel. That way you can prevent unauthorized access.
- keeperofdakeys 12y ago
  ssh-add (at least on linux) has the -c option, which tells ssh-agent to do exactly this.
  - IgorPartola 12y ago
    That's what I use. ssh-add properly does not actually include a UI of any kind. It just calls $SSH_ASKPASS. The linked code essentially provides two wrappers around all this: one to add keys and set up SSH_ASKPASS, and the other to be the script that is $SSH_ASKPASS that uses the Mac OS X GUI to show you the confirmation dialog.
deleted 12y ago
[deleted]
ansible 12y ago
This is my version, which deals with multiple machines using my same remote-mounted home directory: https://gist.github.com/jamesgraves/9af46a544b6dc7da9097 https://gist.github.com/jamesgraves/9af46a544b6dc7da9097 However, it tries to keep one agent running forever, which the OP doesn't like. I am frequently in situations, like using cygwin, where I don't have a nice graphical shell to kill the agent / keyring when I log out. Though I don't often log out either, unless I need to reboot for an OS upgrade. I don't know, either you're going to rely on the screen lock for security, or you aren't. As for mixing work with personal stuff, I've created multiple accounts on my laptop for each to more easily keep them separate.
JoshTriplett 12y ago
If you forward an agent, that doesn't actually provide access to the private key. It provides access to the public key, as well as access to key operations that would allow authenticating to a server using the private key (such as SSH2_AGENTC_SIGN_REQUEST), but it doesn't actually expose the private key for future operations after you stop forwarding the agent. The interaction shown in the "Having fun with an agent" section of the article shows the agent providing the public key and name, not the private key. See the ssh-agent protocol for details: http://api.libssh.org/rfc/PROTOCOL.agent http://api.libssh.org/rfc/PROTOCOL.agent You still should never forward your SSH agent to a system you don't trust. However, if someone compromises such a system, they have not necessarily compromised your SSH private key, though they may have used your agent to access systems you have access to while you were forwarding the agent.
- caf 12y ago
  As the article points out though, they may have used that temporary access to your systems to leverage more permanent access. (And of course if the machine you're connecting from has an sshd listening then there's a good chance they could successfully connect back to it using your agent forwarding, then subvert the agent itself)
- jamiesonbecker 12y ago
  Exactly. Public keys are by definition public so they can be spread freely all over. Forwarding will never compromise your private key. Of course, as you point out, if you forward THROUGH a compromised system, someone could hop on the key forwarding and come along for the ride. And, as another commenter pointed out, they could always hijack your access on the final system to create their own user accounts. And also why a good SSH key manager like https://Userify.com https://Userify.com to distribute your public key can make forwarding easier.
deleted 12y ago
[deleted]
khaki54 12y ago
Also use "hashknownhosts yes" in your user ssh config file. Makes it another level harder to see where you went, though a compromised root would see all of the logs.
- scintill76 12y ago
  I've posted (rambled?) about why I think HashKnownHosts is an overly cumbersome idea, and even a suboptimal security practice: http://blog.joeyhewitt.com/2013/12/openssh-hashknownhosts-a-bad-idea/ http://blog.joeyhewitt.com/2013/12/openssh-hashknownhosts-a-... . Summary: "An opaque, difficult-to-audit trusted key database could be a security liability of its own. This file isn't some MRU auto-completion nicety, it's a security mechanism to prevent man-in-the-middle attacks." If you type in all your connection details by hand every time you ssh (can't store them anywhere, or the bad guys will see the servers you use), your shell doesn't log your ssh command lines in history, you don't mind having no idea what's actually in your known_hosts file, and you think attackers who care will be unable to dictionary-crack your hashes, then HashKnownHosts will accomplish something for you. Otherwise it's dumb, IMO. Open to hearing about something I've missed, though, or just people whose usecase differs enough that my points don't apply to them.

87 points by lelf 12y ago

I would replace recommendation #5 with: use a different key on each physical machine. My home desktop has a different private key than my private laptop, and at work I have yet another SSH private key. This makes more sense to me than separating keys by "purpose", since it aligns better with the threat model (machine gets compromised).

bdarnell 12y ago
You can do both: use a different key for each purpose on each physical machine. This addresses two different threat models: each machine has a different set of keys to address compromise of that machine, and you use a different key for each purpose so that forwarding your agent to a compromised machine doesn't compromise your other keys.
- akerl_ 12y ago
  Partial compromise of a machine's private keys is not a realistic threat profile for the vast majority of users. Unless you're using a different agent per key, which very few people do, your same agent will give out any of your keys that it has stored, regardless of which key you actually used to hit the remote machine. If somebody is in a position to get some of your keys off a machine, they're almost certainly in a position to get all of them. As such, having multiple private keys per system doesn't provide much additional security, it only increases confusion. One of the only threats that separate keys does protect against is if you suspect multiple remotes would be compromised and being able to link your public key to both would be dangerous. But in that case, you also need to be doing so many other things to ensure your single source machine cannot be linked to both remote systems.
  - bdarnell 12y ago
    Agreed; it doesn't really make sense to use multiple keys if you're going to put them all in one agent. That's why the original post recommends using multiple agents at the same time it recommends multiple keys (and the author provides a script to make it easier to manage multiple agents)

Florin_Andrei 12y ago

I'm deploying YubiKey NEO tokens for employees here. It has the form factor of a very small, flattened-out USB stick (almost like a cardboard contour), but it's a smartcard and an OTP generator. The smartcard can generate a pair of public/private SSH keys, and you could then just push the public key wherever. The private key remains on the token, and never leaves it no matter what. You need gpg-agent running in ssh-agent emulation to have ssh use the private key on the token. It requires you to enter a user PIN before it authenticates connections, but the PIN can be cached by gpg-agent for a configurable duration. Enter the wrong user PIN three times in a row and it locks up (can be unlocked via the admin PIN). Basically, everything you know about smartcards applies here - except this is very small and portable. You can't reveal the private SSH key even if you're very sloppy. The OTP can be setup with things such as VPN or other places where you need to authenticate with user/pass. There's an open source authentication server that you can deploy wherever, or you could use the manufacturer's cloud auth. It can also do NFC and FIDO U2F (it's a superset of Google's cheap FIDO U2F Security Key). In a word - awesome.

gcb0 12y ago
what about other users with root access where you are running *-agent?
- Florin_Andrei 12y ago
  One thing is clear: this is a smartcard, the private key never leaves the token, period. gpg-agent acts only as a middleman between ssh and the token, that's all. It's impossible to capture or extract the private key (unless you're the NSA and you have physical possession of the token, I guess). Now, are there ways to attack the authentication via tapping as root into the gpg-agent process? I don't think so, but frankly I'm not that familiar with the sausage-making parts of it. I'd like to learn more about this topic.
  - MichaelGG 12y ago
    > impossible to capture or extract the private key They should get FIPS 140-2 Level 4 certification, then. Other devices that claimed to be so resistant haven't really fared that well when someone actually bothered to look. It might be out if there range of average software hackers, but perhaps easily handled by someone with hardware capabilities. Even the Chrysalis Luna CA3 (I think that model) was breached by a university team. And a sole researcher breached a popular TPM, finding no special protection. So I'm not sure it's justified to say it requires an NSA level adversary. The bar is probably much lower. On personal stuff, I use drive level encryption, then Bitlocker with TPM, then Athena SCS smartcard for logon+EFS, and Bitlocker on a vhd for the VM actually running stuff. Only the last layer, the passphrase on the vhd, is something I count on. The rest is just to make it easy to brick the device and prevent a quick handed adversary from quickly installing a rootkit. In particular, I have very little confidence in Crucial's SSD's drive encryption. For all I know, the key is stored unencrypted on the first sector. But it's totally free, as the device runs AES to balance the 1s and 0s anyways.

IgorPartola 12y ago

My big worry with ssh-agent forwarding is that someone may get root on a random server I connect to and then be able to use my forwarded connection to access another machine. To avoid this I created https://github.com/ipartola/mac-ssh-confirm https://github.com/ipartola/mac-ssh-confirm. Basically, every time you go to use your ssh-agent you get a GUI confirmation dialog with OK/Cancel. That way you can prevent unauthorized access.

keeperofdakeys 12y ago
ssh-add (at least on linux) has the -c option, which tells ssh-agent to do exactly this.
- IgorPartola 12y ago
  That's what I use. ssh-add properly does not actually include a UI of any kind. It just calls $SSH_ASKPASS. The linked code essentially provides two wrappers around all this: one to add keys and set up SSH_ASKPASS, and the other to be the script that is $SSH_ASKPASS that uses the Mac OS X GUI to show you the confirmation dialog.

deleted 12y ago

[deleted]

ansible 12y ago

This is my version, which deals with multiple machines using my same remote-mounted home directory: https://gist.github.com/jamesgraves/9af46a544b6dc7da9097 https://gist.github.com/jamesgraves/9af46a544b6dc7da9097 However, it tries to keep one agent running forever, which the OP doesn't like. I am frequently in situations, like using cygwin, where I don't have a nice graphical shell to kill the agent / keyring when I log out. Though I don't often log out either, unless I need to reboot for an OS upgrade. I don't know, either you're going to rely on the screen lock for security, or you aren't. As for mixing work with personal stuff, I've created multiple accounts on my laptop for each to more easily keep them separate.

JoshTriplett 12y ago

If you forward an agent, that doesn't actually provide access to the private key. It provides access to the public key, as well as access to key operations that would allow authenticating to a server using the private key (such as SSH2_AGENTC_SIGN_REQUEST), but it doesn't actually expose the private key for future operations after you stop forwarding the agent. The interaction shown in the "Having fun with an agent" section of the article shows the agent providing the public key and name, not the private key. See the ssh-agent protocol for details: http://api.libssh.org/rfc/PROTOCOL.agent http://api.libssh.org/rfc/PROTOCOL.agent You still should never forward your SSH agent to a system you don't trust. However, if someone compromises such a system, they have not necessarily compromised your SSH private key, though they may have used your agent to access systems you have access to while you were forwarding the agent.

caf 12y ago
As the article points out though, they may have used that temporary access to your systems to leverage more permanent access. (And of course if the machine you're connecting from has an sshd listening then there's a good chance they could successfully connect back to it using your agent forwarding, then subvert the agent itself)
jamiesonbecker 12y ago
Exactly. Public keys are by definition public so they can be spread freely all over. Forwarding will never compromise your private key. Of course, as you point out, if you forward THROUGH a compromised system, someone could hop on the key forwarding and come along for the ride. And, as another commenter pointed out, they could always hijack your access on the final system to create their own user accounts. And also why a good SSH key manager like https://Userify.com https://Userify.com to distribute your public key can make forwarding easier.

deleted 12y ago

[deleted]

khaki54 12y ago

Also use "hashknownhosts yes" in your user ssh config file. Makes it another level harder to see where you went, though a compromised root would see all of the logs.

scintill76 12y ago
I've posted (rambled?) about why I think HashKnownHosts is an overly cumbersome idea, and even a suboptimal security practice: http://blog.joeyhewitt.com/2013/12/openssh-hashknownhosts-a-bad-idea/ http://blog.joeyhewitt.com/2013/12/openssh-hashknownhosts-a-... . Summary: "An opaque, difficult-to-audit trusted key database could be a security liability of its own. This file isn't some MRU auto-completion nicety, it's a security mechanism to prevent man-in-the-middle attacks." If you type in all your connection details by hand every time you ssh (can't store them anywhere, or the bad guys will see the servers you use), your shell doesn't log your ssh command lines in history, you don't mind having no idea what's actually in your known_hosts file, and you think attackers who care will be unable to dictionary-crack your hashes, then HashKnownHosts will accomplish something for you. Otherwise it's dumb, IMO. Open to hearing about something I've missed, though, or just people whose usecase differs enough that my points don't apply to them.