5 ms·

On the first issue opened by the hacker: > Complete compromise could have been avoided if developers were prohibited from using ForwardAgent yes or not using -

On the first issue opened by the hacker: > Complete compromise could have been avoided if developers were prohibited from using ForwardAgent yes or not using -A in their SSH commands. The flaws with agent forwarding are well documented. I use agent forwarding daily and had no idea it contained well known security holes. If that's the case, why is the feature available by default?

XMPPwocky 8y ago
SSH agent forwarding makes your ssh-agent available to (possibly some subset of) hosts you SSH into. This is its purpose. Unfortunately, it also makes your ssh-agent available to (possibly some subset of) hosts you SSH into.
- scanr 8y ago
  Is there a secure alternative that achieves the same outcome?
  - RL_Quine 8y ago
    `ProxyJump`
    - aidos 8y ago
      Hadn’t seem that before. Article here explains is briefly https://www.madboa.com/blog/2017/11/02/ssh-proxyjump/ https://www.madboa.com/blog/2017/11/02/ssh-proxyjump/
      - theamk 8y ago
        This article is .. weird. It mentions SOCKS5, DynamicForwarding and "decent version of nc", while you don't need anything at all for forwarding connection -- SOCKS is not involved in any way, and initial 1995 release of nc would work just fine. Here is a much better explanation (from [0]): > ProxyJump was added in OpenSSH 7.3 but is nothing more than a shorthand for using ProxyCommand, as in: "ProxyCommand ssh proxy-host -W %h:%p" so the same thing that top poster was talking about. [0] https://superuser.com/questions/1253960/replace-proxyjump-in-ssh-config https://superuser.com/questions/1253960/replace-proxyjump-in...
    - metafunctor 8y ago
      Do you keep your keys on the proxy host, then? Otherwise, "ForwardAgent yes" and you're back to the same situation.
      - syntonym 8y ago
        ProxyJump uses the keys from the original host, not the proxy host.
        metafunctor 8y ago
        I know. That's why I asked. Chained agent forwarding will serve your keys just the same, so ProxyJump is not "a secure alternative that achieves the same outcome".
        syntonym 8y ago
        Are you disagreeing with the "secure alternative" or the "same outcome"? I thought the difference between ProxyJump and agent forwarding is the following: Agent forwarding forwards the agent socket to the proxy server. Thus any ssh connection originating from the proxy server can reuse the agent, and with that has the same access to the agent as the originating host. ProxyJump routes the ssh connection through the proxy host. The crypto takes place between originating host and target host, not between proxy host and target host. ssh connections originating from the proxy host can not access keys from the originating host. But maybe my understanding of ProxyJump is incorrect?
        metafunctor 8y ago
        I know exactly how agent forwarding and ProxyJump work, but I'm having a hard time understanding what you mean. ProxyJump proxies your ssh connection, so connecting from A to B via proxy X the connections go A->X and X->B. You can use AgentForwarding with ProxyJump, in which case agent connections go B->X->A. I cannot see how ProxyJump would somehow be an alternative to AgentForwarding. You can use both independently.
  - XMPPwocky 8y ago
    I'm not sure. A secure, backwards-compatible (with older servers) alternative, which only exposes keys you explicitly choose to expose, should be doable and might help. another option would be for a SSH client to present a full-screen "$HOST is trying to use your your SSH PRIVATE keys. Press enter, then type "~A" to allow." prompt.
  - metafunctor 8y ago
    Here are a few ideas that might help. Use separate keyboard-interactive 2FA (I recommend google-authenticator) for production ssh access. Use a key system which requires confirmation or a PIN to authenticate (such as a Yubikey). Use a persisting ssh connection with Ansible (ControlPersist) to avoid unnecessary multiple authentications. Allow connections only from whitelisted IPs, or Uuse port knocking to open temporary holes in your firewall, or require connections to production infrastructure to go through a VPN. Access production infrastructure from hardware dedicated for that purpose, never do anything else on it. I wish there was a way in ssh to tag connections and only allow agent forwarding to keys with the same tag. That would prevent agent forwarding production keys from a dev host.
    - deleted 8y ago
      [deleted]
    - pas 8y ago
      https://github.com/StanfordSNR/guardian-agent https://github.com/StanfordSNR/guardian-agent
  - pas 8y ago
    https://github.com/StanfordSNR/guardian-agent https://github.com/StanfordSNR/guardian-agent
    - brunoqc 8y ago
      That project looks dead.
- aidos 8y ago
  I never quite understand why there’s not a confirm version. ForwardWithConfirmation or something. I’m active when I need forwarding - would be happy to simply be prompted before it’s allowed.
  - Leace 8y ago
    If you use Yubikey with touch-to-use enabled that'll be basically what you're asking - each authentication will require touching the token.
    - haasted 8y ago
      I just enabled that after seeing this incident. For people in the same boat, it can be done trivially using the YubiKey Manager CLI: https://developers.yubico.com/yubikey-manager/ https://developers.yubico.com/yubikey-manager/
  - rwmj 8y ago
    Some ssh agent implementations do this, notably the one built into Android ConnectBot can be configured to request confirmation each time it is asked to authenticate. Unfortunately ssh-agent (from OpenSSH) does not as far as I know. It's happy to authenticate as many times as requested without any notification.
    - DBFFD990 8y ago
      It can, and it's determined per key when added to the agent. Look for -c here: https://man.openbsd.org/ssh-add https://man.openbsd.org/ssh-add
      - rwmj 8y ago
        Indeed it is - I even checked the man page before posting the comment and completely missed that option.
  - DblPlusUngood 8y ago
    OpenSSH does have confirmation: use the '-c' switch to ssh-add. https://man.openbsd.org/ssh-add https://man.openbsd.org/ssh-add
nightfly 8y ago
> I use agent forwarding daily That's why. It's useful, but you have to be mindful of the security risks involved in using it.
technion 8y ago
I think an issue here is we've been told for a long time "always restrict access to the environment through a bastion host" without much implementation detail discussed after that. Agent forwarding tends to show up as the most efficient way to implement this.
crazydoggers 8y ago
Basically you need to make sure the host you are SSH'ing into with an agent is secure. Otherwise the root user on that host can access your agent socket and connect to any other machines your agent can. So if you SSH -A to a compromised Jenkins server, and you've got all your production keys loaded in your agent, the hacker can now authenticate to all those production machines as well. So don't ever SSH -A into a machine unless you KNOW its secure. The way I think about it is unless I trust the machine enough to leave my private keys on that machine, then I'm not going to SSH -A into it.

XMPPwocky 8y ago

SSH agent forwarding makes your ssh-agent available to (possibly some subset of) hosts you SSH into. This is its purpose. Unfortunately, it also makes your ssh-agent available to (possibly some subset of) hosts you SSH into.

scanr 8y ago
Is there a secure alternative that achieves the same outcome?
- RL_Quine 8y ago
  `ProxyJump`
  - aidos 8y ago
    Hadn’t seem that before. Article here explains is briefly https://www.madboa.com/blog/2017/11/02/ssh-proxyjump/ https://www.madboa.com/blog/2017/11/02/ssh-proxyjump/
    - theamk 8y ago
      This article is .. weird. It mentions SOCKS5, DynamicForwarding and "decent version of nc", while you don't need anything at all for forwarding connection -- SOCKS is not involved in any way, and initial 1995 release of nc would work just fine. Here is a much better explanation (from [0]): > ProxyJump was added in OpenSSH 7.3 but is nothing more than a shorthand for using ProxyCommand, as in: "ProxyCommand ssh proxy-host -W %h:%p" so the same thing that top poster was talking about. [0] https://superuser.com/questions/1253960/replace-proxyjump-in-ssh-config https://superuser.com/questions/1253960/replace-proxyjump-in...
  - metafunctor 8y ago
    Do you keep your keys on the proxy host, then? Otherwise, "ForwardAgent yes" and you're back to the same situation.
    - syntonym 8y ago
      ProxyJump uses the keys from the original host, not the proxy host.
      - metafunctor 8y ago
        I know. That's why I asked. Chained agent forwarding will serve your keys just the same, so ProxyJump is not "a secure alternative that achieves the same outcome".
        syntonym 8y ago
        Are you disagreeing with the "secure alternative" or the "same outcome"? I thought the difference between ProxyJump and agent forwarding is the following: Agent forwarding forwards the agent socket to the proxy server. Thus any ssh connection originating from the proxy server can reuse the agent, and with that has the same access to the agent as the originating host. ProxyJump routes the ssh connection through the proxy host. The crypto takes place between originating host and target host, not between proxy host and target host. ssh connections originating from the proxy host can not access keys from the originating host. But maybe my understanding of ProxyJump is incorrect?
        metafunctor 8y ago
        I know exactly how agent forwarding and ProxyJump work, but I'm having a hard time understanding what you mean. ProxyJump proxies your ssh connection, so connecting from A to B via proxy X the connections go A->X and X->B. You can use AgentForwarding with ProxyJump, in which case agent connections go B->X->A. I cannot see how ProxyJump would somehow be an alternative to AgentForwarding. You can use both independently.
- XMPPwocky 8y ago
  I'm not sure. A secure, backwards-compatible (with older servers) alternative, which only exposes keys you explicitly choose to expose, should be doable and might help. another option would be for a SSH client to present a full-screen "$HOST is trying to use your your SSH PRIVATE keys. Press enter, then type "~A" to allow." prompt.
- metafunctor 8y ago
  Here are a few ideas that might help. Use separate keyboard-interactive 2FA (I recommend google-authenticator) for production ssh access. Use a key system which requires confirmation or a PIN to authenticate (such as a Yubikey). Use a persisting ssh connection with Ansible (ControlPersist) to avoid unnecessary multiple authentications. Allow connections only from whitelisted IPs, or Uuse port knocking to open temporary holes in your firewall, or require connections to production infrastructure to go through a VPN. Access production infrastructure from hardware dedicated for that purpose, never do anything else on it. I wish there was a way in ssh to tag connections and only allow agent forwarding to keys with the same tag. That would prevent agent forwarding production keys from a dev host.
  - deleted 8y ago
    [deleted]
  - pas 8y ago
    https://github.com/StanfordSNR/guardian-agent https://github.com/StanfordSNR/guardian-agent
- pas 8y ago
  https://github.com/StanfordSNR/guardian-agent https://github.com/StanfordSNR/guardian-agent
  - brunoqc 8y ago
    That project looks dead.
aidos 8y ago
I never quite understand why there’s not a confirm version. ForwardWithConfirmation or something. I’m active when I need forwarding - would be happy to simply be prompted before it’s allowed.
- Leace 8y ago
  If you use Yubikey with touch-to-use enabled that'll be basically what you're asking - each authentication will require touching the token.
  - haasted 8y ago
    I just enabled that after seeing this incident. For people in the same boat, it can be done trivially using the YubiKey Manager CLI: https://developers.yubico.com/yubikey-manager/ https://developers.yubico.com/yubikey-manager/
- rwmj 8y ago
  Some ssh agent implementations do this, notably the one built into Android ConnectBot can be configured to request confirmation each time it is asked to authenticate. Unfortunately ssh-agent (from OpenSSH) does not as far as I know. It's happy to authenticate as many times as requested without any notification.
  - DBFFD990 8y ago
    It can, and it's determined per key when added to the agent. Look for -c here: https://man.openbsd.org/ssh-add https://man.openbsd.org/ssh-add
    - rwmj 8y ago
      Indeed it is - I even checked the man page before posting the comment and completely missed that option.
- DblPlusUngood 8y ago
  OpenSSH does have confirmation: use the '-c' switch to ssh-add. https://man.openbsd.org/ssh-add https://man.openbsd.org/ssh-add

nightfly 8y ago

> I use agent forwarding daily That's why. It's useful, but you have to be mindful of the security risks involved in using it.

technion 8y ago

I think an issue here is we've been told for a long time "always restrict access to the environment through a bastion host" without much implementation detail discussed after that. Agent forwarding tends to show up as the most efficient way to implement this.

crazydoggers 8y ago

Basically you need to make sure the host you are SSH'ing into with an agent is secure. Otherwise the root user on that host can access your agent socket and connect to any other machines your agent can. So if you SSH -A to a compromised Jenkins server, and you've got all your production keys loaded in your agent, the hacker can now authenticate to all those production machines as well. So don't ever SSH -A into a machine unless you KNOW its secure. The way I think about it is unless I trust the machine enough to leave my private keys on that machine, then I'm not going to SSH -A into it.