8 ms·

Using OpenSSH Certificate Authentication

110 points by p4bl0 13y ago

jamiesonbecker 13y ago
Excellent article. I like the idea of running my own CA, but prefer the Userify disconnected model instead.
jnazario 13y ago
about time. this has been needed for a long time. ssh mitm attacks were too easy, and legitimately changing server keys (which happens more often than you would expect) would just confuse the issue. so - about time. tatu's decision to NOT use this feature of public key crypto when he designed ssh keys has always been a point of frustration for me.
eik3_de 13y ago
I stumbled across > Logging in directly as root user is a bad idea — even in examples further down.. > ~/ssh_cert_authority/user_ca $ scp -rp user_ca.pub root@example.host.net:/etc/ssh
- dspillett 13y ago
  Doeth what I say, not'th what I do... I think we are all guilty of this far more often then we like to admit. On this particular one, I use root logins far too often rather than observing best practise.
  - eik3_de 13y ago
    well that's what PermitRootLogin no is for
    - peterwwillis 13y ago
      Ya know, people go on and on about disabling root logins. Have you ever had to write a piece of automation to login to a host as a non-root user, interact with a user interface, drop to a normal user shell, then run a sudo scp command, and then pass back non-interactive raw non-TTY access to an scp control program? It's hard. I know because I wrote a really crappy script that does it. You know what's so much easier and more stable? Running scp as root. Sadly, sometimes the best engineering choice is not a good one.
      - vertex-four 13y ago
        I'll point out that Ansible can do this just fine. Assuming you've got a non-root user set up in your node list file... ansible [server] --sudo -m copy -a "src=filename dest=filename" Other parts of Ansible might not be great (specifically, fully automated datacenter orchestration isn't possible without writing your own tools on top of Ansible; I use Salt for that), but it's really good for ad-hoc management without having to put additional infrastructure or daemons in place.
        jamiesonbecker 13y ago
        cool - thx for this. also can use secure tmp files or rsync: rsync --rsync-cmd='sudo rsync' -rpave ssh \ /some/dir/ \ me@remote:/some/dir/ (or, pulling backups for best security) vault$ rsync --rsync-cmd='sudo rsync' -rpave ssh \ backup_user@dbserver:/var/ /backups/var/
      - skywhopper 13y ago
        Sounds like you have other problems besides just needing root access. It's always possible to come up with an edge case, but that doesn't invalidate the rule. Avoid direct root login at all costs.
        peterwwillis 13y ago
        s/problems/requirements/. And I wasn't trying to invalidate a rule, but merely point out that rules are made to be broken ;)
        dspillett 13y ago
        > It's always possible to come up with an edge case, but that doesn't invalidate the rule That is why we have risk registers in professional contexts. Whenever following policy (or the less specifically defined "accepted best practise") is impractical or inconvenient a note is made in the project's risk register with who made the decision, why, and who signed off on accepting the risk of not following policy in this case. Doing it this way forces people to ask themselves "is it really that impractical/inconvenient?", "have I properly considered the possible implications and calculated the risk (waving a hand and saying it'll all be fine because you've done it before does not constitute calculating the risk!)", and "have we applied as much mitigation to the risk as possible (enforcing secure password policy and so forth)". After considering those things then by all means allow password based remote logins for root and put your name against the choice in the register (or try get someone else to if you do not have sufficient authority).
    - e12e 13y ago
      I generally have both: PermitRootLogin no AllowGroups dial-in #or something similar The one time I need to grant someone login, I'll probably scratch my head why they can't log in, then remember to add them to the right group... [edit: and, of course: PasswordAuthentication no ]
  - taejo 13y ago
    > Doeth what I say, not'th what I do... Just by the way, in the period of English with the -eth ending (Early Modern English), the correct grammar would be (wait for it) "Do what I say, not what I do".
- neoCrimeLabs 13y ago
  It's great to see someone who reads something in depth! Thanks for taking the time and commenting. :-) My point with with the comment "Logging in directly as root user is a bad idea — even in examples" was that you should have enough pre-requsite knowledge to understand why that is bad, and that you should not follow my examples 100%. That and I was poking fun at myself for being sloppy. :-)
  - jamiesonbecker 13y ago
    nice :) rsync --rsync-path='sudo rsync' ... ssh remote sudo passwd -l root (note diffs between rhel and deb/ubu)
- marcosdumay 13y ago
  Is it really so bad? There are always priviledge escalation bugs open on Linux, and the attacker can always just try to guess the password, with several different sessions running su. I don't trust my systems to survive local atacks, or at least I trust them much, much less then surviving external attacks. Personaly, I have my machines on a hierarchy of trust. More trusted machines can ssh as root into less trusted one, but not the other way around. It's much easier to automatize things this way.
  - jamiesonbecker 13y ago
    There is literally never a reason for one machine to log into another as root except for your convenience. Put stuff that needs to run as root into a shell script (or use rsync --rsync-cmd) and then use sudo.
- IgorPartola 13y ago
  You know what? Sometimes, it's fine. You should not share a root account with other users. This advice comes from a time where you'd have the "root password" and log into remote servers that way. Then, when someone leaves your organization, you'd hopefully change the root password to make sure they person that left no longer has access. However, there are some concrete cases where logging in as root is just fine: - Making a database backup remotely. You have to give the backup user enough privileges to initiate a database dump, stop the database process, grab the backup, rotate the backups, start the database process up again. You can go through the giant pain of setting up the permissions just so, or you can put a separate SSH public key into /root/.ssh/authorized_keys and just let root do all of this. - Initial VM setup. If you are automating something like provisioning EC2 instances, sure go ahead and set up individual users, but it's a lot easier to first connect as root using your own keypair. - A remote monitor that logs into a single purpose VM to restart a process. Same as the backups example, but more broad. Once again, you could set it up using specific users, but the system would be more brittle. - Lots of other cases where root is really what you want and doing sudo won't work because the session is not interactive. I figure it this way: if I compromised a local account on a server, I'd just add `alias sudo=~/.bin/my-evil-sudo-that-emails-your-password` to your .bashrc. That way I'd get root without any local exploits in the kernel, etc. Basically, adding extra protection around the root account does nothing if you are a sudoer. Some steps you can take to actually increase your security instead of following the advice that is so misunderstood: - Set the root password to a long random string, then lose the string. `su` should never be used by anyone but root. - Disable password logins. Period. Do this now, and do it everywhere. - Disable password-based sudo. Use pam_ssh_agent_auth [1] to escalate privileges and ssh-agent to forward your key through your ssh session. I am not sure why this isn't the default, but it really ought to be. - Make sure your local key is encrypted with a strong passphrase. Once again, use ssh-agent to not have to enter the password every single time. - Set up a VPN to access your servers and only allow ssh from the VPN. - For the truly paranoid, set up two-factor auth over SSH. This is easy to do with Google Authenticator (it's just a random number with the seed being shared and secret and the number being generated every 30 seconds), or using SMS which is just slightly more complex/costly. [1] http://pamsshagentauth.sourceforge.net/ http://pamsshagentauth.sourceforge.net/
  - 13y ago
zobzu 13y ago
similar, but using gpg, also interesting http://web.monkeysphere.info/getting-started-ssh/ http://web.monkeysphere.info/getting-started-ssh/
- IgorPartola 13y ago
  This works really well with a couple of exceptions: 1. It slows down your initial connection while it does another connection to check the key 2. You must convince everyone to use it, which is hard since gpg is hard. I'd love to see more discussion of Monkeysphere on here. Every time I post something about it it gets no votes.
ehPReth 13y ago
Is there a way to have it use a CRL instead of running around and ensuring a certificate is marked as revoked on every device?
- riskable 13y ago
  Unfortunately, no. It's the key reason why this feature isn't widely adopted I think.
  - ehPReth 13y ago
    Ah, darn. Thanks for the reply
- cpach 13y ago
  If you’re interested in proprietary solutions, I think the ”original” SSH server (from SSH Communications Security Oyj) can do stuff like that.
stormbrew 13y ago
This reminded me I wanted to investigate switching to this for our infrastructure, but uh... it seems that on at least ubuntu's build of openssh if you pass a serial number to ssh-keygen it segfaults. Even if it's the default (0). That's kind of terrible.
- voltagex_ 13y ago
  Have you logged a bug report? I can't see anything that exists already from a quick search.
  - stormbrew 13y ago
    Not yet, I will probably do so later though.
est 13y ago
I secretly hope there's more adoption of client certs for browsers, bonus if could share the same cert or openssh and browser.

110 points by p4bl0 13y ago

Excellent article. I like the idea of running my own CA, but prefer the Userify disconnected model instead.

about time. this has been needed for a long time. ssh mitm attacks were too easy, and legitimately changing server keys (which happens more often than you would expect) would just confuse the issue. so - about time. tatu's decision to NOT use this feature of public key crypto when he designed ssh keys has always been a point of frustration for me.

eik3_de 13y ago

I stumbled across > Logging in directly as root user is a bad idea — even in examples further down.. > ~/ssh_cert_authority/user_ca $ scp -rp user_ca.pub root@example.host.net:/etc/ssh

dspillett 13y ago
Doeth what I say, not'th what I do... I think we are all guilty of this far more often then we like to admit. On this particular one, I use root logins far too often rather than observing best practise.
- eik3_de 13y ago
  well that's what PermitRootLogin no is for
  - peterwwillis 13y ago
    Ya know, people go on and on about disabling root logins. Have you ever had to write a piece of automation to login to a host as a non-root user, interact with a user interface, drop to a normal user shell, then run a sudo scp command, and then pass back non-interactive raw non-TTY access to an scp control program? It's hard. I know because I wrote a really crappy script that does it. You know what's so much easier and more stable? Running scp as root. Sadly, sometimes the best engineering choice is not a good one.
    - vertex-four 13y ago
      I'll point out that Ansible can do this just fine. Assuming you've got a non-root user set up in your node list file... ansible [server] --sudo -m copy -a "src=filename dest=filename" Other parts of Ansible might not be great (specifically, fully automated datacenter orchestration isn't possible without writing your own tools on top of Ansible; I use Salt for that), but it's really good for ad-hoc management without having to put additional infrastructure or daemons in place.
      - jamiesonbecker 13y ago
        cool - thx for this. also can use secure tmp files or rsync: rsync --rsync-cmd='sudo rsync' -rpave ssh \ /some/dir/ \ me@remote:/some/dir/ (or, pulling backups for best security) vault$ rsync --rsync-cmd='sudo rsync' -rpave ssh \ backup_user@dbserver:/var/ /backups/var/
    - skywhopper 13y ago
      Sounds like you have other problems besides just needing root access. It's always possible to come up with an edge case, but that doesn't invalidate the rule. Avoid direct root login at all costs.
      - peterwwillis 13y ago
        s/problems/requirements/. And I wasn't trying to invalidate a rule, but merely point out that rules are made to be broken ;)
      - dspillett 13y ago
        > It's always possible to come up with an edge case, but that doesn't invalidate the rule That is why we have risk registers in professional contexts. Whenever following policy (or the less specifically defined "accepted best practise") is impractical or inconvenient a note is made in the project's risk register with who made the decision, why, and who signed off on accepting the risk of not following policy in this case. Doing it this way forces people to ask themselves "is it really that impractical/inconvenient?", "have I properly considered the possible implications and calculated the risk (waving a hand and saying it'll all be fine because you've done it before does not constitute calculating the risk!)", and "have we applied as much mitigation to the risk as possible (enforcing secure password policy and so forth)". After considering those things then by all means allow password based remote logins for root and put your name against the choice in the register (or try get someone else to if you do not have sufficient authority).
  - e12e 13y ago
    I generally have both: PermitRootLogin no AllowGroups dial-in #or something similar The one time I need to grant someone login, I'll probably scratch my head why they can't log in, then remember to add them to the right group... [edit: and, of course: PasswordAuthentication no ]
- taejo 13y ago
  > Doeth what I say, not'th what I do... Just by the way, in the period of English with the -eth ending (Early Modern English), the correct grammar would be (wait for it) "Do what I say, not what I do".
neoCrimeLabs 13y ago
It's great to see someone who reads something in depth! Thanks for taking the time and commenting. :-) My point with with the comment "Logging in directly as root user is a bad idea — even in examples" was that you should have enough pre-requsite knowledge to understand why that is bad, and that you should not follow my examples 100%. That and I was poking fun at myself for being sloppy. :-)
- jamiesonbecker 13y ago
  nice :) rsync --rsync-path='sudo rsync' ... ssh remote sudo passwd -l root (note diffs between rhel and deb/ubu)
marcosdumay 13y ago
Is it really so bad? There are always priviledge escalation bugs open on Linux, and the attacker can always just try to guess the password, with several different sessions running su. I don't trust my systems to survive local atacks, or at least I trust them much, much less then surviving external attacks. Personaly, I have my machines on a hierarchy of trust. More trusted machines can ssh as root into less trusted one, but not the other way around. It's much easier to automatize things this way.
- jamiesonbecker 13y ago
  There is literally never a reason for one machine to log into another as root except for your convenience. Put stuff that needs to run as root into a shell script (or use rsync --rsync-cmd) and then use sudo.
IgorPartola 13y ago
You know what? Sometimes, it's fine. You should not share a root account with other users. This advice comes from a time where you'd have the "root password" and log into remote servers that way. Then, when someone leaves your organization, you'd hopefully change the root password to make sure they person that left no longer has access. However, there are some concrete cases where logging in as root is just fine: - Making a database backup remotely. You have to give the backup user enough privileges to initiate a database dump, stop the database process, grab the backup, rotate the backups, start the database process up again. You can go through the giant pain of setting up the permissions just so, or you can put a separate SSH public key into /root/.ssh/authorized_keys and just let root do all of this. - Initial VM setup. If you are automating something like provisioning EC2 instances, sure go ahead and set up individual users, but it's a lot easier to first connect as root using your own keypair. - A remote monitor that logs into a single purpose VM to restart a process. Same as the backups example, but more broad. Once again, you could set it up using specific users, but the system would be more brittle. - Lots of other cases where root is really what you want and doing sudo won't work because the session is not interactive. I figure it this way: if I compromised a local account on a server, I'd just add `alias sudo=~/.bin/my-evil-sudo-that-emails-your-password` to your .bashrc. That way I'd get root without any local exploits in the kernel, etc. Basically, adding extra protection around the root account does nothing if you are a sudoer. Some steps you can take to actually increase your security instead of following the advice that is so misunderstood: - Set the root password to a long random string, then lose the string. `su` should never be used by anyone but root. - Disable password logins. Period. Do this now, and do it everywhere. - Disable password-based sudo. Use pam_ssh_agent_auth [1] to escalate privileges and ssh-agent to forward your key through your ssh session. I am not sure why this isn't the default, but it really ought to be. - Make sure your local key is encrypted with a strong passphrase. Once again, use ssh-agent to not have to enter the password every single time. - Set up a VPN to access your servers and only allow ssh from the VPN. - For the truly paranoid, set up two-factor auth over SSH. This is easy to do with Google Authenticator (it's just a random number with the seed being shared and secret and the number being generated every 30 seconds), or using SMS which is just slightly more complex/costly. [1] http://pamsshagentauth.sourceforge.net/ http://pamsshagentauth.sourceforge.net/
- 13y ago

zobzu 13y ago

similar, but using gpg, also interesting http://web.monkeysphere.info/getting-started-ssh/ http://web.monkeysphere.info/getting-started-ssh/

IgorPartola 13y ago
This works really well with a couple of exceptions: 1. It slows down your initial connection while it does another connection to check the key 2. You must convince everyone to use it, which is hard since gpg is hard. I'd love to see more discussion of Monkeysphere on here. Every time I post something about it it gets no votes.

ehPReth 13y ago

Is there a way to have it use a CRL instead of running around and ensuring a certificate is marked as revoked on every device?

riskable 13y ago
Unfortunately, no. It's the key reason why this feature isn't widely adopted I think.
- ehPReth 13y ago
  Ah, darn. Thanks for the reply
cpach 13y ago
If you’re interested in proprietary solutions, I think the ”original” SSH server (from SSH Communications Security Oyj) can do stuff like that.

stormbrew 13y ago

This reminded me I wanted to investigate switching to this for our infrastructure, but uh... it seems that on at least ubuntu's build of openssh if you pass a serial number to ssh-keygen it segfaults. Even if it's the default (0). That's kind of terrible.

voltagex_ 13y ago
Have you logged a bug report? I can't see anything that exists already from a quick search.
- stormbrew 13y ago
  Not yet, I will probably do so later though.

est 13y ago

I secretly hope there's more adoption of client certs for browsers, bonus if could share the same cert or openssh and browser.