7 ms·
Yes! SSH certificates are awesome, both for host- and client-verification. Avoiding Trust on First Use is potentially a big benefit, but the workflow improveme
by rhco 4y ago
Yes! SSH certificates are awesome, both for host- and client-verification.
Avoiding Trust on First Use is potentially a big benefit, but the workflow improvements for developers, and especially non-technical people, is a huge win too.
At work, we switched to Step CA [1] about 2 years ago. The workflow for our developers looks like:
1. `ssh client-hosts-01`
2. Browser window opens prompting for AzureAD login
3. SSH connection is accepted
It really is that simple, and is extremely secure. During those 3 steps, we've verified the host key (and not just TOFU'd it!), verified the user identity, and verified that the user should have access to this server.
In the background, we're using `@cert-authority` for host cert verification. A list of "allowed principals" is embedded in the users' cert, which are checked against the hosts' authorized_principals [2] file, so we have total control over who can access which hosts (we're doing this through Azure security groups, so it's all managed at our Azure portal). The generated user cert lasts for 24 hours, so we have some protection against stolen laptops. And finally, the keys are stored in `ssh-agent`, so they work seamlessly with any app that supports `ssh-agent` (either the new Windows named pipe style, or "pageant" style via winssh-pageant [3]) - for us, that means VSCode, DBeaver, and GitLab all work nicely.
My personal wishlist addition for GitHub: Support for `@cert-authority` as an alternative to SSH/GPG keys. That would effectively allow us to delegate access control to our own CA, independent of GitHub.
[1] https://smallstep.com/docs/step-ca https://smallstep.com/docs/step-ca
[2] https://man.openbsd.org/sshd_config#AuthorizedPrincipalsFile https://man.openbsd.org/sshd_config#AuthorizedPrincipalsFile
[3] https://github.com/ndbeals/winssh-pageant https://github.com/ndbeals/winssh-pageant
- xyst 4y agoI like this solution, thanks for sharing. Just need to swap it with my own OIDC compliant federated authentication server.
- v7n 4y agoThat is also the flow for Tailscale SSH https://tailscale.com/kb/1193/tailscale-ssh/ https://tailscale.com/kb/1193/tailscale-ssh/
- deleted 4y ago[deleted]
- deathanatos 4y ago> we've verified the host key (and not just TOFU'd it!), How. Specifically, what I cannot determine from their docs is how the VM obtains a host key/cert signed by the CA. How does the CA know the VM is who the VM says it is? (I.e., the bootstrap problem.) (I assume that you also need your clients to trust the CA … and that has its own issues, but those are mostly human-space ones, to me. In theory, you can hand a dev a laptop pre-initialized with it.)
- mjg59 4y agoI can't say in the general sense, but with GCP you can retrieve the EKpub of a VM's TPM via the control plane, and then use https://github.com/google/go-attestation https://github.com/google/go-attestation to verify that an application key is associated with that TPM and hence that VM
- rhco 4y agoStepCA supports quite a few authentication methods, including an "admin provisioner" (basically a passphrase that can be pasted into the CLI tools' stdin). Because each of our servers are bespoke, we can use the admin provisioner when the server is first being set up (and actually, Ansible handles this part). I don't have experience with it, but StepCA also has Kubernetes support, and I imagine the control plane could authenticate the pod when a cert needs to be issued or renewed.
- tjoff 4y agoIf you are in the terminal and don't have access to a browser?
- hnlmorg 4y agoNot the OP but if anyone doesn’t have access to a browser in my org then I can safely say they’re not accessing from a company laptop and thus should be denied access.
- hamburglar 4y agoNot sure why you'd get downvoted for this comment. This is likely very applicable for many orgs that have operator workstation standards -- they're some kind of window/osx/linux box with a defined/enforced endpoint protection measures, and they all have a browser. Any device I can imagine ssh'ing from that doesn't have a browser is definitely out of policy.
- 0xedd 4y agobecause both of you narrow visioned the scenario to what you do daily. it is a common use case to ssh from a jump server, use ssh based CLI tools and debugging. the issue stems from windows users who are coupled to GUIs. the behavior pattern increases IT and DevOps costs unnecessarily. an alternative example: our org solves the issue with TOTP, required every 8 hours for any operation; from ssh/git CLI based actions (prompted at the terminal) to SSO integrations. decoupling security from unrelated programs. simple and elegant.
- Macha 4y agoThe -J parameter to say will transparently use a jump server and doesn't require the ssh key being on the third party server. I can't speak for tooling on step-ca but my employers in house tooling works similarly and loads the short lived signed cert into your ssh-agent so once you do the initial auth you can do whatever SSH things.
- hamburglar 4y ago
- sam_lowry_ 4y agoNow try to automate that.
- Deathmax 4y agoGitHub does have support for SSH CAs, but it's an Enterprise feature: https://docs.github.com/en/enterprise-cloud@latest/organizations/managing-git-access-to-your-organizations-repositories/about-ssh-certificate-authorities https://docs.github.com/en/enterprise-cloud@latest/organizat...
- rhco 4y agoThat's very interesting, thank you for linking!
- dmagyari 4y agoVault also supports both client and server ssh certificates [1]. I use terraform and vault to sign server certificates at creation time. [1] https://developer.hashicorp.com/vault/docs/secrets/ssh/signed-ssh-certificates https://developer.hashicorp.com/vault/docs/secrets/ssh/signe...
- deleted 4y ago[deleted]
- simiones 4y agoAt work, we switched to Step CA [1] about 2 years ago. The workflow for our developers looks like: 1. `ssh client-hosts-01` 2. Browser window opens prompting for AzureAD login 3. SSH connection is accepted How is that simple, compared to `ssh -i .ssh/my-cert.rsa someone@destination --> connection is accepted, here's your prompt` ?
- lucideer 4y agoThe former is discoverable: it doesn't require developers having ANY knowledge of command switches (no matter how basic) nor following a set of out-of-band instructions; the "how to" is included within the workflow.
- Aeolun 4y agoThis is simple because it doesn’t require you to take any specific actions to make new/different hosts accessible. If you deactivate someone in AD, poof, all their access is magically gone, instead of having to remove their public key from every server.
- mynameisvlad 4y agoIt’s the exact same command as a regular SSH prompt and it generates and uses the cert. that seems very simple. Your command is disingenuous in that it only works if the certificate has already been issued to you. If you were to include issuance, your command would very much turn non-simple.
- 4y ago
- misnome 4y agoOne thing I've never understood about SSH certificates for client identification - it looks like it causes the requirement that _at some point_ ssh private keys and the certificate private key need to both be in the same place? And if this is the case, then doesn't that imply that you need to have a service where users upload their private key? Which would mean you have one single point of attack/DOS/failure that needs to be kept utterly secure at all costs?
- karatinversion 4y agoI thought the way it worked was that the certificate signed with the certificate private key only contains the public key, and the ssh server, after checking the certificate is valid, validates that the client has the private key corresponding to the public key in the certificate.
- sokoloff 4y agoYou give your public key (typically into ~/.ssh/authorized_keys) and then prove you have access to the matching private key as the essential part of the challenge. You always keep the private key.
- calvinmorrison 4y agoAlso - key forwarding. Private key is on your local, you can forward it through ssh so you can hop around from your next destination
- counttheforks 4y agoHow do you get the browser to open? Does it work on all operating systems and ssh clients, such as Android's JuiceSSH?
- philo23 4y agoThis is a bit off topic, but does anyone know how the mechanism that triggers the web page prompt from an ssh connection actually works? Is it some kind of alternate ssh authentication method (like password/publickey) or something entirely out-of-band coming directly from the VPN app intercepting the connection? Ever since I saw it in action with Tailscale I've always wondered how it actually works, and I guess if anyone would know they'd be on HN
- fisian 4y agoSmallstep uses ProxyCommand [0]. Not sure how Tailscale does it. 0: https://smallstep.com/docs/ssh/how-it-works https://smallstep.com/docs/ssh/how-it-works
- v7n 4y agoOOB: ".. during the SSH protocol’s authentication phase, the Tailscale SSH server already knows who the remote party is and takes over, not requiring the SSH client to provide further proof (using the SSH authentication type none)." https://tailscale.com/kb/1193/tailscale-ssh/#how-does-it-work https://tailscale.com/kb/1193/tailscale-ssh/#how-does-it-wor...
- JohnFen 4y agoRequiring the use of a browser, though, limits the usefulness a bit.