3 ms·
MFA for ssh means you can't automate cluster-level ops. Unless, well, you automate the MFA, which defeats the MFA. This drives me a bit nuts about security peo
by AtlasBarfed 5y ago
MFA for ssh means you can't automate cluster-level ops. Unless, well, you automate the MFA, which defeats the MFA.
This drives me a bit nuts about security people in the age of cloudscale. They assume you don't mind MFA'ing every hour and are manually doing logins and accesses for everything. Yeah, uh, I need to script orchestration on several hundred machines at once, and orchestrate/access on the scale of hours or even days for some things like "Big Data" backups or restores.
If certificates are anything like SSL certs and the horrorshow cli tools / options / management involved in those, no thanks. I'd rather have an automated sshkey switchover, or for stateless just routinely cycle the infrastructure with new keys.
It's been a long time tenet of security that you want an open algorithm that gets broadly and publicly challenged so you know its secure. Well, in the age of the state actors, this might not be the whole truth.
I think layering some klugy not-invented-here obfuscation atop the more battletested methods is a useful and important deterrent/delay. Sure someone will figure it out, but they have to TRY HARD. A lot of the institutional attacks seem to be based on human attacks on standardized systems, which HAVE to allow human access and vectors.
So in AWS land, secrets manager is secure unless you get the permissions. Then you have everything. But if each of those secrets has some whacko obfuscation for the various apps, then that is a big slowdown to the human attack vectors.
And the state actors? Well, even they have budgets. They'll probably move on to easier targets. If a state actor is motivated at targeting your company in particular, well, given that they'll have malware in the firmware of your hard drives and motherboards and the like, you're probably helpless.
Finally, what really bothers me about most security is that it leaves one of the most important "canaries in the coal mines" aspects of security: honeypots. Sure do the diligence on securing the access, but how about some turnkey approaches for setting up honeypots to detect when people are poking around? Honeypots are perfect for that, because the devs only care about the stuff they are working on. The intruders are doing the scanning.
- fishpen0 5y agoCluster-level ops really shouldn't be done by direct SSH. Tools like ansible, salt, chef, puppet, etc... all can be run in some form of daemon mode with a central management server for a reason. You should be authenticating against the management service and running your configuration or automation scripts from there, not as a massive pool of CSSH or whatever directly from your laptop. Over the last 6 years I've worked for three different companies that all universally disabled ssh and we never had troubles running management scripts or tooling.
- outworlder 5y agoUntil the daemon breaks and someone has to get in with SSH anyway? Chef is notorious for doing this. Or at least it was before we got rid of it. Some random script would break and then the run would be incomplete. And we couldn't just fix cookbooks as the run wouldn't complete. Some deployments make the daemon approach (that phones home) difficult. Such as management in a corporate network. It's easy to configure AWS and the like to accept requests from well known corporate gateways. It's not as easy to make them from the outside the corporate network in. And even when that's doable, different cloud providers and regions make it difficult. You end up having a bunch of chef (or similar) servers scattered around.
- fishpen0 5y agoIn the rare case this happens we still don't use raw SSH. We rely on something identity-driven like SSM in AWS or IAP in GCP to initiate the tunnel.
- AtlasBarfed 5y agoGCP IAP sounds like Teleport, which we've already run into issues with since the Teleport daemon will die/not accept connections in some situations, while the good ol sshd does. Like: full disks, memory stress, or (I think) the teleport daemon getting killed. SSM sounds like an advanced port knock. Or you could toggle the security group port access, or keep the bastion down and spin it up if you need it.
- AtlasBarfed 5y agoStateful databases or disposable api servers? Disposable API servers you can eliminate ssh access and the like. Containers generally don't run sshd, but they do still often have kubectl/dockerrun if you absolutely need to. SSM sucks on a certain level because the output is capped at ?1MB? I think and you need to poll S3 to use it. Salt daemon polls fine. As you kind of alluded to, well, you can do SSM to "port knock" or simply do a change to a secgrp to flip on the ssh access, and then flip it off afterward. I find Salt/Ansible/k8s too big, you can't "step through" to debug your orchestrations. I would categorize them as "heavyweight". SSH is a good substrate for everything else, including adhoc stuff. Anything that can be run off a laptop can be run off an admin server. Remotely debugged. I love it. With the heavyweight stuff there is too much trial and error: try recipe, it craps, guess what's wrong, it craps, guess again, it craps. The stuff I use actually doesn't require SSH. As long as you can deliver a command and get the output, I can use SSH, SSM (with the crappy limitations but its GREAT for stuff in China), kubectl, salt, dockerrun, teleport (until the token runs out), or combo ssh-to-bastion then do other stuff.
- artificialLimbs 5y agoYou mean like Thinkst Canary? Or OpenCanary if you can't afford $arm&leg?
- AtlasBarfed 5y agoI am not a security consultant, but I've never heard of either. Which says something... but then again if the security group is doing its job with setting up canaries, why would peon dev like me even be awares?