> In my company's current AWS infrastructure, there's no shell access to either the production containers or the host machines. I did write a script to create a

> In my company's current AWS infrastructure, there's no shell access to either the production containers or the host machines. I did write a script to create an ephemeral container that lets me (and future staff) run a shell inside the production network. I'm a fan of this approach, it is hard improve upon the security of a server that doesn't exist. I recently setup an AWS serverless (mainly ECS Fargate) stack for a project and took this approach of spinning up an ephemeral EC2 server as part of a "breakglass" runbook for the rare cases where such access is needed. This, combined with Tailscale userspace networking [0] (so the "breakglass" servers can run on a private subnet and are never exposed to the internet), Pulumi [1] (for managing the lifecycle of the "breakglass" instances) and Yubikey based MFA for short lived credentials [2] (required to spin up the server via Pulumi), I found to work well. This approach is also useful for ensuring that whenever a "breakglass" server is started it is using the latest AMI version (Pulumi's `aws.ec2.get_ami_output()` is useful for this) and runs the usual security updates on startup. The ssh keypair for the server can also be created (and later destroyed) on the fly so there is no need to manage any long term ssh credentials. [0] https://tailscale.com/kb/1113/aws-lambda/ https://tailscale.com/kb/1113/aws-lambda/ [1] https://www.pulumi.com/ https://www.pulumi.com/ [2] https://aws.amazon.com/blogs/security/enhance-programmatic-access-for-iam-users-using-yubikey-for-multi-factor-authentication/ https://aws.amazon.com/blogs/security/enhance-programmatic-a...