3 ms·

How do you deploy your Apps? We exclusively use EKS and having one account per env and app seems like quite an overhead when I think about managing / updating E

How do you deploy your Apps? We exclusively use EKS and having one account per env and app seems like quite an overhead when I think about managing / updating EKS clusters for each one. It also comes with an overhead of base applications that need to run in each cluster by default (like cert-manager, externaldns etc). Right now we’re using one account per env but also see downsides and thought of going the next step to do one account per env and tribe/team.

solatic 3y ago
GCP and AWS both support sharing a single VPC across multiple GCP projects / AWS accounts. See e.g.: https://aws.amazon.com/blogs/networking-and-content-delivery/vpc-sharing-a-new-approach-to-multiple-accounts-and-vpc-management/ https://aws.amazon.com/blogs/networking-and-content-delivery... It's not for the faint of heart though. You need to allocate subnets to individual applications (with relevant capacity planning concerns) plus support is sometimes spotty (e.g. EKS doesn't support it last time I checked). Not worth doing until you have several teams trying to use the same VPC and stepping on each others' toes.
swozey 3y ago
Each app/env has a pipeline that will trigger a tf apply in its directory w/ its assumed AWS role and deploy an env after someone gives it a manual approval after looking at the terraform apply/plan output. So it will start at /terrafrom/app1/staging then once healthchecks succeed another manual approval job for /terraform/app1/production will wait to be approved to depoy. For our EKS apps we do helm rollouts, but most of our services are on ECS so it's mostly just updating a task definition and forcing a redeployment of containers. Each EKS cluster is set up exactly the same aside from the usual things like vpc and ips and things of that nature that switch between them. They all get a set of "base" apps like log chutes and cert manager and all that as soon as they're deployed. Our app environments don't communicate with one another at all. The only relationship between them is our IAM accounts in our security account can assume access into them as admin/etc.

solatic 3y ago

GCP and AWS both support sharing a single VPC across multiple GCP projects / AWS accounts. See e.g.: https://aws.amazon.com/blogs/networking-and-content-delivery/vpc-sharing-a-new-approach-to-multiple-accounts-and-vpc-management/ https://aws.amazon.com/blogs/networking-and-content-delivery... It's not for the faint of heart though. You need to allocate subnets to individual applications (with relevant capacity planning concerns) plus support is sometimes spotty (e.g. EKS doesn't support it last time I checked). Not worth doing until you have several teams trying to use the same VPC and stepping on each others' toes.

swozey 3y ago

Each app/env has a pipeline that will trigger a tf apply in its directory w/ its assumed AWS role and deploy an env after someone gives it a manual approval after looking at the terraform apply/plan output. So it will start at /terrafrom/app1/staging then once healthchecks succeed another manual approval job for /terraform/app1/production will wait to be approved to depoy. For our EKS apps we do helm rollouts, but most of our services are on ECS so it's mostly just updating a task definition and forcing a redeployment of containers. Each EKS cluster is set up exactly the same aside from the usual things like vpc and ips and things of that nature that switch between them. They all get a set of "base" apps like log chutes and cert manager and all that as soon as they're deployed. Our app environments don't communicate with one another at all. The only relationship between them is our IAM accounts in our security account can assume access into them as admin/etc.