3 ms·

Run EC2 instances in an autoscaling group. Provision them with whatever you're comfortable with. Ansible is nice because you can point it at a node to test the

by dolni 6y ago

Run EC2 instances in an autoscaling group. Provision them with whatever you're comfortable with. Ansible is nice because you can point it at a node to test the deploy out without having to always respin the node to try again. Have ansible fire up a docker container on your host. That's it, done. You don't need Kubernetes. Using EC2 segregates everything nicely out of the box. You don't need to be concerned about resource contention or figuring out how to size instances/pods. A container going nuts and filling up your disk won't cause a bunch of other services to experience errors. While convenient for deploys, the biggest issue with Kubernetes is that you introduce an entire new layer of technology that you need expertise in. If you're running your own EC2 instances anyway (which seems likely, since spot pricing is amazing compared to managed stuff) you now have to know how autoscaling works, plus a VPC, plus Kubernetes. The way Kubernetes interacts with cloud providers (at least AWS) adds a ton of complexity. Also, if you have a decent build process for AMIs you can insulate yourself from container registry outages by baking the images into an AMI. There's something really powerful about this approach. If you build an AMI properly, that thing will run _forever_ with a minimal set of external service dependencies.

lkrubner 6y ago
This can go a very long way. Using Ansible and Packer to provision AMIs, and then Terraform to spin up what's needed, goes a long way, without introducing a new layer of technology.
AYBABTME 6y ago
But then you need to make up a deployment system with rollbacks and all the other stuff k8s gives you. I've got the feeling that if you're reasonably comfortable with k8s, there's no harm in starting with it from day 1.
- dolni 6y ago
  > But then you need to make up a deployment system with rollbacks and all the other stuff k8s gives you. This is borderline trivial. If you build an AMI, you update a launch template with the new AMI and use instance refresh (https://aws.amazon.com/blogs/compute/introducing-instance-refresh-for-ec2-auto-scaling/ https://aws.amazon.com/blogs/compute/introducing-instance-re...) to replace instances. Rolling back is the same thing, you just set it back to the previous AMI. Instance refresh took way too long to become a thing, but it was not even very difficult before that. List instances in your autoscaling group, mark them unhealthy one by one. Now, you'll probably want a couple other safety checks in your deployment process to make sure it all goes smoothly. But you're realistically looking at what, a couple of days of work to have something built to your specifications that can deploy any of your services without introducing the added complexity of Kubernetes? > I've got the feeling that if you're reasonably comfortable with k8s, there's no harm in starting with it from day 1. The harm is not that the deployments are hard. In fact, they're quite nice. The problem is that small teams don't have the bandwidth to keep up with the relentless release cadence. They may not have the expertise to get the upgrades done confidently. There is also the fact that, when something does go wrong, you've got a whole lot of complex Kube stuff to churn through. None of that applies to managed Kube, but then you're paying extra for Kube every month. By contrast, you could have just paid extra one time for a relatively simple deployment script in CI.
  - KptMarchewa 6y ago
    Managed kube costs $0.10 per cluster per hour for GKE, and the first one is free. 72$ monthly is nothing.
    - p_l 6y ago
      Dealing with release cadence with GKE was mostly "hey, why the new version is not there yet" :) About the only real issue I encountered was with major API change involving ingresses, when the cluster would continue running, but we had to update our manifests that were hamfistedly done back in 1.4 days.

by dolni 6y ago

lkrubner 6y ago

This can go a very long way. Using Ansible and Packer to provision AMIs, and then Terraform to spin up what's needed, goes a long way, without introducing a new layer of technology.

AYBABTME 6y ago

But then you need to make up a deployment system with rollbacks and all the other stuff k8s gives you. I've got the feeling that if you're reasonably comfortable with k8s, there's no harm in starting with it from day 1.

dolni 6y ago
> But then you need to make up a deployment system with rollbacks and all the other stuff k8s gives you. This is borderline trivial. If you build an AMI, you update a launch template with the new AMI and use instance refresh (https://aws.amazon.com/blogs/compute/introducing-instance-refresh-for-ec2-auto-scaling/ https://aws.amazon.com/blogs/compute/introducing-instance-re...) to replace instances. Rolling back is the same thing, you just set it back to the previous AMI. Instance refresh took way too long to become a thing, but it was not even very difficult before that. List instances in your autoscaling group, mark them unhealthy one by one. Now, you'll probably want a couple other safety checks in your deployment process to make sure it all goes smoothly. But you're realistically looking at what, a couple of days of work to have something built to your specifications that can deploy any of your services without introducing the added complexity of Kubernetes? > I've got the feeling that if you're reasonably comfortable with k8s, there's no harm in starting with it from day 1. The harm is not that the deployments are hard. In fact, they're quite nice. The problem is that small teams don't have the bandwidth to keep up with the relentless release cadence. They may not have the expertise to get the upgrades done confidently. There is also the fact that, when something does go wrong, you've got a whole lot of complex Kube stuff to churn through. None of that applies to managed Kube, but then you're paying extra for Kube every month. By contrast, you could have just paid extra one time for a relatively simple deployment script in CI.
- KptMarchewa 6y ago
  Managed kube costs $0.10 per cluster per hour for GKE, and the first one is free. 72$ monthly is nothing.
  - p_l 6y ago
    Dealing with release cadence with GKE was mostly "hey, why the new version is not there yet" :) About the only real issue I encountered was with major API change involving ingresses, when the cluster would continue running, but we had to update our manifests that were hamfistedly done back in 1.4 days.