4 ms·

To be fair, its also easier than ever before to use spot instances. If you set up an EKS/K8s cluster and install Karpenter. You can configure it very easily to

To be fair, its also easier than ever before to use spot instances. If you set up an EKS/K8s cluster and install Karpenter. You can configure it very easily to use spot instances anytime prices are available for less than on-demand and to use on-demand when spot instances are unavailable or too expensive. You end up never thinking about it but having full availability. In practice it means the ceiling on your bill is the on-demand price, but you usually average out much lower. I suspect as more and more people switch over to this model then the use of spot instances will stay closer to full saturation, with those discounts becoming negligible.

nickjj 3y ago
I think this comes with the limitation that your workloads might be terminated without your control? They have a whole section in their docs about this: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-int... It's all about how your spot instances can be terminated. Perhaps this could interfere with a workload that might take a long while to finish. On paper your app should be resilient to these things but in practice it's essentially an unforgiving chaos monkey that will delete servers from your cluster. If you can handle that, you can definitely save a lot though. Especially web servers where it's expected you'll be responding in a few seconds while maybe your background workers are churning through longer lived tasks. The problem there is you need to already be big enough to have different nodes handling web vs worker traffic. A lot of decently sized apps can operate in a world where maybe you have 2-5 nodes on your cluster and your web and worker pods land on any of them based on Kubernetes' default scheduler.
- Cpoll 3y ago
  Afair Karpenter handles that. AWS gives some lead time before taking away spot instances. Karpenter puts a taint on the node and spins up a replacement. The scheduler gently evicts the pods, and nothing is lost. This does require architecting your services to be shut down tolerant, but that's par for the course. If you're using Kubernetes you've probably already settled on the "[herd, not] cattle, not pets" idea.
  - nickjj 3y ago
    > The scheduler gently evicts the pods, and nothing is lost. If you have heavy duty work being done, it might get kill -9'd after the grace period is surpassed. If you weren't using spot instances you have full control over the grace period. You can of course handle this too, by breaking up your work in ways where it can be interrupted without losing progress but depending on what you're doing this could be very complicated. Basically the takeaway is you can't blindly start using spot instances, even if your app has been running in Kubernetes successfully for a while.
    - Demiurge 3y ago
      If you're writing your processing code, you can also write it with capability to stop and resume more efficiently.

nickjj 3y ago

I think this comes with the limitation that your workloads might be terminated without your control? They have a whole section in their docs about this: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-int... It's all about how your spot instances can be terminated. Perhaps this could interfere with a workload that might take a long while to finish. On paper your app should be resilient to these things but in practice it's essentially an unforgiving chaos monkey that will delete servers from your cluster. If you can handle that, you can definitely save a lot though. Especially web servers where it's expected you'll be responding in a few seconds while maybe your background workers are churning through longer lived tasks. The problem there is you need to already be big enough to have different nodes handling web vs worker traffic. A lot of decently sized apps can operate in a world where maybe you have 2-5 nodes on your cluster and your web and worker pods land on any of them based on Kubernetes' default scheduler.

Cpoll 3y ago
Afair Karpenter handles that. AWS gives some lead time before taking away spot instances. Karpenter puts a taint on the node and spins up a replacement. The scheduler gently evicts the pods, and nothing is lost. This does require architecting your services to be shut down tolerant, but that's par for the course. If you're using Kubernetes you've probably already settled on the "[herd, not] cattle, not pets" idea.
- nickjj 3y ago
  > The scheduler gently evicts the pods, and nothing is lost. If you have heavy duty work being done, it might get kill -9'd after the grace period is surpassed. If you weren't using spot instances you have full control over the grace period. You can of course handle this too, by breaking up your work in ways where it can be interrupted without losing progress but depending on what you're doing this could be very complicated. Basically the takeaway is you can't blindly start using spot instances, even if your app has been running in Kubernetes successfully for a while.
  - Demiurge 3y ago
    If you're writing your processing code, you can also write it with capability to stop and resume more efficiently.