4 ms·

If your code can survive chaos monkey, it can trivially weather this. Netflix, as one example, could run their entire infrastructure on these VMs.

snewman 5y ago
Except that (presumably) there is no guarantee that any spot instances are available. You could wind up in a state where you have no instances running. No user-facing production service can tolerate that.
- roca 5y ago
  Just spin up regular VMs if you can't get spot instances when you need them.
  - deleted 5y ago
    [deleted]
  - xyzzyz 5y ago
    Lack of spot VM means that all spare capacity has been taken up by the regular VMs, meaning that it spots are gone, so are regular ones.
    - roca 5y ago
      Maybe that's true but we haven't seen this happen in practice. Maybe we're just incredibly lucky but we're able to get on-demand instances to recover after spot termination.
    - apendleton 5y ago
      Having worked on infrastructure deployed this way, I can say this oversimplifies a bit. There's virtually never no spot capacity at all, there's no capacity at a given bid price, for a given instance type, in a given region. You can start to scale up on-demand infrastructure using spot price as a signal, before there's literally none left, and you can also run on clusters of heterogeneous instances, and change your instance type mix when one instance type's price goes up. In the worst case you can fail out of a particular region whose spot prices have gotten too high. Lots of options.
      - sitkack 5y ago
        No shade, but this is a very AWS specific view of cloud. Every cloud has a different resource allocation philosophy, not all the assumptions are the same.
        remram 5y ago
        Yeah, GCP spot instances don't vary in price like AWS ("The price adjustments will occur at most once a month").
        ianburrell 5y ago
        There is no longer bidding for spot prices on AWS. They moved to model where pay the fixed spot price which isn't based on capacity. https://aws.amazon.com/blogs/compute/new-amazon-ec2-spot-pricing/ https://aws.amazon.com/blogs/compute/new-amazon-ec2-spot-pri...
        remram 5y ago
        Interesting! I think it still updates more than once a month, but that is a significant change that I wasn't aware of, thank you.
- jayd16 5y ago
  No guarantee but still very likely there are instances available.
- jpdb 5y ago
  Spot is excess capacity. If there's zero excess capacity, that means there's (probably) also no on-demand capacity as well.
  - remram 5y ago
    If you are not being given any excess capacity, that doesn't mean there is no excess capacity. You are just not one of the lucky few, whose numbers are shrinking.
- roland35 5y ago
  I believe AWS can create spot autoscaling groups which include a minimum baseline traditional instances plus a floating level of spot instances. This seems to be a nice compromise.
forty 5y ago
Well except that on peak hours their infrastructure would disappear :) You always need some traditional instances to serve things that need to be available at all time, like your website. Now you could run things using spot instances and switch to traditional during peak hours but that requires confidence that enough capacity is going to be available when you need it (which is going to be at the same time as everyone else, so it's always a bet :))
- cma 5y ago
  Apparently some just gamble that by having some instances of every time, they won't all go down: "However, your cloud provider can pull these virtual machines out from under you at any time without notice (because someone else is willing to pay more). So how do you lower the probability of losing all of your instances to nearly zero? Cloud providers generally need to have a large buffer of capacity and they have many different virtual machine configurations with different CPU and memory sizes. So, don’t just spin up virtual machines of a single configuration type, spin up lots of them from many different configuration types!" https://blog.comma.ai/scaling-for-10x-user-growth/ https://blog.comma.ai/scaling-for-10x-user-growth/ I'm not sure if they still have a fallback of traditional instances.
  - forty 5y ago
    Yes absolutely. On AWS you can use spot instance fleets to implement this strategy https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/how-spot-fleet-works.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/how-spot...

snewman 5y ago

Except that (presumably) there is no guarantee that any spot instances are available. You could wind up in a state where you have no instances running. No user-facing production service can tolerate that.

roca 5y ago
Just spin up regular VMs if you can't get spot instances when you need them.
- deleted 5y ago
  [deleted]
- xyzzyz 5y ago
  Lack of spot VM means that all spare capacity has been taken up by the regular VMs, meaning that it spots are gone, so are regular ones.
  - roca 5y ago
    Maybe that's true but we haven't seen this happen in practice. Maybe we're just incredibly lucky but we're able to get on-demand instances to recover after spot termination.
  - apendleton 5y ago
    Having worked on infrastructure deployed this way, I can say this oversimplifies a bit. There's virtually never no spot capacity at all, there's no capacity at a given bid price, for a given instance type, in a given region. You can start to scale up on-demand infrastructure using spot price as a signal, before there's literally none left, and you can also run on clusters of heterogeneous instances, and change your instance type mix when one instance type's price goes up. In the worst case you can fail out of a particular region whose spot prices have gotten too high. Lots of options.
    - sitkack 5y ago
      No shade, but this is a very AWS specific view of cloud. Every cloud has a different resource allocation philosophy, not all the assumptions are the same.
      - remram 5y ago
        Yeah, GCP spot instances don't vary in price like AWS ("The price adjustments will occur at most once a month").
        ianburrell 5y ago
        There is no longer bidding for spot prices on AWS. They moved to model where pay the fixed spot price which isn't based on capacity. https://aws.amazon.com/blogs/compute/new-amazon-ec2-spot-pricing/ https://aws.amazon.com/blogs/compute/new-amazon-ec2-spot-pri...
        remram 5y ago
        Interesting! I think it still updates more than once a month, but that is a significant change that I wasn't aware of, thank you.
jayd16 5y ago
No guarantee but still very likely there are instances available.
jpdb 5y ago
Spot is excess capacity. If there's zero excess capacity, that means there's (probably) also no on-demand capacity as well.
- remram 5y ago
  If you are not being given any excess capacity, that doesn't mean there is no excess capacity. You are just not one of the lucky few, whose numbers are shrinking.
roland35 5y ago
I believe AWS can create spot autoscaling groups which include a minimum baseline traditional instances plus a floating level of spot instances. This seems to be a nice compromise.

forty 5y ago

Well except that on peak hours their infrastructure would disappear :) You always need some traditional instances to serve things that need to be available at all time, like your website. Now you could run things using spot instances and switch to traditional during peak hours but that requires confidence that enough capacity is going to be available when you need it (which is going to be at the same time as everyone else, so it's always a bet :))

cma 5y ago
Apparently some just gamble that by having some instances of every time, they won't all go down: "However, your cloud provider can pull these virtual machines out from under you at any time without notice (because someone else is willing to pay more). So how do you lower the probability of losing all of your instances to nearly zero? Cloud providers generally need to have a large buffer of capacity and they have many different virtual machine configurations with different CPU and memory sizes. So, don’t just spin up virtual machines of a single configuration type, spin up lots of them from many different configuration types!" https://blog.comma.ai/scaling-for-10x-user-growth/ https://blog.comma.ai/scaling-for-10x-user-growth/ I'm not sure if they still have a fallback of traditional instances.
- forty 5y ago
  Yes absolutely. On AWS you can use spot instance fleets to implement this strategy https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/how-spot-fleet-works.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/how-spot...