9 ms·

Leveraging mispriced AWS spot instances

184 points by ericpauley 4y ago

pid-1 4y ago
> For example, you can make all these substitutions: >c6g.2xlarge→c6g.4xlarge→m6g.4xlarge→r6g.4xlarge→r6gd.4xlarge A long standing ticket in my personal project backlog is comparing different instance types performance. I'm not sure this equivalente is without caveats. Anyhow, the reason "misprices" exist is because: - Many AWS products are elastic but only allow one to choose a single instance type. So you need to guess the best instance for a workload and stick with it. - No AWS product exposes a "Just give me the cheapest VM with x CPU and Y memory" API
- mtrainier12345 4y ago
  There's actually an AWS API that does that. That's create-fleet in 'instant' mode. Docs are here: https://docs.aws.amazon.com/cli/latest/reference/ec2/create-fleet.html https://docs.aws.amazon.com/cli/latest/reference/ec2/create-... In short, you create a request specifying your CPU and memory constraints (you can also specify other constraints like processor manufacturer, memory per vCPU, etc). Then select lowest-price allocation strategy, and fleet "--type instant". Fleet will make a synchronous request provisioning the currently cheapest instance(s) which satisfy the constraints you selected, and will respond with the instance IDs.
- universa1 4y ago
  The end of the article shows a request for just that, doesn't it? No clue about the API though... Depending on your workload you might be able to actually substitute a single 8xlarge with two 4xlarge for example... A while back I was actually doing something like that to save some money :-)
  - pid-1 4y ago
    Wow totally missed that. Cool stuff!
- alFReD-NSH 4y ago
  Autoscaling group with mix instance type spot strategy does that. You can even give weights to instance type, giving more performant/higher capacity higher weight and it can choose the cheapest one with the weight in mind.
- pclmulqdq 4y ago
  AWS doesn't want you to have that API! That's a significant part of their margin.
Havoc 4y ago
Quite surprised that there is this degree of mispricing. I would have thought it’s a market that is big and diverse enough to iron that out. Especially given that the participants in question would tend toward the analytical side of things
- milesvp 4y ago
  I was thinking the same thing. I'm wondering if the price differences are reflecting a general demand for certain sizes? When I was maintaining AWS servers, I don't think it would have been easy for me to take advantage of spot prices that were outside of the sizes I was already using. I'd tuned things such that I knew the sizes I tended to need to have the redundancy I needed, and then could auto scale when necessary. Which means, I would never have bid on spot instances that were bigger than what I needed, because it would have been way more complicated to analyze the state of the system as a whole and make sure scaling happened when it needed to. Which also introduces risk that probably was never worth the savings. So if you had a lot of people like me, you'd get m3.large (or whatever current naming) as the thing that gets bid up the most, because it hit an autoscaling sweet spot
  - Havoc 4y ago
    > it would have been way more complicated to analyze the state of the system as a whole and make sure scaling happened when it needed to Yeah that's probably what's going on here. Complexity & that its just a bit counterintuitive
socialismisok 4y ago
Is there tooling to find the global minimum price for an instance with certain characteristics? I found it easy enough to do that in one region, but I've got some compute workloads that just read/write from S3 and are not latency sensitive. They do need 128 GB RAM and ephemeral disks.
- DJBunnies 4y ago
  > compute workloads that just read/write from S3 > need 128 GB RAM Eh?
  - tyingq 4y ago
    I took "just read/write from S3" to mean that they didn't interact with any other AWS services apart from S3. Such that they didn't care where in the world it ran. Not that they didn't do anything memory intensive.
    - socialismisok 4y ago
      You got it. It's some drone image processing. Read in data from S3, do analysis, write results.
- ericpauley 4y ago
  Spot fleet requests allow you to set minimum specs for instances, and the fleet will be composed of any instances that meet the spec. If it's asynchronous work, you could pick lowest price allocation and not worry too much about interruptions. In fact, if your work is tolerant of interruptions (batch size <2min), you can actually save even more by being interrupted, as you don't get billed for partial hours: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/billing-for-interrupted-spot-instances.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/billing-...
  - socialismisok 4y ago
    Does it launch the fleet at the global minimum price? "Any instances that meet the spec" doesn't seem like it's a worldwide price minimum.
bluelightning2k 4y ago
The article makes a HUGE assumption. They spot an inconsistency between two prices, and decide that the fair market value must be the very highest part of the spread. Anything under this is therefore "under-priced". Is it not possible instead that people are overpaying for the popular ones through sub-optimal bids - instead of simply assuming that only these inflexible/least sophisticated bidding strategies represent the fair market value. They actually go further. They assume that AWS could realize this value, and that encouraging more flexible bids through tooling etc. would move everyone to the top of the spread, instead of smoothing it out towards the average. And that what is essentially a price-increase can be achieved without hurting the overall value (price-performance vs. flexibility). Given the entire point of this is auctioning unused cycles at a discount, clearly any overall increase would decrease the overall demand. Having said this it's a great article. I think the overall quality of the article made it so surprising to see this missed.
- ericpauley 4y ago
  Author here. This is definitely a big assumption. I cut the price differences in half to account for market movement, but the price difference could definitely be more or less especially as these pools are probably thinner markets.
  - bluelightning2k 4y ago
    As I said - it's a great article! This was just one thing I noticed which I pointed out as it made me think. Keep up the good work
- bushbaba 4y ago
  There's underlying capacity as well. Would you rather pay a bit more to get 100 r6g.4xl OR pay a bit less to have 90 r6g.4xl + 10 r6gd.4xl. Many workloads do not have deployment configuration supporting a non-homogenous fleet of instances. Over time this will be addressed, but it could be a current major contributor to the discrepancies viewed.
Moissanite 4y ago
It is completely incorrect to characterize these observations as "mispricing" - this is a quirk of automatically-determined prices across very different products. If the author actually tried to use these instances in any significant volume they would understand the driver - capacity pools are nowhere near equal, and not as interchangeable for AWS as the article implies they would be for a user. Prices reflect demand munged with available capacity - uncommon instance types are uncommon precisely because they aren't used as much, so there aren't the same signals to drive the price up and down automatically. Instances with attached NVMe are available in much lower volumes than others, as are AMD instances. Obviously these pools cannot be used as a drop-in replacement for non-"d" instances or Intel families.
- ericpauley 4y ago
  Author here. The key here is that customers can leverage these pools in addition to their existing pools, improving capacity and price. AWS actually supports this out of the box (including substituting instances with drives) by specifying core and memory requirements directly instead of instance types.
  - Moissanite 4y ago
    Totally agree with that; it is a pretty common approach. The only part I don't agree with is calling out the price differences as some kind of "gotcha" that AWS somehow missed, particularly given the speculative "lost revenue" data which have no basis in reality.
    - ericpauley 4y ago
      See the emphasis on transparent substitutes in the article. This analysis is limited strictly to sets of instances that are fully hardware compatible, meaning AWS could resell one instance as another. There are way more savings to be had as a customer by leveraging instances that aren't transparent substitutes.
      - 4y ago
JoachimSchipper 4y ago
> Compute-optimized (C) instances can substitute for a general-purpose (M) instances of half the size They do have the same amount of memory (and twice the CPU). But if you run a workload that automatically scales to the number of available cores, starting twice the number of processes / threads might well run you out of memory. The article is interesting, but blindly running your code on unexpected instance types may be more "exciting" than the author makes it sound.
- ericpauley 4y ago
  Author here. If you design the workload you can ignore the extra instances. You can actually hide these cpu cores from instances within the AWS api (see setting instance vCPU) so it is truly transparent.
StratusBen 4y ago
As someone who spends entirely too much time thinking about cloud infrastructure costs ( I'm co-founder of https://www.vantage.sh/ https://www.vantage.sh/ which maintains https://ec2instances.info/ https://ec2instances.info/ ) I just want to recognize that amount of effort that went into this blog post to collect the data and express an interesting perspective for a fairly complicated topic. Kudos to the author on producing this.
- pradn 4y ago
  Great work on vantage.sh and ec2instances.info! Quick, small fix: This instance is shown as having 0 GBs of memory, but in fact it has 0.5 GB. https://instances.vantage.sh/aws/ec2/t4g.nano https://instances.vantage.sh/aws/ec2/t4g.nano
  - StratusBen 4y ago
    It's community supported! We just pay the bills and maintain hosting it :) Do you mind opening an issue on the repo here? https://github.com/vantage-sh/ec2instances.info https://github.com/vantage-sh/ec2instances.info Thank you for the report!
    - pradn 4y ago
      Looks like someone opened one :) https://github.com/vantage-sh/ec2instances.info/issues/666 https://github.com/vantage-sh/ec2instances.info/issues/666
- alex_duf 4y ago
  Oh I've used ec2instances.info very _very_ often, so thank you for that. So useful!
- awsthrowawy5767 4y ago
  You should know we use this tool _inside_ of AWS as well. Not in EC2 itself, but many many other places
- cebert 4y ago
phamilton 4y ago
We run a very large installation 100% on spot and have done for a few years. We serve our web traffic, do background work, etc. all on spot instances. We see similar mismatched pricing all the time and take advantage of it. One additional area not called out here is the difference between c5.24xlarge and c5.metal instance pricing. These are pretty much identical hardware but metal instances are often cheaper. As you go down this path, do expect to see a lot of weird things that you'll have to track down. For example, when we introduced metal instances we found that the default ubuntu AMI launched with a powersave cpu governor. Non-metal instances don't support CPU throttling so it never came up with c5.24xlarges. When we first launched metal instances the performance per instance was significantly worse and took a bit of work to track down. Recently we've seen a lot more spot interruptions and it's pushing us to incorporate more 6th gen instances to get us more diversity. We've also temporarily switched to capacity optimized over price optimized and we've enable capacity rebalancing. It's absolutely a win for us from a pricing perspective. Our traffic is extremely variable each day and very seasonal throughout the year. RIs don't make sense given <12 hrs daily peak and 10x difference between July and September. However, just plan for some odd surprises along the way.
- Moissanite 4y ago
  Have you observed metal instances taking longer to boot? I did last time I checked, and the difference was big enough to affect pricing in a non-trivial way, given that performance is the same and that you start paying immediately.
  - phamilton 4y ago
    This is a good point. They do take longer to boot, which might be part of the reason there's a discount, but it hasn't been so significant that we avoid them because diversification is important when running on spot.
  - khuey 4y ago
    Yes they take significantly longer to boot.
- inopinatus 4y ago
TheP1000 4y ago
If you want to leverage cheap spot, use us-east-2 / Ohio region. The prices are typically half of what you see in us-east-1. Also, it really helps to analyze at the AZ level. Certain AZs lack instances or have very low spot availability and contrary to recommended best practice, reducing AZs can sometimes be beneficial (I am looking at you eu-central-1a). While lowest price sounds nice, they can be really messy in terms of spot interruption rate. It is much better to set a max price and choose capacity optimized with as many instances as possible.
- playingalong 4y ago
  > eu-central-1a FYI, AZ names are not universal. Your eu-central-1a might be someone else's eu-central-1b.
  - dastbe 4y ago
    this actually depends on the region. amazon stopped randomizing az names in new regions quite awhile ago, while also offering azid as a guaranteed id in all regions.
hosh 4y ago
One thing to note is volatility. Spot instances are great for workloads that can absorb spot instance interruptions, and those interruptions tend to happen more if everyone else is trying to get spot instances at that time. Stateless web workloads that can startup and shutdown fast are a good example. Some workloads might not. You wouldn't want to run stateful workloads on spot, for instance. In our case, we have something that doesn't handle bootup under load very well, and until we can improve that, the overall reliability is not as good. I also like GCP's way of pricing these: you say whether your workload is preemptible or not, and you get discounts. You automatically get discounts if you run the workload for a long time.
jftuga 4y ago
I wrote a program to get AWS spot instance pricing. This program is similar to using "aws ec2 describe-spot-price-history" but is faster and has a few more options. https://github.com/jftuga/spotprice https://github.com/jftuga/spotprice
kureikain 4y ago
I run a service that has an API. which can help get spot price https://ec2.shop/ https://ec2.shop/ Simplify do: curl 'https://ec2.shop?region=us-west-2&filter=m5&json https://ec2.shop?region=us-west-2&filter=m5&json' | jq You can pipe it to whatever your system store to get the real time price without dealing with AWS Price API
benlivengood 4y ago
Interestingly GCP already offers over 75% discounts for n2d (AMD) spot instances that don't rely on any internal market, and the discounts for other families are fairly close. We see individual spot instances go away every few days which works pretty well for GKE. The older preemptible class of instances restarted every 24 hours which was more of a pain (mitigated a bit with a preemptible killer to spread the restarts out).
bscanlan 4y ago
Fun article, the phenomenon is interesting to see in practice, I've seen it regularly with newer instance types as it can take time for people to add them to their configurations. We're heavy users of spot here in Intercom. I spot-checked our biggest workload, and this week we could have paid around 10% less if we were able to get the cheapest spot host possible in us-east-1 that is suitable for our workload (all 16xlarge Gravitons). However that would be at the cost of fleet stability, I think that to run relatively large production services used in realtime on spot you need to prioritise fleet stability, so choosing the "Capacity Optimized" strategy. We've seen incessant fleet churn when trying out cost optimised strategies.
plantain 4y ago
I've spent a lot of time trying to capitalize on these mispricing - and often they're priced like that because the capacity in that region/configuration is much lower and you are exposed to more more preemptions than in higher priced region/configurations.
Galanwe 4y ago
Just curious, what kind of work flow do you have to run to accept that your host can stop anytime?
- Sebguer 4y ago
  Literally anything stateless.
- gabrielhidasy 4y ago
  Anything that you can easily checkpoint/finish quickly, while needing a ton of computers to do, map-reduce type jobs. For example, let's say you need to process a few million images, each taking a few seconds to process. You can start a manager task that distributes images, and a pool of interruptible workers, when a worker dies you just reissue the images to another.
  - Dylan16807 4y ago
    And lots of spot instance types can be automatically hibernated.
Aeolun 4y ago
My biggest problem with this is that AWS does not seem to have any (easy) API driven way to do this. Even if you need one instance you need to literally use their Fleet API to be able to specify these conditions. I just want to specify the allowed instances in my CreateInstance command. Create an instance in these subnets, with these allowed instances, preferably a spot instance, but if none exists I’ll be happy with a normal one.
perrygeo 4y ago
A few years ago, I noticed the common ec2 instance types (c5.large) prices spiking more frequently than their n and d variants (faster network and additional NVME disks respectively). This prompted me to learn how to use them. There are differences, especially if you actually want to take advantage of the additional resources. Once I learned how to use the d variants, to mount the NVME disks for maximum advantage, to deal with the lack of persistence, and so on... the d spot instances were a steal! A lot of up front cost to use them properly, but performance was excellent and the price remained at the market floor almost 100% of the time. In the long run it was cheaper than the same machine without the fast disk attached! I assumed someone would figure it out and my advantage would disappear as the market corrected. But I underestimated the sheer number number of ec2 instance types to evaluate! While it's true, having a diversity of instance types will help you keep spot costs down and ensure you're never priced out of that one blessed instance type - There's an upfront cost to make that system work though. Engineering teams just don't have the time to test on them all. At worst there are hidden differences that negatively impact performance, at best you might fail to take advantage of the resources and leave them idle (can your app even push 25Gbps or max out an NVME RAID).
sfev 4y ago
If you use EKS, we’ve had a great experience with Karpenter (karpenter.sh) It’ll look at your pods’ cpu and memory requests and choose an appropriate instance type for you, and the cheapest spots where appropriate

184 points by ericpauley 4y ago

> For example, you can make all these substitutions: >c6g.2xlarge→c6g.4xlarge→m6g.4xlarge→r6g.4xlarge→r6gd.4xlarge A long standing ticket in my personal project backlog is comparing different instance types performance. I'm not sure this equivalente is without caveats. Anyhow, the reason "misprices" exist is because: - Many AWS products are elastic but only allow one to choose a single instance type. So you need to guess the best instance for a workload and stick with it. - No AWS product exposes a "Just give me the cheapest VM with x CPU and Y memory" API

mtrainier12345 4y ago
There's actually an AWS API that does that. That's create-fleet in 'instant' mode. Docs are here: https://docs.aws.amazon.com/cli/latest/reference/ec2/create-fleet.html https://docs.aws.amazon.com/cli/latest/reference/ec2/create-... In short, you create a request specifying your CPU and memory constraints (you can also specify other constraints like processor manufacturer, memory per vCPU, etc). Then select lowest-price allocation strategy, and fleet "--type instant". Fleet will make a synchronous request provisioning the currently cheapest instance(s) which satisfy the constraints you selected, and will respond with the instance IDs.
universa1 4y ago
The end of the article shows a request for just that, doesn't it? No clue about the API though... Depending on your workload you might be able to actually substitute a single 8xlarge with two 4xlarge for example... A while back I was actually doing something like that to save some money :-)
- pid-1 4y ago
  Wow totally missed that. Cool stuff!
alFReD-NSH 4y ago
Autoscaling group with mix instance type spot strategy does that. You can even give weights to instance type, giving more performant/higher capacity higher weight and it can choose the cheapest one with the weight in mind.
pclmulqdq 4y ago
AWS doesn't want you to have that API! That's a significant part of their margin.

Havoc 4y ago

Quite surprised that there is this degree of mispricing. I would have thought it’s a market that is big and diverse enough to iron that out. Especially given that the participants in question would tend toward the analytical side of things

milesvp 4y ago
I was thinking the same thing. I'm wondering if the price differences are reflecting a general demand for certain sizes? When I was maintaining AWS servers, I don't think it would have been easy for me to take advantage of spot prices that were outside of the sizes I was already using. I'd tuned things such that I knew the sizes I tended to need to have the redundancy I needed, and then could auto scale when necessary. Which means, I would never have bid on spot instances that were bigger than what I needed, because it would have been way more complicated to analyze the state of the system as a whole and make sure scaling happened when it needed to. Which also introduces risk that probably was never worth the savings. So if you had a lot of people like me, you'd get m3.large (or whatever current naming) as the thing that gets bid up the most, because it hit an autoscaling sweet spot
- Havoc 4y ago
  > it would have been way more complicated to analyze the state of the system as a whole and make sure scaling happened when it needed to Yeah that's probably what's going on here. Complexity & that its just a bit counterintuitive

socialismisok 4y ago

Is there tooling to find the global minimum price for an instance with certain characteristics? I found it easy enough to do that in one region, but I've got some compute workloads that just read/write from S3 and are not latency sensitive. They do need 128 GB RAM and ephemeral disks.

DJBunnies 4y ago
> compute workloads that just read/write from S3 > need 128 GB RAM Eh?
- tyingq 4y ago
  I took "just read/write from S3" to mean that they didn't interact with any other AWS services apart from S3. Such that they didn't care where in the world it ran. Not that they didn't do anything memory intensive.
  - socialismisok 4y ago
    You got it. It's some drone image processing. Read in data from S3, do analysis, write results.
ericpauley 4y ago
Spot fleet requests allow you to set minimum specs for instances, and the fleet will be composed of any instances that meet the spec. If it's asynchronous work, you could pick lowest price allocation and not worry too much about interruptions. In fact, if your work is tolerant of interruptions (batch size <2min), you can actually save even more by being interrupted, as you don't get billed for partial hours: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/billing-for-interrupted-spot-instances.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/billing-...
- socialismisok 4y ago
  Does it launch the fleet at the global minimum price? "Any instances that meet the spec" doesn't seem like it's a worldwide price minimum.

bluelightning2k 4y ago

The article makes a HUGE assumption. They spot an inconsistency between two prices, and decide that the fair market value must be the very highest part of the spread. Anything under this is therefore "under-priced". Is it not possible instead that people are overpaying for the popular ones through sub-optimal bids - instead of simply assuming that only these inflexible/least sophisticated bidding strategies represent the fair market value. They actually go further. They assume that AWS could realize this value, and that encouraging more flexible bids through tooling etc. would move everyone to the top of the spread, instead of smoothing it out towards the average. And that what is essentially a price-increase can be achieved without hurting the overall value (price-performance vs. flexibility). Given the entire point of this is auctioning unused cycles at a discount, clearly any overall increase would decrease the overall demand. Having said this it's a great article. I think the overall quality of the article made it so surprising to see this missed.

ericpauley 4y ago
Author here. This is definitely a big assumption. I cut the price differences in half to account for market movement, but the price difference could definitely be more or less especially as these pools are probably thinner markets.
- bluelightning2k 4y ago
  As I said - it's a great article! This was just one thing I noticed which I pointed out as it made me think. Keep up the good work
bushbaba 4y ago
There's underlying capacity as well. Would you rather pay a bit more to get 100 r6g.4xl OR pay a bit less to have 90 r6g.4xl + 10 r6gd.4xl. Many workloads do not have deployment configuration supporting a non-homogenous fleet of instances. Over time this will be addressed, but it could be a current major contributor to the discrepancies viewed.

Moissanite 4y ago

It is completely incorrect to characterize these observations as "mispricing" - this is a quirk of automatically-determined prices across very different products. If the author actually tried to use these instances in any significant volume they would understand the driver - capacity pools are nowhere near equal, and not as interchangeable for AWS as the article implies they would be for a user. Prices reflect demand munged with available capacity - uncommon instance types are uncommon precisely because they aren't used as much, so there aren't the same signals to drive the price up and down automatically. Instances with attached NVMe are available in much lower volumes than others, as are AMD instances. Obviously these pools cannot be used as a drop-in replacement for non-"d" instances or Intel families.

ericpauley 4y ago
Author here. The key here is that customers can leverage these pools in addition to their existing pools, improving capacity and price. AWS actually supports this out of the box (including substituting instances with drives) by specifying core and memory requirements directly instead of instance types.
- Moissanite 4y ago
  Totally agree with that; it is a pretty common approach. The only part I don't agree with is calling out the price differences as some kind of "gotcha" that AWS somehow missed, particularly given the speculative "lost revenue" data which have no basis in reality.
  - ericpauley 4y ago
    See the emphasis on transparent substitutes in the article. This analysis is limited strictly to sets of instances that are fully hardware compatible, meaning AWS could resell one instance as another. There are way more savings to be had as a customer by leveraging instances that aren't transparent substitutes.
    - 4y ago

JoachimSchipper 4y ago

> Compute-optimized (C) instances can substitute for a general-purpose (M) instances of half the size They do have the same amount of memory (and twice the CPU). But if you run a workload that automatically scales to the number of available cores, starting twice the number of processes / threads might well run you out of memory. The article is interesting, but blindly running your code on unexpected instance types may be more "exciting" than the author makes it sound.

ericpauley 4y ago
Author here. If you design the workload you can ignore the extra instances. You can actually hide these cpu cores from instances within the AWS api (see setting instance vCPU) so it is truly transparent.

StratusBen 4y ago

As someone who spends entirely too much time thinking about cloud infrastructure costs ( I'm co-founder of https://www.vantage.sh/ https://www.vantage.sh/ which maintains https://ec2instances.info/ https://ec2instances.info/ ) I just want to recognize that amount of effort that went into this blog post to collect the data and express an interesting perspective for a fairly complicated topic. Kudos to the author on producing this.

pradn 4y ago
Great work on vantage.sh and ec2instances.info! Quick, small fix: This instance is shown as having 0 GBs of memory, but in fact it has 0.5 GB. https://instances.vantage.sh/aws/ec2/t4g.nano https://instances.vantage.sh/aws/ec2/t4g.nano
- StratusBen 4y ago
  It's community supported! We just pay the bills and maintain hosting it :) Do you mind opening an issue on the repo here? https://github.com/vantage-sh/ec2instances.info https://github.com/vantage-sh/ec2instances.info Thank you for the report!
  - pradn 4y ago
    Looks like someone opened one :) https://github.com/vantage-sh/ec2instances.info/issues/666 https://github.com/vantage-sh/ec2instances.info/issues/666
alex_duf 4y ago
Oh I've used ec2instances.info very _very_ often, so thank you for that. So useful!
awsthrowawy5767 4y ago
You should know we use this tool _inside_ of AWS as well. Not in EC2 itself, but many many other places
cebert 4y ago

phamilton 4y ago

We run a very large installation 100% on spot and have done for a few years. We serve our web traffic, do background work, etc. all on spot instances. We see similar mismatched pricing all the time and take advantage of it. One additional area not called out here is the difference between c5.24xlarge and c5.metal instance pricing. These are pretty much identical hardware but metal instances are often cheaper. As you go down this path, do expect to see a lot of weird things that you'll have to track down. For example, when we introduced metal instances we found that the default ubuntu AMI launched with a powersave cpu governor. Non-metal instances don't support CPU throttling so it never came up with c5.24xlarges. When we first launched metal instances the performance per instance was significantly worse and took a bit of work to track down. Recently we've seen a lot more spot interruptions and it's pushing us to incorporate more 6th gen instances to get us more diversity. We've also temporarily switched to capacity optimized over price optimized and we've enable capacity rebalancing. It's absolutely a win for us from a pricing perspective. Our traffic is extremely variable each day and very seasonal throughout the year. RIs don't make sense given <12 hrs daily peak and 10x difference between July and September. However, just plan for some odd surprises along the way.

Moissanite 4y ago
Have you observed metal instances taking longer to boot? I did last time I checked, and the difference was big enough to affect pricing in a non-trivial way, given that performance is the same and that you start paying immediately.
- phamilton 4y ago
  This is a good point. They do take longer to boot, which might be part of the reason there's a discount, but it hasn't been so significant that we avoid them because diversification is important when running on spot.
- khuey 4y ago
  Yes they take significantly longer to boot.
inopinatus 4y ago

TheP1000 4y ago

If you want to leverage cheap spot, use us-east-2 / Ohio region. The prices are typically half of what you see in us-east-1. Also, it really helps to analyze at the AZ level. Certain AZs lack instances or have very low spot availability and contrary to recommended best practice, reducing AZs can sometimes be beneficial (I am looking at you eu-central-1a). While lowest price sounds nice, they can be really messy in terms of spot interruption rate. It is much better to set a max price and choose capacity optimized with as many instances as possible.

playingalong 4y ago
> eu-central-1a FYI, AZ names are not universal. Your eu-central-1a might be someone else's eu-central-1b.
- dastbe 4y ago
  this actually depends on the region. amazon stopped randomizing az names in new regions quite awhile ago, while also offering azid as a guaranteed id in all regions.

hosh 4y ago

One thing to note is volatility. Spot instances are great for workloads that can absorb spot instance interruptions, and those interruptions tend to happen more if everyone else is trying to get spot instances at that time. Stateless web workloads that can startup and shutdown fast are a good example. Some workloads might not. You wouldn't want to run stateful workloads on spot, for instance. In our case, we have something that doesn't handle bootup under load very well, and until we can improve that, the overall reliability is not as good. I also like GCP's way of pricing these: you say whether your workload is preemptible or not, and you get discounts. You automatically get discounts if you run the workload for a long time.

jftuga 4y ago

I wrote a program to get AWS spot instance pricing. This program is similar to using "aws ec2 describe-spot-price-history" but is faster and has a few more options. https://github.com/jftuga/spotprice https://github.com/jftuga/spotprice

kureikain 4y ago

I run a service that has an API. which can help get spot price https://ec2.shop/ https://ec2.shop/ Simplify do: curl 'https://ec2.shop?region=us-west-2&filter=m5&json https://ec2.shop?region=us-west-2&filter=m5&json' | jq You can pipe it to whatever your system store to get the real time price without dealing with AWS Price API

benlivengood 4y ago

Interestingly GCP already offers over 75% discounts for n2d (AMD) spot instances that don't rely on any internal market, and the discounts for other families are fairly close. We see individual spot instances go away every few days which works pretty well for GKE. The older preemptible class of instances restarted every 24 hours which was more of a pain (mitigated a bit with a preemptible killer to spread the restarts out).

bscanlan 4y ago

Fun article, the phenomenon is interesting to see in practice, I've seen it regularly with newer instance types as it can take time for people to add them to their configurations. We're heavy users of spot here in Intercom. I spot-checked our biggest workload, and this week we could have paid around 10% less if we were able to get the cheapest spot host possible in us-east-1 that is suitable for our workload (all 16xlarge Gravitons). However that would be at the cost of fleet stability, I think that to run relatively large production services used in realtime on spot you need to prioritise fleet stability, so choosing the "Capacity Optimized" strategy. We've seen incessant fleet churn when trying out cost optimised strategies.

plantain 4y ago

I've spent a lot of time trying to capitalize on these mispricing - and often they're priced like that because the capacity in that region/configuration is much lower and you are exposed to more more preemptions than in higher priced region/configurations.

Galanwe 4y ago

Just curious, what kind of work flow do you have to run to accept that your host can stop anytime?

Sebguer 4y ago
Literally anything stateless.
gabrielhidasy 4y ago
Anything that you can easily checkpoint/finish quickly, while needing a ton of computers to do, map-reduce type jobs. For example, let's say you need to process a few million images, each taking a few seconds to process. You can start a manager task that distributes images, and a pool of interruptible workers, when a worker dies you just reissue the images to another.
- Dylan16807 4y ago
  And lots of spot instance types can be automatically hibernated.

Aeolun 4y ago

My biggest problem with this is that AWS does not seem to have any (easy) API driven way to do this. Even if you need one instance you need to literally use their Fleet API to be able to specify these conditions. I just want to specify the allowed instances in my CreateInstance command. Create an instance in these subnets, with these allowed instances, preferably a spot instance, but if none exists I’ll be happy with a normal one.

perrygeo 4y ago

A few years ago, I noticed the common ec2 instance types (c5.large) prices spiking more frequently than their n and d variants (faster network and additional NVME disks respectively). This prompted me to learn how to use them. There are differences, especially if you actually want to take advantage of the additional resources. Once I learned how to use the d variants, to mount the NVME disks for maximum advantage, to deal with the lack of persistence, and so on... the d spot instances were a steal! A lot of up front cost to use them properly, but performance was excellent and the price remained at the market floor almost 100% of the time. In the long run it was cheaper than the same machine without the fast disk attached! I assumed someone would figure it out and my advantage would disappear as the market corrected. But I underestimated the sheer number number of ec2 instance types to evaluate! While it's true, having a diversity of instance types will help you keep spot costs down and ensure you're never priced out of that one blessed instance type - There's an upfront cost to make that system work though. Engineering teams just don't have the time to test on them all. At worst there are hidden differences that negatively impact performance, at best you might fail to take advantage of the resources and leave them idle (can your app even push 25Gbps or max out an NVME RAID).

sfev 4y ago

If you use EKS, we’ve had a great experience with Karpenter (karpenter.sh) It’ll look at your pods’ cpu and memory requests and choose an appropriate instance type for you, and the cheapest spots where appropriate