9 ms·
> Spot instances are underpriced, as most tenants underestimate their tolerance for preemption or overestimate its likelihood. > In reality, instance preemptio
by paulddraper 4y ago
> Spot instances are underpriced, as most tenants underestimate their tolerance for preemption or overestimate its likelihood.
> In reality, instance preemption is rare in most instance families
I have often seen the following:
1. The demand for spot instances increases for a period of time.
2. Supply is inelastic so prices quickly rise to the full bid amounts.
3. Most users are unwilling to go hours without any spot instances running, so they use very high bids.
You either have to tolerate hours of "down time" every few months, or pay exorbitant prices during once in a while.
On multiple occasions, I've seen the spot instances priced significantly higher than on-demand.
EDIT: Apparently AWS changed the Spot pricing a few years ago though. My anecdotes are 5 years old.
- vasco 4y ago> I've seen the spot instances priced significantly higher than on-demand How is the cap price not on-demand cost? Why would you not just swap to on-demand at the turning point?
- idunno246 4y agothere's a cost to engineer something that automatically switches or someone going in and manually changing it. so the spot prices has to be higher than ondemand + switching costs. the new pricing models(a couple years old) though have mostly alleviated this
- paulddraper 4y agoBecause you don't want to be pre-empted.
- alien_ 4y agoThis is no longer the case. Back in 2018 AWS changed the pricing model so now there's no more bidding. This means "bidding" more won't protect your capacity, AWS will just take it if they need it. Prices are now relatively flat, and changing slightly over time by a fraction of a cent per day with a cap at the OnDemand price, while previously there used to be lots of from 0.1x to 10x the on demand price and with swings even multiple times within a day.
- jacurtis 4y agoThis shocks me too. When I first had to design a system around spot instances I assumed it caps out at the on-demand price. But in practice that is very much not the case. The spot price can routinely go above on-demand, with all the downsides of spot still being applied to the instance. It is because many systems are only built to support spot. When running a workload, they might not want to give up availability so they just pay the higher price. To be fair, this is a game that AWS engineers. They essentially want to kick people off the infrastructure to free it up for other uses. So this is a way to signal how badly you don't want to be kicked off. A lot of times after a price goes above on-demand pricing, it will dip down below fairly quickly as workloads all re-adjust. Some companies are willing to play that gamble, that a short period above on-demand is still cheaper in the long run when you average out the invoice at the end of the month.
- psanford 4y agoWhat is the reason anyone would pay more for a spot instance than an on-demand instance?
- great_psy 4y agoMaybe poorly architected software, where it’s cheaper to increase the value of a variable than have a bunch of API calls that check the current onDemand price and call different APIs depending on which number is smaller.
- sokoloff 4y agoIf an instance on-demand is, for the sake of easy discussion, $1.00/hr and the spot rate is usually $0.30/hr, it might make sense to bid $1.25/hr (or $1.26/hr) if you didn’t want to get pre-empted. Most hours, you’d pay something close to $0.30/hr. Some hours, you’d pay over a $1.00/hr, but you’d save money overall against on-demand. (This ignores Reserved Instances and Savings Plans.)
- alien_ 4y agoThis is no longer the case since 2018, see my comment above.
- ericpauley 4y agoInteresting. While my data does show this being the case in a handful of pools (1-5 out of 13k pools at any given time), it appears to occur only exceedingly rarely.
- jacurtis 4y agoTo be fair, its also easier than ever before to use spot instances. If you set up an EKS/K8s cluster and install Karpenter. You can configure it very easily to use spot instances anytime prices are available for less than on-demand and to use on-demand when spot instances are unavailable or too expensive. You end up never thinking about it but having full availability. In practice it means the ceiling on your bill is the on-demand price, but you usually average out much lower. I suspect as more and more people switch over to this model then the use of spot instances will stay closer to full saturation, with those discounts becoming negligible.
- nickjj 4y agoI think this comes with the limitation that your workloads might be terminated without your control? They have a whole section in their docs about this: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-int... It's all about how your spot instances can be terminated. Perhaps this could interfere with a workload that might take a long while to finish. On paper your app should be resilient to these things but in practice it's essentially an unforgiving chaos monkey that will delete servers from your cluster. If you can handle that, you can definitely save a lot though. Especially web servers where it's expected you'll be responding in a few seconds while maybe your background workers are churning through longer lived tasks. The problem there is you need to already be big enough to have different nodes handling web vs worker traffic. A lot of decently sized apps can operate in a world where maybe you have 2-5 nodes on your cluster and your web and worker pods land on any of them based on Kubernetes' default scheduler.
- Cpoll 4y agoAfair Karpenter handles that. AWS gives some lead time before taking away spot instances. Karpenter puts a taint on the node and spins up a replacement. The scheduler gently evicts the pods, and nothing is lost. This does require architecting your services to be shut down tolerant, but that's par for the course. If you're using Kubernetes you've probably already settled on the "[herd, not] cattle, not pets" idea.