12 ms·

Farewell to the Era of Cheap EC2 Spot Instances

254 points by ericpauley 3y ago

austinshea 3y ago
This seems to ignore the fact that the prices are consistently low if you are tracking the lesser used instance types/regions, even when talking explicitly within AWS. Why would that change?
- ericpauley 3y ago
  It's tough to know for sure. Many of the instance types in these regions are at the price floor, so there could be increases in aggregate demand that just don't show up because the DCs are overbuilt. These markets are thinner so if demand shifts to them (big if because moving regions is hard) we might quickly see price spikes.
- cyberax 3y ago
  Spot prices don't directly reflect capacity. EC2 Spot used to be a real spot market, with actual real auctions where higher bids displaced lower bids. But it was changed a while ago, so prices are now set algorithmically based on predicted demand/supply. We run stateless calculations on EC2 across regions, and we definitely see that instances are harder to come by. Especially instances with GPUs. And for many instance types, the price advantage of EC2 Spot compared to committed spending is not significant anymore.
  - MichaelZuo 3y ago
    Huh, do you remember what the rationale for the change was?
    - derefr 3y ago
      I would guess they were seeing thundering herds of bots trying to fight to grab spot-instance stock whenever any appeared below the bots' set cost threshold. Basically like scalper bots.
    - cyberax 3y ago
      Well, I was one of the engineers that made the change :) I'm not sure how much I can tell, but the public reason was: "to make pricing more predictable". Basically, one of the problems was customers who just set the spot price to 10x of the nominal price and leave the bids unattended. This was usually fine, when the price was 0.2x of the nominal price. But sometimes EC2 instance capacity crunches happened, and these high bids actually started competing with each other. As a result, customers could easily get 100 _times_ higher bill than they expected. Another issue was that EC2 spot internally in AWS was implemented as a "bolted on" service that was not on the EC2 instance launch path, so it was easy to game the market. You could do very fun things, like: 1. You need an instance type that is right now under heavy contention. Not to worry! There's a way to get these instances! 2. You create a small VPC with only a couple of available IPs. 3. Then you submit a thousand EC2 Spot bids at 10x the price. 4. Your bids win and EC2 terminates other customers' instances. After all, you're willing to pay more! 5. EC2 then tries to launch these 1000 EC2 instances into the VPC. And fails, because there aren't any IPs available (see item 2). 6. Whoopsie. The bids are cancelled and EC2 instances are returned to the pool. Oh, and you're not charged anything because instances failed to launch. 7. Profit! Now there is plenty of capacity and you can submit bids at a normal price.
      - MichaelZuo 3y ago
        > Basically, one of the problems was customers who just set the spot price to 10x of the nominal price and leave the bids unattended. This was usually fine, when the price was 0.2x of the nominal price. But sometimes EC2 instance capacity crunches happened, and these high bids actually started competing with each other. As a result, customers could easily get 100 _times_ higher bill than they expected. Thanks for sharing, though I'm confused about this part. It seems like expected behaviour? Any possible spot/auction system I can think of would have the inherent possibility of a sudden surge in pricing.
        cyberax 3y ago
        Sure. It was the correct behavior, inherent to any market system. Like Texas electrical spot market: https://theconversation.com/whats-behind-15-000-electricity-bills-in-texas-155822 https://theconversation.com/whats-behind-15-000-electricity-... But it turns out that this is not a customer-friendly behavior. So AWS decided to remove the bidding out of the equation, and instead terminate instances based on a complicated scoring system. The idea is that it's easier to deal with the missing compute capacity, which you notice right away, rather than be blindsided with a 100x bill at the end of the month.
jnsaff2 3y ago
Are the tables in the post generated by ChatGPT? Most of the percentages are just wildly incorrect.
- deleted 3y ago
  [deleted]
- deleted 3y ago
  [deleted]
epberry 3y ago
I know at AWS they drill into folks that spot capacity isn’t guaranteed capacity. I didn’t even think about more demand for spot reducing supply or additionally AWS just simply not investing in as much hardware to accommodate spot because of lack of growth. That said, we're seeing leading indicators that this trend will reverse in as cloud growth returns, https://www.vantage.sh/cloud-cost-report/2023-q1 https://www.vantage.sh/cloud-cost-report/2023-q1
- ericpauley 3y ago
  The q1 on-demand metrics are definitely very interesting! One possible cause of this trend is actually still consistent with spot demand growth in aggregate: when customers deploy spot fleets with on-demand backup capacity and all spot prices get closer to on-demand, those fleets would become more likely to revert to on-demand, rather than a more expensive pool of spot instances. Measuring this would be a great opportunity to leverage the vantage point that your team has!
  - alien_ 3y ago
    Except for Karpenter and EMR, no other AWS services (including Spot fleets) failover between Spot and OnDemand when losing Spot capacity. You simply run Spot from other instance types, and if no Spot capacity is available you run at reduced capacity. Alternative solutions such as AutoSpotting.io are doing it and allow you to run with failover capacity.
- kenhwang 3y ago
  Even on-demand isn't guaranteed capacity. There have been a handful of times in my career where I've tried to spin up a lot more instances and got met with an out of capacity error. There's a special kind of reserved instance for guaranteed capacity.
  - electroly 3y ago
    You have the option to get On-Demand Capacity Reservations now which are separate from RIs. Capacity reservations don't require 1- or 3-year commitments like RIs do. This can be used if you want to get your savings through Regional RIs or Savings Plans instead.
kmeisthax 3y ago
Damn it, there goes my plan of using spot GPU instances to train AI cheaply.
- Havoc 3y ago
  Check out tensordock
- jeremyjh 3y ago
  I was wondering if this could be a contributing factor. There is almost unlimited demand for training models - and a lot of value to derived from doing so - and it seems like a perfect workload for spot.
- Zenst 3y ago
  Seems like such jobs would be suited to a batch submit type service? Does anybody offer that as does seem a viable area to offer if not.
- deleted 3y ago
  [deleted]
- dougSF70 3y ago
  I have just purchased 5% of a data center's Deep Learning servers. 10 GPU / 384 GB of RAM, 2 x Xeon 10-Core processors. I am putting them in a solar powered data center and plan to lease them out at $0.2 per GPU hour.
paulddraper 3y ago
> Spot instances are underpriced, as most tenants underestimate their tolerance for preemption or overestimate its likelihood. > In reality, instance preemption is rare in most instance families I have often seen the following: 1. The demand for spot instances increases for a period of time. 2. Supply is inelastic so prices quickly rise to the full bid amounts. 3. Most users are unwilling to go hours without any spot instances running, so they use very high bids. You either have to tolerate hours of "down time" every few months, or pay exorbitant prices during once in a while. On multiple occasions, I've seen the spot instances priced significantly higher than on-demand. EDIT: Apparently AWS changed the Spot pricing a few years ago though. My anecdotes are 5 years old.
- vasco 3y ago
  > I've seen the spot instances priced significantly higher than on-demand How is the cap price not on-demand cost? Why would you not just swap to on-demand at the turning point?
  - idunno246 3y ago
    there's a cost to engineer something that automatically switches or someone going in and manually changing it. so the spot prices has to be higher than ondemand + switching costs. the new pricing models(a couple years old) though have mostly alleviated this
  - paulddraper 3y ago
    Because you don't want to be pre-empted.
    - alien_ 3y ago
      This is no longer the case. Back in 2018 AWS changed the pricing model so now there's no more bidding. This means "bidding" more won't protect your capacity, AWS will just take it if they need it. Prices are now relatively flat, and changing slightly over time by a fraction of a cent per day with a cap at the OnDemand price, while previously there used to be lots of from 0.1x to 10x the on demand price and with swings even multiple times within a day.
andrewstuart 3y ago
EDIT: if you really need GPU compute power, go buy a GPU from the computer store around the corner - they are cheap, fast and available. I gave up on ec2 when they started requiring you "request for quota" to start a gpu instance. You have to "request for quota" even if you want to run a single instance. You have to specify which specific instance type you want. You have to specify which region you want it in. Then you sit back and wait for some human to "approve your request". In my case this took 24 hours. In this time I literally could have walked (not driven) down to the local computer store, bought a computer, pushed it back to my house in a shopping cart, spent a few hours configuring it and still be left with 16 hours to have a sleep, eat and do some other things. AWS quota system is so far from "scalable" and "elastic" that it's effectively useless. You can't design any sort of infrastructure around that sort of quota system. I dumped AWS at that point. Mind you, Azure has exactly the same quota system. Seriously, just rent a server from Ionos or Hetzner or get a fast Internet connection and self host. It's faster and better and cheaper than any of the big clouds.
- sabalaba 3y ago
  Lambda GPU Cloud is extremely cheap as well (1/4 the price of Amazon) and we have recently added lots of capacity.
- Cpoll 3y ago
  > and still be left with 16 hours to have a sleep, eat and do some other things. Well there's your problem. You don't need to sit back the whole time while you're waiting for approval, it's okay to leave your chair. Jokes aside, I think this probably depends on use-case. You can't easily return the computer. And you can't easily buy (and then return) 100 computers if you have a large one-time computation to run.
- toomuchtodo 3y ago
  We’ve come full circle. Cloud providers have turned into dedicated server providers.
- andrewstuart 3y ago
  Also, I felt that Amazon was playing sketchy little games with spot instances. It's not impossible but quite technically challenging to find out how much you are actually paying for a spot instance. It felt really dodgy to me that I might have started a spot instance thinking I was paying the minimum listed rate but somewhere amongst the formulas and terms and conditions AWS actually decided I would be paying maximum rate. They don't actually tell you up front what your spot instance are costing, no doubt it would not be hard for them to do so, but it seems a deliberate strategy to hide this information. It's just not worth the game playing when self hosting or renting a server from Ionos is cheaper and takes away the uncertainty.
milesward 3y ago
The analysis ignores the basics: AWS controls the supply. At some point there’s a limit to that control via supply chain (meaning they wouldn’t be able to make it cheaper when they want to) but if the number is going up, it’s pretty likely that’s totally intentional.
- deleted 3y ago
  [deleted]
- ericpauley 3y ago
  "Ignore" is a pretty strong word here. This is an analysis of demand. Of course Amazon controls supply, but allowing compute to sit idle at these price points is not profitable so we can assume that they are selling available compute. The question then is whether aggregate instance demand is increasing with respect to available supply and pushing prices up. You'll have to explain what you mean by "intentional" given raising prices above market equilibrium would leave compute idle (again, very unlikely). In that way amazon is just a market actor (albeit a large one) just like customers.
moomoo11 3y ago
I stopped using cloud hosting until _it is actually necessary_ I have two home workstations that are collecting dust. I have 6 and 16 core CPUs with HT hooked up to 32 and 128gb ram. I figured out how to make my dynamic IP still serve requests accounting for IP changes using a background script. If someone has a better solution I’d appreciate that. Both these machines are more than enough to do everything I could dream of building and they host dozens of my little apps and services that I use for myself. I will only use cloud when necessary for commercial product and services.
- kobalsky 3y ago
  >I figured out how to make my dynamic IP still serve requests accounting for IP changes using a background script. If someone has a better solution I’d appreciate that. cloudflare costs $0, scales up to your gigabit connection without blinking, and there are docker containers that keep the IP updated like it if were dynamic dns service.
  - moomoo11 3y ago
    Thx. Is there a specific name for this offering? Sorry first time using cloud flare lol, just digging around.
- tecleandor 3y ago
  - Rent a cheap VPS with an static ip address (for example, the most basic at Vultr) - Set up wireguard or tailscale/headscale (or your preferred VPN) both on the VPS and your workstations, so you can access the services in your workstations from the VPS through the VPN - In the VPS configure Nginx/traefik/caddy (or your preferred reverse proxy) pointing to the services in your workstations Ta-da! Now people access through your VPS, but the processing is done at home. The VPS is cheap, and it doesn't matter if you home IP address changes. You could even take your workstation, move it to any place with internet connection, and it would keep serving pages without changing anything.
  - moomoo11 3y ago
Zetice 3y ago
Yeah but welcome to the era of steeply discounted savings plans. Isn’t it like half of the cost of in demand, or something kind of crazy? Had a couple of calls with their “please don’t leave us” team and they cut our bill more or less in half (more through fixing my boneheaded designs though).
kbumsik 3y ago
I notice the spot cost has grown up for 3 months recently, especially in U.S. regions, while some other regions are not affected at all. Why is it happening to the U.S. regions especially?
arecurrence 3y ago
There's a service that handles obtaining the cheapest spot instances to your specs by a former aws engineer https://github.com/LeanerCloud/AutoSpotting https://github.com/LeanerCloud/AutoSpotting Effectively, it starts up cheap spot instances (based on specified criteria) across a variety of instance types to replace whatever regular instance in an autoscaling group comes online and then spins down the regular instance. EG: That m4a you wanted may be expensive... but nobody is using m4ad so it's 85% off and it meets the specified CPU/RAM requirements... auto spotting will spin it up instead. Having used it on and off over the years it is sometimes eyebrow raising to see 4xl boxes running cheaper than the xl box they replaced :)
- shmoogy 3y ago
  I use spotinst and spot instances have been noticeably more expensive the past two months :(
- mnutt 3y ago
  It works, but it may depend on how resilient your workload is. We have timed spot instances that come online shortly before the top of the hour in the morning eastern, and via specifying instance alternates we'd often get instances that were much larger than our low-ish bids would warrant. Then most of them would get pre-empted <2 minutes later and there would be a bunch of churn as we had to replace them with what we should have gotten originally.
  - alien_ 3y ago
    The author of AutoSpotting here, glad to see it mentioned in here and happy to help clarify such issues. Are you sharing this as an experience of using AutoSpotting or other mechanisms to launch Spot instances? What you described may still happen occasionally and it happened a lot with older versions, but the recent versions of AutoSpotting, especially the commercial edition available on the AWS Marketplace (although the community edition available on GitHub also does it to some degree) should avoid such situations in the default configuration should be much more reliable than before. Actually when it comes to capacity, when Spot capacity is not available and we failover to on-demand, the latest AutoSpotting version will failover between multiple instance types with the same diversification used for Spot instances so you're pretty much guaranteed to have capacity even better than your initial single-instance ASG. We also use the capacity-optimized-prioritized allocation strategy with a custom priority based on lowest cost, but also preferring recent instance types if available for more performance and lower carbon emissions. Let me know if you have further questions about AutoSpotting, Im happy to help.
pachico 3y ago
The article fails to recommend to use re-balancing feature that allows you to run spots in production without worrying too much. It doesn't mention either that price-capacity-optimized allocation strategy. If you want to run spots in production, please read about three two. There is no major drama if you apply them.
- ericpauley 3y ago
  The point of the article is pricing, not preemption. The strategies you’re talking about improve preemption but they won’t save you money if pricing is increasing across the board.
  - pachico 3y ago
    They do as they might be the reasons why you can use spots in the first place

254 points by ericpauley 3y ago

This seems to ignore the fact that the prices are consistently low if you are tracking the lesser used instance types/regions, even when talking explicitly within AWS. Why would that change?

ericpauley 3y ago
It's tough to know for sure. Many of the instance types in these regions are at the price floor, so there could be increases in aggregate demand that just don't show up because the DCs are overbuilt. These markets are thinner so if demand shifts to them (big if because moving regions is hard) we might quickly see price spikes.
cyberax 3y ago
Spot prices don't directly reflect capacity. EC2 Spot used to be a real spot market, with actual real auctions where higher bids displaced lower bids. But it was changed a while ago, so prices are now set algorithmically based on predicted demand/supply. We run stateless calculations on EC2 across regions, and we definitely see that instances are harder to come by. Especially instances with GPUs. And for many instance types, the price advantage of EC2 Spot compared to committed spending is not significant anymore.
- MichaelZuo 3y ago
  Huh, do you remember what the rationale for the change was?
  - derefr 3y ago
    I would guess they were seeing thundering herds of bots trying to fight to grab spot-instance stock whenever any appeared below the bots' set cost threshold. Basically like scalper bots.
  - cyberax 3y ago
    Well, I was one of the engineers that made the change :) I'm not sure how much I can tell, but the public reason was: "to make pricing more predictable". Basically, one of the problems was customers who just set the spot price to 10x of the nominal price and leave the bids unattended. This was usually fine, when the price was 0.2x of the nominal price. But sometimes EC2 instance capacity crunches happened, and these high bids actually started competing with each other. As a result, customers could easily get 100 _times_ higher bill than they expected. Another issue was that EC2 spot internally in AWS was implemented as a "bolted on" service that was not on the EC2 instance launch path, so it was easy to game the market. You could do very fun things, like: 1. You need an instance type that is right now under heavy contention. Not to worry! There's a way to get these instances! 2. You create a small VPC with only a couple of available IPs. 3. Then you submit a thousand EC2 Spot bids at 10x the price. 4. Your bids win and EC2 terminates other customers' instances. After all, you're willing to pay more! 5. EC2 then tries to launch these 1000 EC2 instances into the VPC. And fails, because there aren't any IPs available (see item 2). 6. Whoopsie. The bids are cancelled and EC2 instances are returned to the pool. Oh, and you're not charged anything because instances failed to launch. 7. Profit! Now there is plenty of capacity and you can submit bids at a normal price.
    - MichaelZuo 3y ago
      > Basically, one of the problems was customers who just set the spot price to 10x of the nominal price and leave the bids unattended. This was usually fine, when the price was 0.2x of the nominal price. But sometimes EC2 instance capacity crunches happened, and these high bids actually started competing with each other. As a result, customers could easily get 100 _times_ higher bill than they expected. Thanks for sharing, though I'm confused about this part. It seems like expected behaviour? Any possible spot/auction system I can think of would have the inherent possibility of a sudden surge in pricing.
      - cyberax 3y ago
        Sure. It was the correct behavior, inherent to any market system. Like Texas electrical spot market: https://theconversation.com/whats-behind-15-000-electricity-bills-in-texas-155822 https://theconversation.com/whats-behind-15-000-electricity-... But it turns out that this is not a customer-friendly behavior. So AWS decided to remove the bidding out of the equation, and instead terminate instances based on a complicated scoring system. The idea is that it's easier to deal with the missing compute capacity, which you notice right away, rather than be blindsided with a 100x bill at the end of the month.

jnsaff2 3y ago

Are the tables in the post generated by ChatGPT? Most of the percentages are just wildly incorrect.

deleted 3y ago
[deleted]
deleted 3y ago
[deleted]

epberry 3y ago

I know at AWS they drill into folks that spot capacity isn’t guaranteed capacity. I didn’t even think about more demand for spot reducing supply or additionally AWS just simply not investing in as much hardware to accommodate spot because of lack of growth. That said, we're seeing leading indicators that this trend will reverse in as cloud growth returns, https://www.vantage.sh/cloud-cost-report/2023-q1 https://www.vantage.sh/cloud-cost-report/2023-q1

ericpauley 3y ago
The q1 on-demand metrics are definitely very interesting! One possible cause of this trend is actually still consistent with spot demand growth in aggregate: when customers deploy spot fleets with on-demand backup capacity and all spot prices get closer to on-demand, those fleets would become more likely to revert to on-demand, rather than a more expensive pool of spot instances. Measuring this would be a great opportunity to leverage the vantage point that your team has!
- alien_ 3y ago
  Except for Karpenter and EMR, no other AWS services (including Spot fleets) failover between Spot and OnDemand when losing Spot capacity. You simply run Spot from other instance types, and if no Spot capacity is available you run at reduced capacity. Alternative solutions such as AutoSpotting.io are doing it and allow you to run with failover capacity.
kenhwang 3y ago
Even on-demand isn't guaranteed capacity. There have been a handful of times in my career where I've tried to spin up a lot more instances and got met with an out of capacity error. There's a special kind of reserved instance for guaranteed capacity.
- electroly 3y ago
  You have the option to get On-Demand Capacity Reservations now which are separate from RIs. Capacity reservations don't require 1- or 3-year commitments like RIs do. This can be used if you want to get your savings through Regional RIs or Savings Plans instead.

kmeisthax 3y ago

Damn it, there goes my plan of using spot GPU instances to train AI cheaply.

Havoc 3y ago
Check out tensordock
jeremyjh 3y ago
I was wondering if this could be a contributing factor. There is almost unlimited demand for training models - and a lot of value to derived from doing so - and it seems like a perfect workload for spot.
Zenst 3y ago
Seems like such jobs would be suited to a batch submit type service? Does anybody offer that as does seem a viable area to offer if not.
deleted 3y ago
[deleted]
dougSF70 3y ago
I have just purchased 5% of a data center's Deep Learning servers. 10 GPU / 384 GB of RAM, 2 x Xeon 10-Core processors. I am putting them in a solar powered data center and plan to lease them out at $0.2 per GPU hour.

paulddraper 3y ago

> Spot instances are underpriced, as most tenants underestimate their tolerance for preemption or overestimate its likelihood. > In reality, instance preemption is rare in most instance families I have often seen the following: 1. The demand for spot instances increases for a period of time. 2. Supply is inelastic so prices quickly rise to the full bid amounts. 3. Most users are unwilling to go hours without any spot instances running, so they use very high bids. You either have to tolerate hours of "down time" every few months, or pay exorbitant prices during once in a while. On multiple occasions, I've seen the spot instances priced significantly higher than on-demand. EDIT: Apparently AWS changed the Spot pricing a few years ago though. My anecdotes are 5 years old.

vasco 3y ago
> I've seen the spot instances priced significantly higher than on-demand How is the cap price not on-demand cost? Why would you not just swap to on-demand at the turning point?
- idunno246 3y ago
  there's a cost to engineer something that automatically switches or someone going in and manually changing it. so the spot prices has to be higher than ondemand + switching costs. the new pricing models(a couple years old) though have mostly alleviated this
- paulddraper 3y ago
  Because you don't want to be pre-empted.
  - alien_ 3y ago
    This is no longer the case. Back in 2018 AWS changed the pricing model so now there's no more bidding. This means "bidding" more won't protect your capacity, AWS will just take it if they need it. Prices are now relatively flat, and changing slightly over time by a fraction of a cent per day with a cap at the OnDemand price, while previously there used to be lots of from 0.1x to 10x the on demand price and with swings even multiple times within a day.

andrewstuart 3y ago

EDIT: if you really need GPU compute power, go buy a GPU from the computer store around the corner - they are cheap, fast and available. I gave up on ec2 when they started requiring you "request for quota" to start a gpu instance. You have to "request for quota" even if you want to run a single instance. You have to specify which specific instance type you want. You have to specify which region you want it in. Then you sit back and wait for some human to "approve your request". In my case this took 24 hours. In this time I literally could have walked (not driven) down to the local computer store, bought a computer, pushed it back to my house in a shopping cart, spent a few hours configuring it and still be left with 16 hours to have a sleep, eat and do some other things. AWS quota system is so far from "scalable" and "elastic" that it's effectively useless. You can't design any sort of infrastructure around that sort of quota system. I dumped AWS at that point. Mind you, Azure has exactly the same quota system. Seriously, just rent a server from Ionos or Hetzner or get a fast Internet connection and self host. It's faster and better and cheaper than any of the big clouds.

sabalaba 3y ago
Lambda GPU Cloud is extremely cheap as well (1/4 the price of Amazon) and we have recently added lots of capacity.
Cpoll 3y ago
> and still be left with 16 hours to have a sleep, eat and do some other things. Well there's your problem. You don't need to sit back the whole time while you're waiting for approval, it's okay to leave your chair. Jokes aside, I think this probably depends on use-case. You can't easily return the computer. And you can't easily buy (and then return) 100 computers if you have a large one-time computation to run.
toomuchtodo 3y ago
We’ve come full circle. Cloud providers have turned into dedicated server providers.
andrewstuart 3y ago
Also, I felt that Amazon was playing sketchy little games with spot instances. It's not impossible but quite technically challenging to find out how much you are actually paying for a spot instance. It felt really dodgy to me that I might have started a spot instance thinking I was paying the minimum listed rate but somewhere amongst the formulas and terms and conditions AWS actually decided I would be paying maximum rate. They don't actually tell you up front what your spot instance are costing, no doubt it would not be hard for them to do so, but it seems a deliberate strategy to hide this information. It's just not worth the game playing when self hosting or renting a server from Ionos is cheaper and takes away the uncertainty.

milesward 3y ago

The analysis ignores the basics: AWS controls the supply. At some point there’s a limit to that control via supply chain (meaning they wouldn’t be able to make it cheaper when they want to) but if the number is going up, it’s pretty likely that’s totally intentional.

deleted 3y ago
[deleted]
ericpauley 3y ago
"Ignore" is a pretty strong word here. This is an analysis of demand. Of course Amazon controls supply, but allowing compute to sit idle at these price points is not profitable so we can assume that they are selling available compute. The question then is whether aggregate instance demand is increasing with respect to available supply and pushing prices up. You'll have to explain what you mean by "intentional" given raising prices above market equilibrium would leave compute idle (again, very unlikely). In that way amazon is just a market actor (albeit a large one) just like customers.

moomoo11 3y ago

I stopped using cloud hosting until _it is actually necessary_ I have two home workstations that are collecting dust. I have 6 and 16 core CPUs with HT hooked up to 32 and 128gb ram. I figured out how to make my dynamic IP still serve requests accounting for IP changes using a background script. If someone has a better solution I’d appreciate that. Both these machines are more than enough to do everything I could dream of building and they host dozens of my little apps and services that I use for myself. I will only use cloud when necessary for commercial product and services.

kobalsky 3y ago
>I figured out how to make my dynamic IP still serve requests accounting for IP changes using a background script. If someone has a better solution I’d appreciate that. cloudflare costs $0, scales up to your gigabit connection without blinking, and there are docker containers that keep the IP updated like it if were dynamic dns service.
- moomoo11 3y ago
  Thx. Is there a specific name for this offering? Sorry first time using cloud flare lol, just digging around.
tecleandor 3y ago
- Rent a cheap VPS with an static ip address (for example, the most basic at Vultr) - Set up wireguard or tailscale/headscale (or your preferred VPN) both on the VPS and your workstations, so you can access the services in your workstations from the VPS through the VPN - In the VPS configure Nginx/traefik/caddy (or your preferred reverse proxy) pointing to the services in your workstations Ta-da! Now people access through your VPS, but the processing is done at home. The VPS is cheap, and it doesn't matter if you home IP address changes. You could even take your workstation, move it to any place with internet connection, and it would keep serving pages without changing anything.
- moomoo11 3y ago

Zetice 3y ago

Yeah but welcome to the era of steeply discounted savings plans. Isn’t it like half of the cost of in demand, or something kind of crazy? Had a couple of calls with their “please don’t leave us” team and they cut our bill more or less in half (more through fixing my boneheaded designs though).

kbumsik 3y ago

I notice the spot cost has grown up for 3 months recently, especially in U.S. regions, while some other regions are not affected at all. Why is it happening to the U.S. regions especially?

arecurrence 3y ago

There's a service that handles obtaining the cheapest spot instances to your specs by a former aws engineer https://github.com/LeanerCloud/AutoSpotting https://github.com/LeanerCloud/AutoSpotting Effectively, it starts up cheap spot instances (based on specified criteria) across a variety of instance types to replace whatever regular instance in an autoscaling group comes online and then spins down the regular instance. EG: That m4a you wanted may be expensive... but nobody is using m4ad so it's 85% off and it meets the specified CPU/RAM requirements... auto spotting will spin it up instead. Having used it on and off over the years it is sometimes eyebrow raising to see 4xl boxes running cheaper than the xl box they replaced :)

shmoogy 3y ago
I use spotinst and spot instances have been noticeably more expensive the past two months :(
mnutt 3y ago
It works, but it may depend on how resilient your workload is. We have timed spot instances that come online shortly before the top of the hour in the morning eastern, and via specifying instance alternates we'd often get instances that were much larger than our low-ish bids would warrant. Then most of them would get pre-empted <2 minutes later and there would be a bunch of churn as we had to replace them with what we should have gotten originally.
- alien_ 3y ago
  The author of AutoSpotting here, glad to see it mentioned in here and happy to help clarify such issues. Are you sharing this as an experience of using AutoSpotting or other mechanisms to launch Spot instances? What you described may still happen occasionally and it happened a lot with older versions, but the recent versions of AutoSpotting, especially the commercial edition available on the AWS Marketplace (although the community edition available on GitHub also does it to some degree) should avoid such situations in the default configuration should be much more reliable than before. Actually when it comes to capacity, when Spot capacity is not available and we failover to on-demand, the latest AutoSpotting version will failover between multiple instance types with the same diversification used for Spot instances so you're pretty much guaranteed to have capacity even better than your initial single-instance ASG. We also use the capacity-optimized-prioritized allocation strategy with a custom priority based on lowest cost, but also preferring recent instance types if available for more performance and lower carbon emissions. Let me know if you have further questions about AutoSpotting, Im happy to help.

pachico 3y ago

The article fails to recommend to use re-balancing feature that allows you to run spots in production without worrying too much. It doesn't mention either that price-capacity-optimized allocation strategy. If you want to run spots in production, please read about three two. There is no major drama if you apply them.

ericpauley 3y ago
The point of the article is pricing, not preemption. The strategies you’re talking about improve preemption but they won’t save you money if pricing is increasing across the board.
- pachico 3y ago
  They do as they might be the reasons why you can use spots in the first place