4 ms·

RIs like this are great, but the biggest savings we've found is moving everything we can to Spot Instances. We're hoping Aurora on spot becomes a thing as that'

RIs like this are great, but the biggest savings we've found is moving everything we can to Spot Instances. We're hoping Aurora on spot becomes a thing as that's really our only remaining RI/on-demand cost.

matwood 5y ago
Yeah, spots is where it's at. The problem is to leverage spots, the application in question needs to be 'cloud native'. Many companies moving to the cloud are simply picking up legacy app servers and dropping them on an ec2 instances and declaring success. Those will simply not survive the properties of spots.
mayank 5y ago
> Aurora on spot How would that work for a database? And have you considered or tried Aurora Serverless?
- Thristle 5y ago
  I guess the same way as aurora serverless. In general, RDS uses a seperate storage layer from the actual instances so you can do vertical scale/upgrades with 0 downtime (either read replica goes down or replica becomes master)
- phamilton 5y ago
  We run 3 nodes 24/7 (writer + 2 readers) and have RIs for them. But during daytime hours we autoscale and run an addition 8 or 9 readers. Some of these run for just a few hours and could easily run on spot (especially with minimum duration). Serverless couldn't provide enough capacity for us (at peak we use up to 300 vCPUs in this cluster). That was on v1. v2 might change that when it supports postgres support.
tedmiston 5y ago
Relevant: https://github.com/cloudutil/AutoSpotting https://github.com/cloudutil/AutoSpotting I've seen some third party services that automate migration to / replacement with spot instances, but haven't used them yet personally. Going serverless, in many places, has been the most effective cost optimization for me.
- vasili111 5y ago
  Why serverless is cheap? How Amazon can offer serverless on lower cost than instances? What I mean, Amazon anyway needs to run instances and build on top of them serverless. So were the cost reduction happens with serverless for Amazon?
  - phamilton 5y ago
    The amount of waste we experience can be really high. At Remind we started tracking this using a metric we call OUCH (Overprivisioned Underutilized Cpu Hours). Consider a k8s/ECS installation that autoscales. Each application will have a CPU target. In order to prevent spiky traffic patterns from overwhelming the running containers, we target 70% CPU. As usage goes above 70% CPU, we will launch more containers. The k8s cluster will have a reservation target. In order to allow fast launching of new containers, we want the cluster to have only an 80% occupancy rate. If more than 80% of the cluster is reserved, we launch more instances and expand the cluster. So to run my autoscaling container-based applications in a way that will be reactive and respond to incoming load, I have to leave 44% (1 - .7*.8) of my hardware idle. If we also factor in that AWS itself doesn't target a 100% occupancy rate (because then nobody could launch new instances), each unit of CPU I actually use requires a significant amount of idle infrastructure. Easily double, possibly triple in the larger scheme of things. All of that is either directly paid for (k8s nodes) or indirectly paid for (ec2 pricing inevitably has idle capacity costs baked in). With serverless, we would eliminate many of those inefficiencies. With ultra fast launch times, we don't need to give containers headroom to handle spikes. By not running our compute cluster, we can instantly launch everything we need, eliminating k8s overprovisioning. We're left with AWS idle capacity as the only waste, and AWS mostly has solved that with the spot market. The math doesn't always come out ahead, but there's a lot of opportunity for serverless to be cheaper in many cases.
deleted 5y ago
[deleted]

matwood 5y ago

Yeah, spots is where it's at. The problem is to leverage spots, the application in question needs to be 'cloud native'. Many companies moving to the cloud are simply picking up legacy app servers and dropping them on an ec2 instances and declaring success. Those will simply not survive the properties of spots.

mayank 5y ago

> Aurora on spot How would that work for a database? And have you considered or tried Aurora Serverless?

Thristle 5y ago
I guess the same way as aurora serverless. In general, RDS uses a seperate storage layer from the actual instances so you can do vertical scale/upgrades with 0 downtime (either read replica goes down or replica becomes master)
phamilton 5y ago
We run 3 nodes 24/7 (writer + 2 readers) and have RIs for them. But during daytime hours we autoscale and run an addition 8 or 9 readers. Some of these run for just a few hours and could easily run on spot (especially with minimum duration). Serverless couldn't provide enough capacity for us (at peak we use up to 300 vCPUs in this cluster). That was on v1. v2 might change that when it supports postgres support.

tedmiston 5y ago

Relevant: https://github.com/cloudutil/AutoSpotting https://github.com/cloudutil/AutoSpotting I've seen some third party services that automate migration to / replacement with spot instances, but haven't used them yet personally. Going serverless, in many places, has been the most effective cost optimization for me.

vasili111 5y ago
Why serverless is cheap? How Amazon can offer serverless on lower cost than instances? What I mean, Amazon anyway needs to run instances and build on top of them serverless. So were the cost reduction happens with serverless for Amazon?
- phamilton 5y ago
  The amount of waste we experience can be really high. At Remind we started tracking this using a metric we call OUCH (Overprivisioned Underutilized Cpu Hours). Consider a k8s/ECS installation that autoscales. Each application will have a CPU target. In order to prevent spiky traffic patterns from overwhelming the running containers, we target 70% CPU. As usage goes above 70% CPU, we will launch more containers. The k8s cluster will have a reservation target. In order to allow fast launching of new containers, we want the cluster to have only an 80% occupancy rate. If more than 80% of the cluster is reserved, we launch more instances and expand the cluster. So to run my autoscaling container-based applications in a way that will be reactive and respond to incoming load, I have to leave 44% (1 - .7*.8) of my hardware idle. If we also factor in that AWS itself doesn't target a 100% occupancy rate (because then nobody could launch new instances), each unit of CPU I actually use requires a significant amount of idle infrastructure. Easily double, possibly triple in the larger scheme of things. All of that is either directly paid for (k8s nodes) or indirectly paid for (ec2 pricing inevitably has idle capacity costs baked in). With serverless, we would eliminate many of those inefficiencies. With ultra fast launch times, we don't need to give containers headroom to handle spikes. By not running our compute cluster, we can instantly launch everything we need, eliminating k8s overprovisioning. We're left with AWS idle capacity as the only waste, and AWS mostly has solved that with the spot market. The math doesn't always come out ahead, but there's a lot of opportunity for serverless to be cheaper in many cases.

deleted 5y ago

[deleted]