4 ms·

Ask HN: Heavy AWS users, how to optimize your usage to reduce bills?

11 points by sharmi 9y ago

Do you employ any rule of the thumb or do you use any external services? How much were you able to save? Which one would you recommend?

danieka 9y ago
Some quick tips: If you use OpsWorks you can have time-based instances. We use this in our staging environment so it is only online when we're at work. We used to overestimate how large instances we needed. Our servers usually run just fine on micro-instances. We self host JIRA, Jenkins, our own backoffice and other stuff on a small instance. The memory is not enough out of the box so we use swap space instead which is fine for our own services. Hosting static websites on S3 + CloudFront is really cheap. Hope some of this applies to your situation. EDIT: Guess I'm not a heavy AWS user :) Hope someone replies with more relevant comments.
- sharmi 9y ago
  Thanks. I was not aware of opsworks. Will check it out.
atsaloli 9y ago
If I had to significantly reduce my AWS cost, I'd turn to Corey Quinn of "Last Week in AWS" (https://lastweekinaws.com https://lastweekinaws.com) who fixes AWS bills for a living https://www.linkedin.com/in/coquinn https://www.linkedin.com/in/coquinn
jakozaur 9y ago
I use CloudHealth: http://cloudhealthtech.com/ http://cloudhealthtech.com/ https://www.slideshare.net/JacekMigda/how-to-squeeze-aws-costs https://www.slideshare.net/JacekMigda/how-to-squeeze-aws-cos...
all_usernames 9y ago
It would be useful to know your workload or primary services used (EC2, CloudFront, RDS, S3...) in order to point to advice. My top 3: - Buy reserved EC2/RDS/Elasticache capacity - this can be done even with no upfront payment, but you save more with up-front payment. This will save you roughly 30-60%. - Use spot instances. - For S3, use Reduced Redundancy storage where appropriate.
jozzas 9y ago
My employer's AWS footprint is 5000+ EC2 instances. We use CloudHealth for: - Reserved Instance recommendations & purchasing - typically 12 month partial upfront - CPU, Mem, Disk stats pushed up for EC2 & EBS to identify wastage & generate rightsizing recommendations - Reports generated that go to management of "Optimization Opportunities" in $ for all areas that report to them (this one has the greatest effect in getting things actioned) - Regular reports also go direct to application owners of potential optimisations on EC2, EBS, config, unused ELB, old snapshots, etc. We also have a tool that starts and stops instances for people based on a schedule tag, which saves us a LOT of money. We report on people not using it for dev / test / uat.
eb0la 9y ago
1 - Do not run very large jobs on AWS spot instances only (specifically GPU instances) 2 - If you have to run something large, make sure it can be paused and resumed later. 3 - Run that job on the cheapest Spot/OnDemand instance you need. 4 - If the price is right on the other side of the spot/demand side, provision a new machine, checkpoint the job, and restart.
hijinks 9y ago
Run as much as I can in kubernetes Have 2 ASG's in my kubernetes cluster 1 is for dedicated (or RI) instances that run more of stateful apps. The ones I don't want to move around a lot The other ASG is for spot instances. I use a lambda job to replace a instance spun up with a spot instance. Those will stay up as long as it stays below the bid price. I've saved companies around 50-60% min of their ec2 bill this way.
- j_s 9y ago
  Is there anything available open source that even comes close to the lambda job you're mentioning? It sounds like it does the job of finding the cheapest possible price at runtime, and would probably have to be pretty custom. I think an open source baseline would demonstrate you could do the customization, but with that kind of savings you have the opportunity to do this full time as a consultant, open source or not!

11 points by sharmi 9y ago

Do you employ any rule of the thumb or do you use any external services? How much were you able to save? Which one would you recommend?

danieka 9y ago

Some quick tips: If you use OpsWorks you can have time-based instances. We use this in our staging environment so it is only online when we're at work. We used to overestimate how large instances we needed. Our servers usually run just fine on micro-instances. We self host JIRA, Jenkins, our own backoffice and other stuff on a small instance. The memory is not enough out of the box so we use swap space instead which is fine for our own services. Hosting static websites on S3 + CloudFront is really cheap. Hope some of this applies to your situation. EDIT: Guess I'm not a heavy AWS user :) Hope someone replies with more relevant comments.

sharmi 9y ago
Thanks. I was not aware of opsworks. Will check it out.

atsaloli 9y ago

If I had to significantly reduce my AWS cost, I'd turn to Corey Quinn of "Last Week in AWS" (https://lastweekinaws.com https://lastweekinaws.com) who fixes AWS bills for a living https://www.linkedin.com/in/coquinn https://www.linkedin.com/in/coquinn

jakozaur 9y ago

I use CloudHealth: http://cloudhealthtech.com/ http://cloudhealthtech.com/ https://www.slideshare.net/JacekMigda/how-to-squeeze-aws-costs https://www.slideshare.net/JacekMigda/how-to-squeeze-aws-cos...

all_usernames 9y ago

It would be useful to know your workload or primary services used (EC2, CloudFront, RDS, S3...) in order to point to advice. My top 3: - Buy reserved EC2/RDS/Elasticache capacity - this can be done even with no upfront payment, but you save more with up-front payment. This will save you roughly 30-60%. - Use spot instances. - For S3, use Reduced Redundancy storage where appropriate.

jozzas 9y ago

My employer's AWS footprint is 5000+ EC2 instances. We use CloudHealth for: - Reserved Instance recommendations & purchasing - typically 12 month partial upfront - CPU, Mem, Disk stats pushed up for EC2 & EBS to identify wastage & generate rightsizing recommendations - Reports generated that go to management of "Optimization Opportunities" in $ for all areas that report to them (this one has the greatest effect in getting things actioned) - Regular reports also go direct to application owners of potential optimisations on EC2, EBS, config, unused ELB, old snapshots, etc. We also have a tool that starts and stops instances for people based on a schedule tag, which saves us a LOT of money. We report on people not using it for dev / test / uat.

eb0la 9y ago

1 - Do not run very large jobs on AWS spot instances only (specifically GPU instances) 2 - If you have to run something large, make sure it can be paused and resumed later. 3 - Run that job on the cheapest Spot/OnDemand instance you need. 4 - If the price is right on the other side of the spot/demand side, provision a new machine, checkpoint the job, and restart.

hijinks 9y ago

Run as much as I can in kubernetes Have 2 ASG's in my kubernetes cluster 1 is for dedicated (or RI) instances that run more of stateful apps. The ones I don't want to move around a lot The other ASG is for spot instances. I use a lambda job to replace a instance spun up with a spot instance. Those will stay up as long as it stays below the bid price. I've saved companies around 50-60% min of their ec2 bill this way.

j_s 9y ago
Is there anything available open source that even comes close to the lambda job you're mentioning? It sounds like it does the job of finding the cheapest possible price at runtime, and would probably have to be pretty custom. I think an open source baseline would demonstrate you could do the customization, but with that kind of savings you have the opportunity to do this full time as a consultant, open source or not!