2 ms·
Ask HN: How do you (or do you) optimize your AWS development environment costs?
I'm interested in what tools, solutions or services people use to optimize the 24/7 running development and test resources you have.
Have you found the effort of setting up and maintaining existing solutions, such as AWS Instance Scheduler or Cloud Custodian, worth the savings you generate with them?
Have you found or built yourself solutions that your development staff is happy with? Can they for example easily adjust the default schedules when needed? Can the solutions be further integrated with e.g your CI/CD pipelines, so that environments can be brought up/down on demand?
Full disclosure: I am thinking about building something for this space.
- gregjor 3y agoMonitor the resources, look for over-provisioning. I see a lot of that. Use reserved instances rather than spot pricing. Multiple tools/services in the space you describe.
- ja-mu 3y agoDo you see any scheduling around off hours? The tools I was referring to specifically have this capability to schedule downtime for your resources outside of the hours when they are normally used.
- oriettaxx 3y agomany have this in mind, that at the end do not do much: you can automatically scale down EC2 instances at night, but then spider comes and hit all your alarms (and you want to avoid night alarms).
- gregjor 3y agoThat hasn't come up with my customers. They run web applications they need live 24/7. We do see traffic patterns but no times with zero activity. I have seen over-provisioning with almost every customer I take on. Once we look at actual usage we can scale back. Often they have things like multiple web servers and load balancers they don't need, excessive database replication and backup retention. I've seen fully redundant setups in secondary regions for failover, but no failover plan or testing. That can all add up.
- ja-mu 3y agoHow do you customers develop these 24/7 web applications they have? Are these development resources needed 24/7?
- gregjor 3y agoI'm not sure what you're asking. Sometimes development happens on developer computers, staged for review to instances/databases spun up for that purpose. Sometimes the developers have a parallel but scaled-down copy of the production environment. The web applications I work on, with one or two exceptions in the past, don't have complex dev requirements. Often deployment amounts to "git pull." If you work with complex dev environments and CI/CD pipelines that cost enough to care about, maybe look at how to simplify that. How development and deployment happens varies tremendously, so I don't have any general advice beyond "simplify" and examine the processes and resources your developers use and say they need. Every process and resource should have cost, benefit, and risk assigned. Left to themselves programmers too often add complexity and process for its own sake, or out of unexamined adherence to "best practices."
- ja-mu 3y agoSorry for being vague. It's these a bit larger and more complex dev environments and pipelines that I had in mind. I've worked with many companies where these are the norm - and very rarely see anything done about the costs. Largely this seems to be because existing tooling doesn't really support more complex dev team use cases.
- oriettaxx 3y agoI think that anything that helps reducing aws bill is more then welcome: AWS is getting tons of $$$ that company can easily save: everybody in the IT (management included) knows and clearly see it from the huge monthly bills they get compared to some years before, but how to spot it? There are some AWS tools to do it, but they cost too, and of course cannot spot many ways to save money, but there are. On of my customer told me: we can easily pay you a full month of the saving we get after your cost optimization work (it ended up a lot of $$ :), but the hard part is that to be able to help them save money, you kind of need full root access, and you wont be able to easily get it unless the company is small (or not structured).
- oriettaxx 3y agoas to how save money: just check every item in the bill, once you optimize assure cost do not increase monthly. Try to investigate traffic, too: not many knows that, if you are using a VPC (and everybody should, AWS correctly states) you pay the traffic from your instances to S3, and this can be quite a lot! you can zeros that cost by creating a (free of costs) S3 gateway. I think AWS has several internal tools that tell them the amount of $$ customers are just wasting: then they LOL every end of the month ;)