8 ms·

Just published a tiny CLI utility to save some bucks on AWS development environment while I'm not really working. https://github.com/aramalipoor/aws-cost-saver

Just published a tiny CLI utility to save some bucks on AWS development environment while I'm not really working. https://github.com/aramalipoor/aws-cost-saver https://github.com/aramalipoor/aws-cost-saver The idea is to stop/shutdown any resources (such as EC2 instances, RDS databases, ECS tasks, etc.) when you're done for the day and restore them back up next time you continue. Currently it's in a very early stage, a hobby side-project :) Ideally I'm thinking to provide a CloudFormation template or terraform config for example, to provision a scheduled Lambda with predefined "start of work" and "end of work" times and have it automatically do that for you. I'd like to know if something like this would be useful for you too and what do you think about it?

adamlklein 6y ago
This sounds like a good idea! I had been looking at creating a lamba to check for the existence of pods, services, alternate namespaces etc., as I learn K8s on AWS, to shut down the cluster when I'm not working on it.
- aramalipoor 6y ago
  Thanks. Shutting down K8S cluster when not using is a really good idea. Maybe one day I play around with AWS's k8s cluster and add the trick to aws-cost-saver. Pull requests are also welcome if you have time :) the logic must be similar to EC2 or RDS tricks: https://github.com/aramalipoor/aws-cost-saver/blob/master/src/tricks/stop-rds-database-instances.trick.ts https://github.com/aramalipoor/aws-cost-saver/blob/master/sr...
WrtCdEvrydy 6y ago
This is amazing.... Edit: I wonder if you could put this inside of a lambda with just a role.
- aramalipoor 6y ago
  Thanks. Agree with you, having it in a Lambda with a scheduled trigger or even manual trigger will be really useful.
  - banana_giraffe 6y ago
    Or an API gateway. I have something similar (though very bespoke for me) that has a simple API Gateway web page as it's front-end. It also opens up a firewall entry for my IP so I can connect to the machine. Depending on my mood I either use the web page or some simple curl aliases to start and stop an instance.
- redacted3212 6y ago
  CloudCustodian has offered this for quite some time. We use it at our company in all our accounts.
swyx 6y ago
interesting work! i feel like this can't be a new problem. what comparable tools exist for doing this today? its basically just spinning up a staging env while you work and spinning down when you knock off work right? almost like a heroku dyno spinning up for you.
- aliswe 6y ago
  I would regard it as a fairly new problem actually, the same age as the "developer resources on a per-minute basis" phenomenon.
z3t4 6y ago
Im very interested in learning about your work flows, like how does a typical day look like? I do all my work on a local pc and only use servers for production. So im interesting in how/why you use the cloud stack
- jen20 6y ago
  How do you test or develop against cloud resources like ECS using a local PC?
  - ununoctium87 6y ago
    https://github.com/localstack/localstack https://github.com/localstack/localstack Though I find a stronger argument in using containerisation to ensure an identical application run-time environment no matter the infra it’s running on. If we are talking about infra engineering, then sure. That’s where I find localstack helps
    - hyeomans 6y ago
      What happens when you need 40 micro-services running to render your frontend?
      - smabie 6y ago
        You ditch all the microservices and develop the app like a normal human being.
        pests 6y ago
        That's an answer to a question he didn't ask.
  - krono 6y ago
    Ok, I don't know how else to ask and I'm seriously not trying to be an arse but how do you? I'm genuinely interested as it's something I've been struggling with myself lately.
vageli 6y ago
Are you aware of cloud custodian out of Capital One? [0] Does this and much, much more (like delete noncompliant security groups, etc). [0]: https://cloudcustodian.io/ https://cloudcustodian.io/
- 1vuio0pswjnm7 6y ago
  I'm aware that Capital One was found guilty of not properly securing customer information when they started moving it to "the cloud" (AWS). I'm also aware that the person who orchestrated the data breach, by exploiting the bank's AWS (mis)configuration, had been hired by Amazon as a developer. Maybe the bank's IT staff created "cloud custodian" after the incident. https://www.marketwatch.com/story/capital-one-fined-80-million-for-breach-that-accessed-data-of-106-million-card-holders-2020-08-06 https://www.marketwatch.com/story/capital-one-fined-80-milli...
  - destory-everyth 6y ago
    Custodian was already there , they have a strict pipeline but what I hear is that the environment where the hack took place was bypassing the normal organisational pipe governance and did not even have coverage of the custodian on the account. Basically shadow it. I still put a lot of blame on the way AWS Iam makes it incredibly hard to stop the use of the credentials outside the vpc in the event they are stolen , for example source ip restrictions do not work if the bucked it using kms encryption because kms will decrypt on users behalf appearing to come from a different ip than the user . The called via is a farce that only work with about 4 services out of hundreds and the meta data v2 with its ip level TTL is a marketing gimmick
- aramalipoor 6y ago
  Thanks for sharing this. Looks like a sophisticated solution, although might be too complex and enterprise-y for what I needed as a sleepy developer, but is definitely worth having a look :)

adamlklein 6y ago

This sounds like a good idea! I had been looking at creating a lamba to check for the existence of pods, services, alternate namespaces etc., as I learn K8s on AWS, to shut down the cluster when I'm not working on it.

aramalipoor 6y ago
Thanks. Shutting down K8S cluster when not using is a really good idea. Maybe one day I play around with AWS's k8s cluster and add the trick to aws-cost-saver. Pull requests are also welcome if you have time :) the logic must be similar to EC2 or RDS tricks: https://github.com/aramalipoor/aws-cost-saver/blob/master/src/tricks/stop-rds-database-instances.trick.ts https://github.com/aramalipoor/aws-cost-saver/blob/master/sr...

WrtCdEvrydy 6y ago

This is amazing.... Edit: I wonder if you could put this inside of a lambda with just a role.

aramalipoor 6y ago
Thanks. Agree with you, having it in a Lambda with a scheduled trigger or even manual trigger will be really useful.
- banana_giraffe 6y ago
  Or an API gateway. I have something similar (though very bespoke for me) that has a simple API Gateway web page as it's front-end. It also opens up a firewall entry for my IP so I can connect to the machine. Depending on my mood I either use the web page or some simple curl aliases to start and stop an instance.
redacted3212 6y ago
CloudCustodian has offered this for quite some time. We use it at our company in all our accounts.

swyx 6y ago

interesting work! i feel like this can't be a new problem. what comparable tools exist for doing this today? its basically just spinning up a staging env while you work and spinning down when you knock off work right? almost like a heroku dyno spinning up for you.

aliswe 6y ago
I would regard it as a fairly new problem actually, the same age as the "developer resources on a per-minute basis" phenomenon.

z3t4 6y ago

Im very interested in learning about your work flows, like how does a typical day look like? I do all my work on a local pc and only use servers for production. So im interesting in how/why you use the cloud stack

jen20 6y ago
How do you test or develop against cloud resources like ECS using a local PC?
- ununoctium87 6y ago
  https://github.com/localstack/localstack https://github.com/localstack/localstack Though I find a stronger argument in using containerisation to ensure an identical application run-time environment no matter the infra it’s running on. If we are talking about infra engineering, then sure. That’s where I find localstack helps
  - hyeomans 6y ago
    What happens when you need 40 micro-services running to render your frontend?
    - smabie 6y ago
      You ditch all the microservices and develop the app like a normal human being.
      - pests 6y ago
        That's an answer to a question he didn't ask.
- krono 6y ago
  Ok, I don't know how else to ask and I'm seriously not trying to be an arse but how do you? I'm genuinely interested as it's something I've been struggling with myself lately.

vageli 6y ago

Are you aware of cloud custodian out of Capital One? [0] Does this and much, much more (like delete noncompliant security groups, etc). [0]: https://cloudcustodian.io/ https://cloudcustodian.io/

1vuio0pswjnm7 6y ago
I'm aware that Capital One was found guilty of not properly securing customer information when they started moving it to "the cloud" (AWS). I'm also aware that the person who orchestrated the data breach, by exploiting the bank's AWS (mis)configuration, had been hired by Amazon as a developer. Maybe the bank's IT staff created "cloud custodian" after the incident. https://www.marketwatch.com/story/capital-one-fined-80-million-for-breach-that-accessed-data-of-106-million-card-holders-2020-08-06 https://www.marketwatch.com/story/capital-one-fined-80-milli...
- destory-everyth 6y ago
  Custodian was already there , they have a strict pipeline but what I hear is that the environment where the hack took place was bypassing the normal organisational pipe governance and did not even have coverage of the custodian on the account. Basically shadow it. I still put a lot of blame on the way AWS Iam makes it incredibly hard to stop the use of the credentials outside the vpc in the event they are stolen , for example source ip restrictions do not work if the bucked it using kms encryption because kms will decrypt on users behalf appearing to come from a different ip than the user . The called via is a farce that only work with about 4 services out of hundreds and the meta data v2 with its ip level TTL is a marketing gimmick
aramalipoor 6y ago
Thanks for sharing this. Looks like a sophisticated solution, although might be too complex and enterprise-y for what I needed as a sleepy developer, but is definitely worth having a look :)