6 ms·

Just the other day I was looking for an official docker image that includes the AWS CLI. On top of that, and mainly, I was looking to find more documentation or

Just the other day I was looking for an official docker image that includes the AWS CLI. On top of that, and mainly, I was looking to find more documentation or tooling to better automate the deployment of new AWS projects. Does anyone here have any experience of (starting from scratch or with no AWS resources) setting up policies/users/resources/configurations via something similar to the Deployment Managers of GCP and Azure?.. preferably something declarative or via templates? Bash-my-AWS looks like a great step towards the goal I have in mind but I may also just be unaware of other tooling or AWS capabilities.

zbruhnke 7y ago
Gruntwork has a lot of Open Source tooling around AWS and their new guides are pretty great for some of what you're mentioning https://gruntwork.io/guides/ https://gruntwork.io/guides/ I am in no way affiliated with them other than being a customer
weberc2 7y ago
If I’m understanding you correctly, I think you want CloudFormation?
- TheSpiciestDev 7y ago
  Thanks, AWS CloudFormation looks like what I've experienced with other cloud service providers.
  - stevekemp 7y ago
    The other alternative is terraform: https://www.terraform.io/ https://www.terraform.io/
  - mypalmike 7y ago
    There is also AWS Cloud Development Kit, which generates CloudFormation from Typescript, C#, Java, or Python. https://aws.amazon.com/blogs/developer/getting-started-with-the-aws-cloud-development-kit-and-python/ https://aws.amazon.com/blogs/developer/getting-started-with-...
- TheSpiciestDev 7y ago
  Thank you again for this direction! I just finished a POC that generates 90% of the AWS services I use per client/project/application. The remaining 10% is DNS stuff that I can easily do by hand, but with a few clicks I get everything provisioned with much less human error (buckets, Lambdas, API Gateways, Cloudfront distributions, etc.) The formation definition is ~1000 lines of JSON, but it explicitly describes everything I need and it takes in parameters - it's wonderful! Thank you again!
natekupp 7y ago
We use Pulumi to manage both our GCP and AWS resources, and we really like it. You might consider using Terraform directly if you want something more mature.
cosaquee 7y ago
At work we are using Terraform to manage everything that is related to AWS resources, including accounts, IAM policies and groups. We also used Serverless framework and CloudFormation, but Terraform is what works for us and I can recommend it as a main IaaS tool
avip 7y ago
Like it or not (I do...) terraform is the de-facto industry standard, and pretty much the only mature cloud resources management tool I'm aware of. It is unwise IMHO to use CloudFormation currently unless you're provisioning resources so obscure they didn't yet make it to tf aws provider. BTW your Dockerfile pretty much boils down to: FROM alpine:3.10 RUN apk add --no-cache \ python3 RUN pip3 install awscli COPY config /root/.aws/ COPY credentials /root/.aws/
- yjftsjthsd-h 7y ago
  > unless you're provisioning resources so obscure they didn't yet make it to tf aws provider. Isn't there precedent for terraform getting support for things before cloudformation?
  - LilBytes 7y ago
    I'd say it's more and more common that CF doesn't support X resource or pattern than anything else. We've got custom resources _everywhere_ instead and only just started on our journey of using TF instead. CDK is trying to drive up adoption though I've not used it yet so can't provide any opinions.
- crtlaltdel 7y ago
  if you don’t want to copy your credentials into the container you can supply them via env vars when you docker run commands in the container
- bob33 7y ago
  ansible and serverless are also very powerful IAC tools that let you deploy on top of CloudFormation but give you a much nicer way to do so. Terraform does require state which is a pain point of it for some. Ansible let's you just run their scripts and you don't have to worry about state in S3 or Dynamo DB.
- Galanwe 7y ago
  Have a look at CDK. It's a framework on top of CF to use python/javascript/etc made by AWS. I've been trying it out recently to try to move away from TF and it's a promising alternative.
parfamz 7y ago
I would strongly recommend to use cloud formation through a typed proxy like troposphere. Also would not recommend to use terraform at all since you will run into warts and fundamental issues quickly. I have done projects with both and my current blessed workflow is a custom python driver which uses CF via troposphere and minimal boto3 as glue. Also I work at AWS.
- ak217 7y ago
  Several of the warts in Terraform were fixed in 0.12. While I think the HCL DSL was a mistake and prefer the CloudFormation YAML, CloudFormation has its share of warts as well, and the TF community has been doing better than CF in staying up-to-date with the AWS API updates - which reflects quite poorly on AWS actually. > would not recommend to use terraform at all since you will run into warts and fundamental issues It's not a good look to be employed by the 800 pound gorilla and bash your company's competitor without mentioning specifics.
  - LilBytes 7y ago
    0.12 fixed and introduced warts. It's a buggy mess, but it is at least has better coverage than CF.
- arman_ashrafian 7y ago
  Have you tried the CDK yet instead of using Troposphere?

zbruhnke 7y ago

Gruntwork has a lot of Open Source tooling around AWS and their new guides are pretty great for some of what you're mentioning https://gruntwork.io/guides/ https://gruntwork.io/guides/ I am in no way affiliated with them other than being a customer

weberc2 7y ago

If I’m understanding you correctly, I think you want CloudFormation?

TheSpiciestDev 7y ago
Thanks, AWS CloudFormation looks like what I've experienced with other cloud service providers.
- stevekemp 7y ago
  The other alternative is terraform: https://www.terraform.io/ https://www.terraform.io/
- mypalmike 7y ago
  There is also AWS Cloud Development Kit, which generates CloudFormation from Typescript, C#, Java, or Python. https://aws.amazon.com/blogs/developer/getting-started-with-the-aws-cloud-development-kit-and-python/ https://aws.amazon.com/blogs/developer/getting-started-with-...
TheSpiciestDev 7y ago
Thank you again for this direction! I just finished a POC that generates 90% of the AWS services I use per client/project/application. The remaining 10% is DNS stuff that I can easily do by hand, but with a few clicks I get everything provisioned with much less human error (buckets, Lambdas, API Gateways, Cloudfront distributions, etc.) The formation definition is ~1000 lines of JSON, but it explicitly describes everything I need and it takes in parameters - it's wonderful! Thank you again!

natekupp 7y ago

We use Pulumi to manage both our GCP and AWS resources, and we really like it. You might consider using Terraform directly if you want something more mature.

cosaquee 7y ago

At work we are using Terraform to manage everything that is related to AWS resources, including accounts, IAM policies and groups. We also used Serverless framework and CloudFormation, but Terraform is what works for us and I can recommend it as a main IaaS tool

avip 7y ago

Like it or not (I do...) terraform is the de-facto industry standard, and pretty much the only mature cloud resources management tool I'm aware of. It is unwise IMHO to use CloudFormation currently unless you're provisioning resources so obscure they didn't yet make it to tf aws provider. BTW your Dockerfile pretty much boils down to: FROM alpine:3.10 RUN apk add --no-cache \ python3 RUN pip3 install awscli COPY config /root/.aws/ COPY credentials /root/.aws/

yjftsjthsd-h 7y ago
> unless you're provisioning resources so obscure they didn't yet make it to tf aws provider. Isn't there precedent for terraform getting support for things before cloudformation?
- LilBytes 7y ago
  I'd say it's more and more common that CF doesn't support X resource or pattern than anything else. We've got custom resources _everywhere_ instead and only just started on our journey of using TF instead. CDK is trying to drive up adoption though I've not used it yet so can't provide any opinions.
crtlaltdel 7y ago
if you don’t want to copy your credentials into the container you can supply them via env vars when you docker run commands in the container
bob33 7y ago
ansible and serverless are also very powerful IAC tools that let you deploy on top of CloudFormation but give you a much nicer way to do so. Terraform does require state which is a pain point of it for some. Ansible let's you just run their scripts and you don't have to worry about state in S3 or Dynamo DB.
Galanwe 7y ago
Have a look at CDK. It's a framework on top of CF to use python/javascript/etc made by AWS. I've been trying it out recently to try to move away from TF and it's a promising alternative.

parfamz 7y ago

I would strongly recommend to use cloud formation through a typed proxy like troposphere. Also would not recommend to use terraform at all since you will run into warts and fundamental issues quickly. I have done projects with both and my current blessed workflow is a custom python driver which uses CF via troposphere and minimal boto3 as glue. Also I work at AWS.

ak217 7y ago
Several of the warts in Terraform were fixed in 0.12. While I think the HCL DSL was a mistake and prefer the CloudFormation YAML, CloudFormation has its share of warts as well, and the TF community has been doing better than CF in staying up-to-date with the AWS API updates - which reflects quite poorly on AWS actually. > would not recommend to use terraform at all since you will run into warts and fundamental issues It's not a good look to be employed by the 800 pound gorilla and bash your company's competitor without mentioning specifics.
- LilBytes 7y ago
  0.12 fixed and introduced warts. It's a buggy mess, but it is at least has better coverage than CF.
arman_ashrafian 7y ago
Have you tried the CDK yet instead of using Troposphere?