9 ms·

I like CloudFormation. Unfortunately it is very unwieldy to write CloudFormation templates directly, and we're not about to start using the AWS CFN GUI editor!

I like CloudFormation. Unfortunately it is very unwieldy to write CloudFormation templates directly, and we're not about to start using the AWS CFN GUI editor! It seems like the assembly of the AWS ecosystem. Does anyone else have a favourite hammer for this particular nail? I'd love to have something better than our home-baked solution, but I'm yet to find anything which doesn't introduce other flaws, such as an incomplete implementation (missing parameters or resource types) or ultimately making a leaky abstraction on top of CloudFormation somehow. I ended up brewing a reasonably straightforward solution using Python as a (minimal) DSL which emits JSON. Its primary purpose is to support the whole of the CFN ecosystem (not just implement some small part of EC2, for instance) while also not trying to be too clever. It has about 50-100 lines of python which implements helper functions such as ref(), join() and load_user_data(), and not many other things. There is an almost 1-to-1 correspondence between the generated CFN configuration and the python source. As a bonus it checks for a few common mistakes like broken refs or parameters which aren't used. I have heard that similar solutions have been reinvented in a few places, including the BBC. But I'm yet to see a good public solution!

otterley 11y ago
Some people like SparkleFormation (http://www.sparkleformation.io/ http://www.sparkleformation.io/). I'll warn you, though, that it's not a good example of how to program in Ruby. It abuses method_missing to the point that it makes your implementations difficult to debug.
- nikolay 11y ago
  I also don't like their made-up terms such as "dynamics", etc. The documentation is pretty confusing as well.
andystanton 11y ago
Troposphere (https://github.com/cloudtools/troposphere https://github.com/cloudtools/troposphere) is a mature Python CloudFormation solution that sounds similar to your home brewed one.
- wingsonfire 11y ago
  Yes.. It is mature and very active too. AWS keeps on adding services and also make them available on CF. Troposphere community is very quick in implementing them..
andystanton 11y ago
If you'd consider something other than CloudFormation, there is also Hashicorp's Terraform. It has an AWS provider (https://terraform.io/docs/providers/aws/index.html https://terraform.io/docs/providers/aws/index.html) which creates resources and maintains the state in a file that you can store in version control (https://terraform.io/docs/state/index.html https://terraform.io/docs/state/index.html).
- eropple 11y ago
  Terraform, as an idea, is brilliant. Mitchell and company isolated a hugely important need and tried to fill it, and I give them all the credit in the world for that. Cross-platform cloud provisioning? Gimme. But I cannot in good conscience not relate what a disastrous experience Terraform has been for me at both jobs and clients. Writing reusable code in Terraform is an exercise in frustration due to the extreme clumsiness of HCL (which, I understand, was used because "YAML is complicated"--well, that's true, but YAML isn't a good solution either, you're HashiCorp, you wrote Vagrant, you already know how to do this!). The application architecture is reckless and full of race conditions; your state will be hosed if one resource errors out at the wrong time, while other resources are being successfully updated--the resources that return successfully after the failed resource will on many occasions fail to be persisted to state. What's more, application testing seems to be at best an afterthought: there have been regressions in the providers that will break your existing states. I would under no circumstances use Terraform if I didn't have clients who had selected it before I was working with them. If in AWS, I would use CloudFormation, with a tool like Cfer[1] (which is excellent, reliable code) or SparkleFramework[2] (which is more full-featured but I hope you never need to debug it) to provision my stuff. (Full disclosure: I'm building a much, much better provisioner for multi-provider cloud infrastructure. Neither of the projects I recommend are mine; mine's not done yet.) [1] - https://github.com/seanedwards/cfer https://github.com/seanedwards/cfer [2] - http://www.sparkleformation.io/ http://www.sparkleformation.io/
  - duggan 11y ago
    > Full disclosure: I'm building a much, much better provisioner for multi-provider cloud infrastructure. Neither of the projects I recommend are mine; mine's not done yet. One of the convenient things about software that doesn't exist is that it doesn't have any bugs. Let your software speak for itself when it exists; until then, this seems an undeserved critique of software, and a team, that is solving problems every day.
swindmill 11y ago
https://github.com/cloudtools/troposphere https://github.com/cloudtools/troposphere is a good option here Also https://github.com/russellballestrini/botoform https://github.com/russellballestrini/botoform looks to be a newer solution in this space Terraform is another option and then there's the model we're actively moving towards at work: using Ansible to abstract and completely replace calls to CloudFormation with a combination of existing and bespoke modules to dynamically spin up the infrastructure we need.
hibikir 11y ago
A big problem of many tools out there, and of cloud formation in general, is that validation is a mess. And the bigger the template, the bigger the problem, even with existing tools. There are validation problems even within specific tools: Just look at the RDS setup alone: A ton of options that are often mutually exclusive. It's brutal. And don't get me started with security groups. At Monsanto, we built our own toolset, and open sourced it. It is all Scala, so it might be a bit of a learning curve for many folks, but there's an actual attempt in there at making sure that if you can write it, have the tool blow up before it gets to AWS, which then realizes something went wrong, and that it has to roll everything back. https://github.com/MonsantoCo/cloudformation-template-generator https://github.com/MonsantoCo/cloudformation-template-genera...
lukeschlather 11y ago
Using a DSL is tempting. I've found the AWS CLI best, and a lot of the time I think it's easier just to write a Ruby script using the SDK. This obviously doesn't necessarily handle teardown very well, and it tends to be copying boilerplate and modifying it, but I find it the most straightforward thing, and simple, if a little verbose.
empath75 11y ago
There's already a python module called troposphere.
nzoschke 11y ago
I am asking this question often. We are using very advanced CloudFormation in the open source Convox platform. https://github.com/convox/rack https://github.com/convox/rack I have touched every corner of CF including lots of Custom Resources. Right now we are using the golang template tools and tests to generate our templates. But I have lots of needs and ideas for improving this. A CF template compiler and simulator should be possible, giving us all tons of confidence in making template changes and therefore any infrastructure update. I have some sketches that I haven't published yet. And I strongly believe CF is the best tool in this space if you're all in on AWS. Let Amazon be responsible for operating a transactional infrastructure mutation service. It's ridiculously hard to do this right. If you want to brainstorm some ideas send me a message :)
- wise_young_man 11y ago
  One of the things that I thought would be neat is an open source CloudFormation that could work for multiple cloud vendors, possibly using a driver pattern. Also, you might want to update your HN profile with contact info.
  - nzoschke 11y ago
    Thanks, I updated my profile with contact info. Terraform is awesome if you want an OSS project to manage multiple cloud vendors. But I think that infrastructure change management is a really hard problem and the state of the art solution is how AWS runs CloudFormation as a managed service. Once it's set up properly, it's amazing watching what CloudFormation can do. It can execute updating 20 instances to roll out a new AMI, and then roll the whole operation back on demand or if a failure happens. All with no application downtime in the cluster!
vankap 11y ago
Create the infrastructure manually and then use Cloudformer tool to generate the template based on the already created infrastructure. You can then edit the generated template to make it more maintainable and you have a nice reusable template.
obulpathi 11y ago
CloudFormation is not very elegant. Things become complicated with the concept region, zonal resources (same AMI is represented by a different id in different region etc). Try Google Clouds's Deployment Manager. Functionally similar, but Google Cloud Deployment Manager is far easier (everything is a global resource), Jinja based templates (you get to write for's and if's, evaluate lists, dictionaries inside templates .. )
Corrado 11y ago
I built a JSON templating system[0] (based on Handlebars) that is project based. Basically, you create new JSON files for each project and the system generates a CFN template that contains everything that project needs (EC2 instances, RDS instances, security groups, etc.) It's still a work in progress but I'm using it in production in my day job and I'm pretty happy with how it works. More help is always accepted. :) [0] https://github.com/rnhurt/CFNBuilder https://github.com/rnhurt/CFNBuilder

otterley 11y ago

Some people like SparkleFormation (http://www.sparkleformation.io/ http://www.sparkleformation.io/). I'll warn you, though, that it's not a good example of how to program in Ruby. It abuses method_missing to the point that it makes your implementations difficult to debug.

nikolay 11y ago
I also don't like their made-up terms such as "dynamics", etc. The documentation is pretty confusing as well.

andystanton 11y ago

Troposphere (https://github.com/cloudtools/troposphere https://github.com/cloudtools/troposphere) is a mature Python CloudFormation solution that sounds similar to your home brewed one.

wingsonfire 11y ago
Yes.. It is mature and very active too. AWS keeps on adding services and also make them available on CF. Troposphere community is very quick in implementing them..

andystanton 11y ago

If you'd consider something other than CloudFormation, there is also Hashicorp's Terraform. It has an AWS provider (https://terraform.io/docs/providers/aws/index.html https://terraform.io/docs/providers/aws/index.html) which creates resources and maintains the state in a file that you can store in version control (https://terraform.io/docs/state/index.html https://terraform.io/docs/state/index.html).

eropple 11y ago
Terraform, as an idea, is brilliant. Mitchell and company isolated a hugely important need and tried to fill it, and I give them all the credit in the world for that. Cross-platform cloud provisioning? Gimme. But I cannot in good conscience not relate what a disastrous experience Terraform has been for me at both jobs and clients. Writing reusable code in Terraform is an exercise in frustration due to the extreme clumsiness of HCL (which, I understand, was used because "YAML is complicated"--well, that's true, but YAML isn't a good solution either, you're HashiCorp, you wrote Vagrant, you already know how to do this!). The application architecture is reckless and full of race conditions; your state will be hosed if one resource errors out at the wrong time, while other resources are being successfully updated--the resources that return successfully after the failed resource will on many occasions fail to be persisted to state. What's more, application testing seems to be at best an afterthought: there have been regressions in the providers that will break your existing states. I would under no circumstances use Terraform if I didn't have clients who had selected it before I was working with them. If in AWS, I would use CloudFormation, with a tool like Cfer[1] (which is excellent, reliable code) or SparkleFramework[2] (which is more full-featured but I hope you never need to debug it) to provision my stuff. (Full disclosure: I'm building a much, much better provisioner for multi-provider cloud infrastructure. Neither of the projects I recommend are mine; mine's not done yet.) [1] - https://github.com/seanedwards/cfer https://github.com/seanedwards/cfer [2] - http://www.sparkleformation.io/ http://www.sparkleformation.io/
- duggan 11y ago
  > Full disclosure: I'm building a much, much better provisioner for multi-provider cloud infrastructure. Neither of the projects I recommend are mine; mine's not done yet. One of the convenient things about software that doesn't exist is that it doesn't have any bugs. Let your software speak for itself when it exists; until then, this seems an undeserved critique of software, and a team, that is solving problems every day.

swindmill 11y ago

https://github.com/cloudtools/troposphere https://github.com/cloudtools/troposphere is a good option here Also https://github.com/russellballestrini/botoform https://github.com/russellballestrini/botoform looks to be a newer solution in this space Terraform is another option and then there's the model we're actively moving towards at work: using Ansible to abstract and completely replace calls to CloudFormation with a combination of existing and bespoke modules to dynamically spin up the infrastructure we need.

hibikir 11y ago

A big problem of many tools out there, and of cloud formation in general, is that validation is a mess. And the bigger the template, the bigger the problem, even with existing tools. There are validation problems even within specific tools: Just look at the RDS setup alone: A ton of options that are often mutually exclusive. It's brutal. And don't get me started with security groups. At Monsanto, we built our own toolset, and open sourced it. It is all Scala, so it might be a bit of a learning curve for many folks, but there's an actual attempt in there at making sure that if you can write it, have the tool blow up before it gets to AWS, which then realizes something went wrong, and that it has to roll everything back. https://github.com/MonsantoCo/cloudformation-template-generator https://github.com/MonsantoCo/cloudformation-template-genera...

lukeschlather 11y ago

Using a DSL is tempting. I've found the AWS CLI best, and a lot of the time I think it's easier just to write a Ruby script using the SDK. This obviously doesn't necessarily handle teardown very well, and it tends to be copying boilerplate and modifying it, but I find it the most straightforward thing, and simple, if a little verbose.

empath75 11y ago

There's already a python module called troposphere.

nzoschke 11y ago

I am asking this question often. We are using very advanced CloudFormation in the open source Convox platform. https://github.com/convox/rack https://github.com/convox/rack I have touched every corner of CF including lots of Custom Resources. Right now we are using the golang template tools and tests to generate our templates. But I have lots of needs and ideas for improving this. A CF template compiler and simulator should be possible, giving us all tons of confidence in making template changes and therefore any infrastructure update. I have some sketches that I haven't published yet. And I strongly believe CF is the best tool in this space if you're all in on AWS. Let Amazon be responsible for operating a transactional infrastructure mutation service. It's ridiculously hard to do this right. If you want to brainstorm some ideas send me a message :)

wise_young_man 11y ago
One of the things that I thought would be neat is an open source CloudFormation that could work for multiple cloud vendors, possibly using a driver pattern. Also, you might want to update your HN profile with contact info.
- nzoschke 11y ago
  Thanks, I updated my profile with contact info. Terraform is awesome if you want an OSS project to manage multiple cloud vendors. But I think that infrastructure change management is a really hard problem and the state of the art solution is how AWS runs CloudFormation as a managed service. Once it's set up properly, it's amazing watching what CloudFormation can do. It can execute updating 20 instances to roll out a new AMI, and then roll the whole operation back on demand or if a failure happens. All with no application downtime in the cluster!

vankap 11y ago

Create the infrastructure manually and then use Cloudformer tool to generate the template based on the already created infrastructure. You can then edit the generated template to make it more maintainable and you have a nice reusable template.

obulpathi 11y ago

CloudFormation is not very elegant. Things become complicated with the concept region, zonal resources (same AMI is represented by a different id in different region etc). Try Google Clouds's Deployment Manager. Functionally similar, but Google Cloud Deployment Manager is far easier (everything is a global resource), Jinja based templates (you get to write for's and if's, evaluate lists, dictionaries inside templates .. )

Corrado 11y ago

I built a JSON templating system[0] (based on Handlebars) that is project based. Basically, you create new JSON files for each project and the system generates a CFN template that contains everything that project needs (EC2 instances, RDS instances, security groups, etc.) It's still a work in progress but I'm using it in production in my day job and I'm pretty happy with how it works. More help is always accepted. :) [0] https://github.com/rnhurt/CFNBuilder https://github.com/rnhurt/CFNBuilder