4 ms·
Congratulations! I've been a fan of Vagrant for a long time and have done some Vault as well. I haven't yet had a chance to try Terraform, but everyone says it
by pjungwir 8y ago
Congratulations! I've been a fan of Vagrant for a long time and have done some Vault as well. I haven't yet had a chance to try Terraform, but everyone says it is better than Cloudformation. :-)
I'm curious what you think of this crazy idea?:
With databases, some ecosystems use declarative migrations, where you keep a description of your schema, and the system automatically takes you from here to there. I've heard .Net people say how nice this is. On the other hand with Rails & Django you have delta scripts that give imperative directions for getting from here to there, rather than a declaration of the end state. I like that solution a lot, because sometimes you need more control over what to do. (For example, changing a one-to-many relationship to many-to-many, and moving the existing data.)
It seems to me that Cloudformation and Terraform take the declarative approach to cloud management, and there is no tool for the imperative approach. (Cloudformation has Change Sets but you can't write those or edit them; they just tell you its plan.) Resizing your database's EBS volume may be an example where you want to give careful instructions. Try that in Cloudformation and it will destroy the old volume and attach a new one. :-)
I'd love a tool that lets you keep a history of "cloud migrations", where you write both the "up" and the "down". It could be as simple as some organization around a bunch of fog/boto/aws-sdk scripts, although it would be even nicer if it had helper methods like in Rails, e.g. add_autoscaling_group, and those would usually know how to reverse themselves.
The really great feature would be to have "transactional DDL" for cloud migrations. With database migrations you get this for Postgres but not MySQL, and it's really nice. I know this isn't 100% possible, but CloudFormation comes close already with its rollback attempts. You'd definitely need your own layer of helper methods for this, because I can't imagine implementing it without something like a Command pattern. (The way AWS auto-generates their API tools, maybe it wouldn't even be that difficult to generate an identical API that just records what you want to do.)
I've pitched this idea a few times to AWS people, but maybe you should do it instead. :-)
Anyway, Hashicorp has contributed a ton to my developer happiness, so I wish you great success. Oh, and congratulations on your wedding!
- scarface74 8y agoIt seems to me that Cloudformation and Terraform take the declarative approach to cloud management, and there is no tool for the imperative approach. With CloudFormation you can use custom lambda backed resources.
- pjungwir 8y agoHmm, thanks for mentioning this. It looks like it's just over a month old? Here are the docs I found: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources-lambda.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui... https://aws.amazon.com/blogs/aws/cloudformation-macros/ https://aws.amazon.com/blogs/aws/cloudformation-macros/ That could be really useful!
- scarface74 8y agoIt’s been around for at least two years from what I can tell.
- codeaken 8y agoI'd love a tool that lets you keep a history of "cloud migrations", where you write both the "up" and the "down". We did a dirty solution for this with Ansible. We have a base playbook (with roles and all that jazz) that sets up a server and then we can write patches which are separate playbooks. After the patch has run we simply store the patchlevel in a file on the server so that we dont "double-apply" any patches on the next patch-run.
- philsnow 8y agoI've asked aws in the past for an API that lets you describe a sequence of changes in a batch, it tells you whether there's any problems with the sequence (like "at step nine of this batch, you tried to create an instance in a vpc subnet that was deleted by step four of this batch"). And then if there are no issues with the batch, another API lets you have aws execute the batch. It sounds a lot like the DDL transactions you mentioned.
- ajcodez 8y agoI wrote a tool to auto migrate postres using HashiCorp Config Language. It has the issues you describe — hard to rename tables and adding indexes can take a long time on existing data. It’s pretty convenient if you work within the constraints however. https://github.com/aj0strow/pgschema https://github.com/aj0strow/pgschema