4 ms·

The OP says he opted not to use Chef/Puppet and went with baked AMIs instead. In my experience, Chef simply takes too long to build a machine. I use it to ensur

The OP says he opted not to use Chef/Puppet and went with baked AMIs instead. In my experience, Chef simply takes too long to build a machine. I use it to ensure a repeatable, self-documented machine setup, but then if I'm on AWS I snapshot an AMI for quick scaling. This also lets you more easily use Amazon auto-scaling tools like Cloud Formation, which bring up new machines based on an AMI. But having the Chef script is still great for knowing what's on your box and having "source code" to change it. One thing I'm curious about re the OP's process: he says he is using Fabric for deploys. Does that mean every time he deploys new code he has to snapshot a new AMI? In that case, why use Fabric at all? I'd be worried about auto-deloying AMIs with outdated code. Since EC2 instances are "disposable," one approach is to never "update" an instance, but instead you release new code by simply launching fresh instances with the latest code, then destroying the old ones.

scottvdp 13y ago
The baked AMI comes up and pulls, so it doesn't need to be baked each time. If you take the route you just mentioned, I highly recommend using Netflix's Asgard (https://github.com/Netflix/asgard https://github.com/Netflix/asgard) and check out their recently released Aminator (https://github.com/Netflix/aminator https://github.com/Netflix/aminator). Asgard specifically makes AMI based deploys outrageously easy.
imperialWicket 13y ago
My preference is to make a base ami that's configured with puppet and a couple core user/keys. I make this instance automatically hit our puppetmaster and use things like security_groups and user data to figure out what type of node it is. One of the things that puppet installs is a script to generate a new ami from the built instance, this script optionally deletes the puppet package. Now you have an easy way to build your ami from the ground up, you only need to worry about core OS package enhancements on the base ami, and you can use your created AMI with autoscaling behind an ELB. For VERY rapid changes - direct git access is optional (even on startup for current code pulls), but I tend to side with the Netflix guys and focus on an AMI as the minimum unit of deployment. Check out asgard to make this easier (senseless self plug: I built an asgard ami this weekend - http://imperialwicket.com/netflix-asgard-ubuntu-1204-lts-ami-release-11101 http://imperialwicket.com/netflix-asgard-ubuntu-1204-lts-ami...).
rbc 13y ago
I'm a fan of CFEngine and use it for the things that many use Puppet or Chef to do. I think the key advantage of some kind of system domain specific language, is it helps document how things are configured. The only way to figure out what's going on in AMIs is mounting their file systems and running diffs to see what changed AMI, to AMI. Having a large number of AMI's can also be a problem in the respect that they proliferate, particularly when you deploy new applications, based on old applications, but slightly different. On the other hand, if you are using auto-scaling, Chef/Puppet/CFEngine may not be able to finish their work in time if they have to do a lot of work. You have to strike the right balance between what needs to be in the AMI (provisioning) and automation (post-install).

scottvdp 13y ago

The baked AMI comes up and pulls, so it doesn't need to be baked each time. If you take the route you just mentioned, I highly recommend using Netflix's Asgard (https://github.com/Netflix/asgard https://github.com/Netflix/asgard) and check out their recently released Aminator (https://github.com/Netflix/aminator https://github.com/Netflix/aminator). Asgard specifically makes AMI based deploys outrageously easy.

imperialWicket 13y ago

My preference is to make a base ami that's configured with puppet and a couple core user/keys. I make this instance automatically hit our puppetmaster and use things like security_groups and user data to figure out what type of node it is. One of the things that puppet installs is a script to generate a new ami from the built instance, this script optionally deletes the puppet package. Now you have an easy way to build your ami from the ground up, you only need to worry about core OS package enhancements on the base ami, and you can use your created AMI with autoscaling behind an ELB. For VERY rapid changes - direct git access is optional (even on startup for current code pulls), but I tend to side with the Netflix guys and focus on an AMI as the minimum unit of deployment. Check out asgard to make this easier (senseless self plug: I built an asgard ami this weekend - http://imperialwicket.com/netflix-asgard-ubuntu-1204-lts-ami-release-11101 http://imperialwicket.com/netflix-asgard-ubuntu-1204-lts-ami...).

rbc 13y ago

I'm a fan of CFEngine and use it for the things that many use Puppet or Chef to do. I think the key advantage of some kind of system domain specific language, is it helps document how things are configured. The only way to figure out what's going on in AMIs is mounting their file systems and running diffs to see what changed AMI, to AMI. Having a large number of AMI's can also be a problem in the respect that they proliferate, particularly when you deploy new applications, based on old applications, but slightly different. On the other hand, if you are using auto-scaling, Chef/Puppet/CFEngine may not be able to finish their work in time if they have to do a lot of work. You have to strike the right balance between what needs to be in the AMI (provisioning) and automation (post-install).