3 ms·

Can anyone point us towards good "server configuration management for dummies (who happen to run startups)" documentation? I've been looking at Puppet and Chef

Can anyone point us towards good "server configuration management for dummies (who happen to run startups)" documentation? I've been looking at Puppet and Chef, but we're not just Rails anymore and I can't imagine that the Ruby world's tools are the only popular ones. I just don't know where to look for how to make an informed choice, how to get started, etc. It feels like there is a large gap in available knowledge between running a couple of processes on a single server and anything I'd trust to some paying customers.

bretthoerner 15y ago
Those aren't just the Ruby world's tools, fwiw. We use Puppet at Disqus (a Python shop). Puppet and Chef are used by pretty much any modern web deployment that didn't already have a system in something more dated (cfengine? custom made?). At least that's the feeling I get. I can't speak much to Chef but Puppet has great docs. You might just get an intro book for one or the other and dive in.
- cparedes 15y ago
  Chef has decent documentation, but the last time I checked, it was a lot less polished than Puppet's documentation. With that said, the folks in #chef on irc.freenode.net are quite helpful, and it's possible that the documentation has improved since the release of 0.10.
IgorPartola 15y ago
Puppet is pretty great and at TransLoc we don't use any Ruby. We combine it with packaging all of our code as Debian packages and it makes bringing up a new server a matter of installing puppet, then running the puppet manifests. Note that the easiest way to start is to not use any of puppet's built-in client-server stuff (puppetd command). Instead run everything like this: $ git clone $REMOTE_REPO_WITH_MANIFESTS puppet $ ls manifests modules $ sudo puppet --modulepath=modules/ --debug manifests/site.pp You could use something like fabric if you only have 1-2 servers.
rmaccloy 15y ago
Puppet and Chef are pretty agnostic. You can use them to manage only your infrastructure (packages, user accounts, databases) and leave app deployment up to something like capistrano or fabric if you like (although at a certain level of app complexity it makes sense to integrate.) cfengine has been around longer, but seems to have negative adoption since puppet became well-known. I know a few people who swear by it, though. FWIW, at my current startup I use Chef to manage a mid-sized collection of servers (app, load balancers, DBs/replicas, task processors) where the app is written mostly in Python, and it took a ton of pain out of scaling up to deal with a very large paying customer two nights ago. Puppet has better documentation, but learning either is a bit of a trial and error process I think. You might find Blueprint (https://github.com/devstructure/blueprint https://github.com/devstructure/blueprint) useful to bootstrap (the authors are around in the HN community, I think.)
- howradical 15y ago
  Blueprint author here. Our goal is dead simple configuration management, so it might be just what you need. Check it out and feel free to email me with any questions matt@devstructure.com Blueprint (https://github.com/devstructure/blueprint https://github.com/devstructure/blueprint) Blueprint I/O (https://github.com/devstructure.com/blueprint-io https://github.com/devstructure.com/blueprint-io)
josegonzalez 15y ago
I personally use Chef to deploy PHP-based web applications (very little ruby, only running God to be honest), and at work we deploy Python, Ruby, PHP, and all the services and subsystems our applications require. The biggest pain is that the docs were iffy, at least for me. I wasn't the one setting up the servers at work, but when setting up a Chef cookbook for personal use, it was a pain to both learn the DSL and deal with the silliness caused by bugs that hadn't quite made it to release yet. At the end of the day though, its more just experimenting with the tools available until you're comfortable with one. You can also look into Fabric, which is a Python server automation tool.
2mur 15y ago
Here's[1] a nice multi-part graduated walk-through of deploying Django with chef. I'm working using it as a guidepost on deploying my Django stuff (currently hitting a VM on my macbook so I can rollback to a snapshot and make sure it works right). [1]http://ericholscher.com/blog/2010/nov/8/building-django-app-server-chef/ http://ericholscher.com/blog/2010/nov/8/building-django-app-...
cparedes 15y ago
The consensus on CM tool choice is simply, "pick one that's comfortable for you and your team," and, "it doesn't matter which one you use, just be sure you're using configuration management." It's definitely better than dealing with shell scripts, since a lot of things you write in your CM language gets abstracted automatically for whatever operating system you're using (this is the biggest takeaway - you really don't want to rewrite shell scripts just because you've decided to add a different Linux distro or UNIX in your network.) I've used both Puppet and Chef extensively, and they both do the same thing, they just go about it differently. (shameless plug of a blog post I wrote that compared the two: http://redbluemagenta.com/2011/05/21/puppet-vs-chef http://redbluemagenta.com/2011/05/21/puppet-vs-chef) Also, it's almost always about how you approach configuration management - thankfully, with Puppet and Chef, you don't have to manage everything on a system in your CM right away. If you have to manage SSH authorized_key entries and it's getting annoying to do it by hand, then write a Puppet module / Chef cookbook to manage it for you. Then go on to manage /etc/hosts entries, then to deploying packages... you'll hit a point where you can reasonably rebuild a server from scratch without thinking about it. There's other CM systems out there too: bcfg2, cfengine, etc.
atsaloli 15y ago
To help you make an informed choice, this summary of the Configuration Management Summit may be useful: http://www.usenix.org/publications/login/2010-10/openpdfs/ConfigMgt10reports.pdf http://www.usenix.org/publications/login/2010-10/openpdfs/Co... To help you get started with Cfengine (my specialty): http://www.cfengine.com/pages/tech http://www.cfengine.com/pages/tech http://www.verticalsysadmin.com/cfengine/cfengine_examples.tar http://www.verticalsysadmin.com/cfengine/cfengine_examples.t...
atsaloli 15y ago
Here is a great article on getting started with configuration management, with lots of workable tips: http://www.usenix.org/publications/login/2011-04/openpdfs/Lueninghoener.pdf http://www.usenix.org/publications/login/2011-04/openpdfs/Lu...

bretthoerner 15y ago

Those aren't just the Ruby world's tools, fwiw. We use Puppet at Disqus (a Python shop). Puppet and Chef are used by pretty much any modern web deployment that didn't already have a system in something more dated (cfengine? custom made?). At least that's the feeling I get. I can't speak much to Chef but Puppet has great docs. You might just get an intro book for one or the other and dive in.

cparedes 15y ago
Chef has decent documentation, but the last time I checked, it was a lot less polished than Puppet's documentation. With that said, the folks in #chef on irc.freenode.net are quite helpful, and it's possible that the documentation has improved since the release of 0.10.

IgorPartola 15y ago

Puppet is pretty great and at TransLoc we don't use any Ruby. We combine it with packaging all of our code as Debian packages and it makes bringing up a new server a matter of installing puppet, then running the puppet manifests. Note that the easiest way to start is to not use any of puppet's built-in client-server stuff (puppetd command). Instead run everything like this: $ git clone $REMOTE_REPO_WITH_MANIFESTS puppet $ ls manifests modules $ sudo puppet --modulepath=modules/ --debug manifests/site.pp You could use something like fabric if you only have 1-2 servers.

rmaccloy 15y ago

Puppet and Chef are pretty agnostic. You can use them to manage only your infrastructure (packages, user accounts, databases) and leave app deployment up to something like capistrano or fabric if you like (although at a certain level of app complexity it makes sense to integrate.) cfengine has been around longer, but seems to have negative adoption since puppet became well-known. I know a few people who swear by it, though. FWIW, at my current startup I use Chef to manage a mid-sized collection of servers (app, load balancers, DBs/replicas, task processors) where the app is written mostly in Python, and it took a ton of pain out of scaling up to deal with a very large paying customer two nights ago. Puppet has better documentation, but learning either is a bit of a trial and error process I think. You might find Blueprint (https://github.com/devstructure/blueprint https://github.com/devstructure/blueprint) useful to bootstrap (the authors are around in the HN community, I think.)

howradical 15y ago
Blueprint author here. Our goal is dead simple configuration management, so it might be just what you need. Check it out and feel free to email me with any questions matt@devstructure.com Blueprint (https://github.com/devstructure/blueprint https://github.com/devstructure/blueprint) Blueprint I/O (https://github.com/devstructure.com/blueprint-io https://github.com/devstructure.com/blueprint-io)

josegonzalez 15y ago

I personally use Chef to deploy PHP-based web applications (very little ruby, only running God to be honest), and at work we deploy Python, Ruby, PHP, and all the services and subsystems our applications require. The biggest pain is that the docs were iffy, at least for me. I wasn't the one setting up the servers at work, but when setting up a Chef cookbook for personal use, it was a pain to both learn the DSL and deal with the silliness caused by bugs that hadn't quite made it to release yet. At the end of the day though, its more just experimenting with the tools available until you're comfortable with one. You can also look into Fabric, which is a Python server automation tool.

2mur 15y ago

Here's[1] a nice multi-part graduated walk-through of deploying Django with chef. I'm working using it as a guidepost on deploying my Django stuff (currently hitting a VM on my macbook so I can rollback to a snapshot and make sure it works right). [1]http://ericholscher.com/blog/2010/nov/8/building-django-app-server-chef/ http://ericholscher.com/blog/2010/nov/8/building-django-app-...

cparedes 15y ago

The consensus on CM tool choice is simply, "pick one that's comfortable for you and your team," and, "it doesn't matter which one you use, just be sure you're using configuration management." It's definitely better than dealing with shell scripts, since a lot of things you write in your CM language gets abstracted automatically for whatever operating system you're using (this is the biggest takeaway - you really don't want to rewrite shell scripts just because you've decided to add a different Linux distro or UNIX in your network.) I've used both Puppet and Chef extensively, and they both do the same thing, they just go about it differently. (shameless plug of a blog post I wrote that compared the two: http://redbluemagenta.com/2011/05/21/puppet-vs-chef http://redbluemagenta.com/2011/05/21/puppet-vs-chef) Also, it's almost always about how you approach configuration management - thankfully, with Puppet and Chef, you don't have to manage everything on a system in your CM right away. If you have to manage SSH authorized_key entries and it's getting annoying to do it by hand, then write a Puppet module / Chef cookbook to manage it for you. Then go on to manage /etc/hosts entries, then to deploying packages... you'll hit a point where you can reasonably rebuild a server from scratch without thinking about it. There's other CM systems out there too: bcfg2, cfengine, etc.

atsaloli 15y ago

To help you make an informed choice, this summary of the Configuration Management Summit may be useful: http://www.usenix.org/publications/login/2010-10/openpdfs/ConfigMgt10reports.pdf http://www.usenix.org/publications/login/2010-10/openpdfs/Co... To help you get started with Cfengine (my specialty): http://www.cfengine.com/pages/tech http://www.cfengine.com/pages/tech http://www.verticalsysadmin.com/cfengine/cfengine_examples.tar http://www.verticalsysadmin.com/cfengine/cfengine_examples.t...

atsaloli 15y ago

Here is a great article on getting started with configuration management, with lots of workable tips: http://www.usenix.org/publications/login/2011-04/openpdfs/Lueninghoener.pdf http://www.usenix.org/publications/login/2011-04/openpdfs/Lu...