22 ms·

DevOps in Milliseconds

39 points by pedoh 15y ago

ww520 15y ago
Speaking of deployment, anyone has a good suggestion for a simple deploying system in AWS? Ideally a generic EC2 instance is booted up with an apptype passed in, and based on the apptype it downloads the app code from S3 and starts up the app. The code is pushed to S3 by dev when it's ready. Is there such a system?
- jat850 15y ago
  Are you think of something like chef? http://www.opscode.com/chef/ http://www.opscode.com/chef/ It maybe doesn't fit the bill of simple, but it's not too dreadfully hard to get set up if you only want it to fulfill some relatively straightforward tasks like you describe.
  - ww520 15y ago
    Thanks. I'll take a look at it again. The last time I looked at it (along with Puppet and others), it seemed to be rather complicated with setting up Chef Server, CouchDB, RabbitMQ, and Solr. Ideally Chef just runs at the boot time of a EC2 instance, reads the recipes off from S3 and continues from there.
    - joevandyk 15y ago
      You want chef-solo. No need for chef server.
    - pedoh 15y ago
      Chef and Puppet can definitely do this for you, without servers involved. Another great use of Chef and Puppet can be seen in Vagrant (http://vagrantup.com/ http://vagrantup.com/). Vagrant uses VirtualBox (http://www.virtualbox.org/ http://www.virtualbox.org/) to create virtual instances easily (REALLY easily) and then can automatically provision them via Chef or Puppet. Sweet stuff. I love the ability to bring up a new server with the things I want to have so quickly.
    - davnola 15y ago
      opscode now have a hosted chef server platform https://manage.opscode.com/ https://manage.opscode.com/
  - snprbob86 15y ago
    I did a whole bunch of research into DevOps and Chef (or Puppet) were the clear winners. That's really kinda sad because Heroku's model is just sooo much more pleasant as a developer... at least to start. I'd really love to see a better abstraction than "idempotent wrappers for shell commands" which is what Chef and Puppet basically are. Most applications are simple enough to run as a single process with a read-only file system and log to standard out. Surely someone could use the Heroku Procfile model a reasonable starting point and create escape hatches down to lower levels as you need them.
- pedoh 15y ago
  We're using Puppet for our deployments, and not just initial deployments, but for code deployments as well. The puppet node classifier talks to a MySQL database that has the "right" versions for a particular server in it; upgrading consists of changing the version for a server to the desired version, and then running Puppet. I think you could create an EC2 instance that automatically runs Puppet (or Chef) on startup pretty easily. And as someone else mentioned here, you don't have to do a complicated setup with a server to get going, either with Chef or Puppet.
- cwp 15y ago
  I use a system based on Fabric. It fires up an instance, connects via SSH, installs and configures everything needed for the app, saves an AMI, then terminates the instance. Then deployment is just a matter of launching an instance with that AMI.
  - mattdeboard 15y ago
    I built out one of these at work, and while it is indeed pretty neat, it still has a lot of pain points and complexity -- at least in our case -- that could be ameliorated by and is better suited for Puppet/Chef.
ams6110 15y ago
If you're interested in this stuff, there is a "Camp DevOps" in Chicago next month. http://campdevops.com/ http://campdevops.com/ Involves some of the same folks who put on the excellent Erlang Camp I attended last year, on the basis of that experience, I'd expect this to be a good value.
mopoke 15y ago
Apart from a sentence at the end, there's no mention at all of culture and people which is really the key to doing DevOps. You can use all of the tools mentioned but still encounter the same problems as a traditional ops setup if you don't get the teams working together.
- pedoh 15y ago
  mopoke, You're absolutely correct; without the people processes in place, the world's best tools aren't going to do anything for you. I'll ponder over that for the next blog post.
ajtaylor 15y ago
I'm probably not the only one who noticed, but the second screenshot (for the Graphite stats) is almost certainly from an ExtJS app. Is there an open source equivalent of ExtJS? The pricing seems to have gone up quite a bit from the last time I check a year or so ago. I'd bet it's still a good value for the money, but options are nice to have.
- mnutt 15y ago
  ExtJS is open source. (GPLv3) Since Graphite is Apache 2.0, it looks like they're only able to use it due to some sort of exemption from Sencha. I don't know of any total replacements for ExtJS but something like Sproutcore may come close.
  - pedoh 15y ago
    ExtJS has two commercial licenses and an open source license: http://www.sencha.com/products/extjs/license/ http://www.sencha.com/products/extjs/license/
  - ajtaylor 15y ago
    It's only open source for open source projects. I would be using at $work, so a commercial license is my only option. Way back when, I remember the "discussions" that took place when ExtJS backed off from it's original true open source license. Perhaps I'll look at Sproutcore now that it's had more time to mature.
pedoh 15y ago
By the way, AppNexus is hiring. I've been working with a number of AppNexians for 7 years. The employees at AppNexus are bright and motivated, but also bring a sense of fun to the office. I've also appreciated the family values that AppNexus believes in. My boss has even gotten on my case for not scheduling vacation, because he knows that ultimately burning out does me nor the company any good. If you're interested in hearing more, check out http://appnexus.com/careers/ http://appnexus.com/careers/ or drop me an email.
krallja 15y ago
Still using Subversion? I think that should be your next improvement target.
- pedoh 15y ago
  I can't argue with that one, but gravity's a bit tough. Several of my colleagues are using git-svn, and would probably love to flip over to git.

39 points by pedoh 15y ago

ww520 15y ago

Speaking of deployment, anyone has a good suggestion for a simple deploying system in AWS? Ideally a generic EC2 instance is booted up with an apptype passed in, and based on the apptype it downloads the app code from S3 and starts up the app. The code is pushed to S3 by dev when it's ready. Is there such a system?

jat850 15y ago
Are you think of something like chef? http://www.opscode.com/chef/ http://www.opscode.com/chef/ It maybe doesn't fit the bill of simple, but it's not too dreadfully hard to get set up if you only want it to fulfill some relatively straightforward tasks like you describe.
- ww520 15y ago
  Thanks. I'll take a look at it again. The last time I looked at it (along with Puppet and others), it seemed to be rather complicated with setting up Chef Server, CouchDB, RabbitMQ, and Solr. Ideally Chef just runs at the boot time of a EC2 instance, reads the recipes off from S3 and continues from there.
  - joevandyk 15y ago
    You want chef-solo. No need for chef server.
  - pedoh 15y ago
    Chef and Puppet can definitely do this for you, without servers involved. Another great use of Chef and Puppet can be seen in Vagrant (http://vagrantup.com/ http://vagrantup.com/). Vagrant uses VirtualBox (http://www.virtualbox.org/ http://www.virtualbox.org/) to create virtual instances easily (REALLY easily) and then can automatically provision them via Chef or Puppet. Sweet stuff. I love the ability to bring up a new server with the things I want to have so quickly.
  - davnola 15y ago
    opscode now have a hosted chef server platform https://manage.opscode.com/ https://manage.opscode.com/
- snprbob86 15y ago
  I did a whole bunch of research into DevOps and Chef (or Puppet) were the clear winners. That's really kinda sad because Heroku's model is just sooo much more pleasant as a developer... at least to start. I'd really love to see a better abstraction than "idempotent wrappers for shell commands" which is what Chef and Puppet basically are. Most applications are simple enough to run as a single process with a read-only file system and log to standard out. Surely someone could use the Heroku Procfile model a reasonable starting point and create escape hatches down to lower levels as you need them.
pedoh 15y ago
We're using Puppet for our deployments, and not just initial deployments, but for code deployments as well. The puppet node classifier talks to a MySQL database that has the "right" versions for a particular server in it; upgrading consists of changing the version for a server to the desired version, and then running Puppet. I think you could create an EC2 instance that automatically runs Puppet (or Chef) on startup pretty easily. And as someone else mentioned here, you don't have to do a complicated setup with a server to get going, either with Chef or Puppet.
cwp 15y ago
I use a system based on Fabric. It fires up an instance, connects via SSH, installs and configures everything needed for the app, saves an AMI, then terminates the instance. Then deployment is just a matter of launching an instance with that AMI.
- mattdeboard 15y ago
  I built out one of these at work, and while it is indeed pretty neat, it still has a lot of pain points and complexity -- at least in our case -- that could be ameliorated by and is better suited for Puppet/Chef.

ams6110 15y ago

If you're interested in this stuff, there is a "Camp DevOps" in Chicago next month. http://campdevops.com/ http://campdevops.com/ Involves some of the same folks who put on the excellent Erlang Camp I attended last year, on the basis of that experience, I'd expect this to be a good value.

mopoke 15y ago

Apart from a sentence at the end, there's no mention at all of culture and people which is really the key to doing DevOps. You can use all of the tools mentioned but still encounter the same problems as a traditional ops setup if you don't get the teams working together.

pedoh 15y ago
mopoke, You're absolutely correct; without the people processes in place, the world's best tools aren't going to do anything for you. I'll ponder over that for the next blog post.

ajtaylor 15y ago

I'm probably not the only one who noticed, but the second screenshot (for the Graphite stats) is almost certainly from an ExtJS app. Is there an open source equivalent of ExtJS? The pricing seems to have gone up quite a bit from the last time I check a year or so ago. I'd bet it's still a good value for the money, but options are nice to have.

mnutt 15y ago
ExtJS is open source. (GPLv3) Since Graphite is Apache 2.0, it looks like they're only able to use it due to some sort of exemption from Sencha. I don't know of any total replacements for ExtJS but something like Sproutcore may come close.
- pedoh 15y ago
  ExtJS has two commercial licenses and an open source license: http://www.sencha.com/products/extjs/license/ http://www.sencha.com/products/extjs/license/
- ajtaylor 15y ago
  It's only open source for open source projects. I would be using at $work, so a commercial license is my only option. Way back when, I remember the "discussions" that took place when ExtJS backed off from it's original true open source license. Perhaps I'll look at Sproutcore now that it's had more time to mature.

pedoh 15y ago

By the way, AppNexus is hiring. I've been working with a number of AppNexians for 7 years. The employees at AppNexus are bright and motivated, but also bring a sense of fun to the office. I've also appreciated the family values that AppNexus believes in. My boss has even gotten on my case for not scheduling vacation, because he knows that ultimately burning out does me nor the company any good. If you're interested in hearing more, check out http://appnexus.com/careers/ http://appnexus.com/careers/ or drop me an email.

krallja 15y ago

Still using Subversion? I think that should be your next improvement target.

pedoh 15y ago
I can't argue with that one, but gravity's a bit tough. Several of my colleagues are using git-svn, and would probably love to flip over to git.