4 ms·

I made a script to automate django deployments on Amazon EC2

91 points by flexterra 15y ago

pbreit 15y ago
I would really like to see more Fabric scripts like this available. Thanks for this one.
- jpadilla 15y ago
  I agree. Very awesome script and pretty easy to build on top of this, kinda like Chef Recipes.
- neeleshs 15y ago
  Here's the one that I wrote for my Linode setup. Nothing specific to linode, though. https://github.com/neeleshs/django_project_template/blob/master/fabfile.py https://github.com/neeleshs/django_project_template/blob/mas...
hoop 15y ago
Very cool. Curious why you didn't use fabric.api.put instead of `run(_write_to(config_file, "/home/%(SERVER_USERNAME)s/nginx.conf" % fabconf))` ?
- flexterra 15y ago
  You are right, I will fix that
jpadilla 15y ago
Took less than 2 minutes to deploy to EC2!
zitterbewegung 15y ago
How does this work? If I change my app on my local machine do I use fab to redeploy it?
- flexterra 15y ago
  Currently the script only helps creating an EC2 instante and setting it up for a django app. I plan to add common "re-deployment" stuff, but is very hard to decide what to include because everybody uses django differently. We need to develop best practices and conventions.
  - tomazmuraus 15y ago
    Any special reason why you haven't used something like libcloud (deploy_node ...) - http://libcloud.apache.org/ http://libcloud.apache.org/? Then the script would work with multiple providers, not just Amazon.
    - flexterra 15y ago
      Did not know about it, but it looks very nice. I will give it a try.
gourneau 15y ago
Cool! Can you please describe how to get preexisting Django apps served with this in more detail. edit: reading the config file turns out to be a good place to start.
- flexterra 15y ago
  Just edit the config file and run the fab fab command. That will setup the server and create an empty django app in /home/ubuntu/webapps. After that just replace that app with yours. You still might need to install application specific dependencies.
amikhailov 15y ago
thanks, it's very interested, did you try capistrano before?
- Terretta 15y ago
  Django is Python, so is Fabric. Capistrano is Ruby, so is Rails.
- ScotterC 15y ago
  If you're looking for a similar script for Ruby, check out Rubber. https://github.com/wr0ngway/rubber https://github.com/wr0ngway/rubber
Joakal 15y ago
Would it be better to use a configuration management tool that does eventual automation to nodes on pull requests?
ez77 15y ago
On a related note, is there a repository of generic (though distribution-specific) deployment scripts for VPSes [+]? In my personal case I'm thinking along the lines of setting up BIND, sendmail and tweaking your HTTP server in one go, but the more choices the better. [+] It would clearly apply to physical servers as well, but probably this carefree approach would be used most on VPSes.
- morganpyne 15y ago
  It's not quite exactly what you asked for, but have you seen the StackScripts from Linode? http://www.linode.com/stackscripts/ http://www.linode.com/stackscripts/ I have personally found that even distro-specific scripts require some tweaking across various hosting providers due to the default configuration of VPSs being different in each case. It has generally not been worth the effort for me to write complete bootstrap scripts so generic that they will work on anybody's VPS, but it's usually trivially easy to adapt whatever I have written to the quirks of a new hosting environment if the need arises.
  - ez77 15y ago
    Thanks! Since I'm far from launching any serious work, I'm still learning with cheaper VPSes, but thanks for your tip. In particular, the term 'bootstrap scripts' will prove handy. (Similar standard terms, suitable for Googling, will be welcomed.)
nzoschke 15y ago
Nice. The state of the art here is to use AWS CloudFormation to bring up the instance. With CloudFormation you can specify the base AMI and other resources, such as an Elastic IP, and AWS will either bring up the stack or rollback the entire transaction if something goes wrong. Then, you specify the script to set up the AMI as a userdata script, and that will be run as root when the instance boots. The downside here is that the CloudFormation syntax is very funky, and 100% AWS specific. This might justify sticking with your strategy of simple fabric scripts.
- bmelton 15y ago
  I like dotcloud for this, but before I fell in love with dotcloud, this is exactly the sort of solution I was looking for.
ridruejo 15y ago
Not exactly the same, but BitNami provides ready-made Django AMIs http://bitnami.org/stack/djangostack http://bitnami.org/stack/djangostack
jokull 15y ago
I maintain a CLI tool that does something similar but for WSGI apps, and with a slightly different stack (uWSGI and daemontools). It uses libcloud to create instances so rackspace and linode are also supported among others. https://github.com/jokull/kraftwerk https://github.com/jokull/kraftwerk
ronnix 15y ago
Cool. Fabric rocks! People may also want to check out Ian Bicking's Silver Lining tool: http://cloudsilverlining.org/ http://cloudsilverlining.org/
alonswartz 15y ago
A little late to the conversation, but none the less... You could use the TurnKey Hub for easy deployment of a pre-configured Django appliance to Amazon EC2. If you prefer the command line you can use the Hub's API or CLI, for example: $ hub-launch django [-options] If you've enabled TKLBAM (encrypted backups of changes to files, databases, package management state, even users and groups), you can launch a new cloud server which will be automatically restored upon boot: $ hub-launch $BACKUP_ID Take that a step further and say you're developing your Django app in a local VM. When you're ready to deploy to the cloud you just trigger a backup, then launch it in the cloud.

91 points by flexterra 15y ago

pbreit 15y ago

I would really like to see more Fabric scripts like this available. Thanks for this one.

jpadilla 15y ago
I agree. Very awesome script and pretty easy to build on top of this, kinda like Chef Recipes.
neeleshs 15y ago
Here's the one that I wrote for my Linode setup. Nothing specific to linode, though. https://github.com/neeleshs/django_project_template/blob/master/fabfile.py https://github.com/neeleshs/django_project_template/blob/mas...

hoop 15y ago

Very cool. Curious why you didn't use fabric.api.put instead of `run(_write_to(config_file, "/home/%(SERVER_USERNAME)s/nginx.conf" % fabconf))` ?

flexterra 15y ago
You are right, I will fix that

jpadilla 15y ago

Took less than 2 minutes to deploy to EC2!

zitterbewegung 15y ago

How does this work? If I change my app on my local machine do I use fab to redeploy it?

flexterra 15y ago
Currently the script only helps creating an EC2 instante and setting it up for a django app. I plan to add common "re-deployment" stuff, but is very hard to decide what to include because everybody uses django differently. We need to develop best practices and conventions.
- tomazmuraus 15y ago
  Any special reason why you haven't used something like libcloud (deploy_node ...) - http://libcloud.apache.org/ http://libcloud.apache.org/? Then the script would work with multiple providers, not just Amazon.
  - flexterra 15y ago
    Did not know about it, but it looks very nice. I will give it a try.

gourneau 15y ago

Cool! Can you please describe how to get preexisting Django apps served with this in more detail. edit: reading the config file turns out to be a good place to start.

flexterra 15y ago
Just edit the config file and run the fab fab command. That will setup the server and create an empty django app in /home/ubuntu/webapps. After that just replace that app with yours. You still might need to install application specific dependencies.

amikhailov 15y ago

thanks, it's very interested, did you try capistrano before?

Terretta 15y ago
Django is Python, so is Fabric. Capistrano is Ruby, so is Rails.
ScotterC 15y ago
If you're looking for a similar script for Ruby, check out Rubber. https://github.com/wr0ngway/rubber https://github.com/wr0ngway/rubber

Joakal 15y ago

Would it be better to use a configuration management tool that does eventual automation to nodes on pull requests?

ez77 15y ago

On a related note, is there a repository of generic (though distribution-specific) deployment scripts for VPSes [+]? In my personal case I'm thinking along the lines of setting up BIND, sendmail and tweaking your HTTP server in one go, but the more choices the better. [+] It would clearly apply to physical servers as well, but probably this carefree approach would be used most on VPSes.

morganpyne 15y ago
It's not quite exactly what you asked for, but have you seen the StackScripts from Linode? http://www.linode.com/stackscripts/ http://www.linode.com/stackscripts/ I have personally found that even distro-specific scripts require some tweaking across various hosting providers due to the default configuration of VPSs being different in each case. It has generally not been worth the effort for me to write complete bootstrap scripts so generic that they will work on anybody's VPS, but it's usually trivially easy to adapt whatever I have written to the quirks of a new hosting environment if the need arises.
- ez77 15y ago
  Thanks! Since I'm far from launching any serious work, I'm still learning with cheaper VPSes, but thanks for your tip. In particular, the term 'bootstrap scripts' will prove handy. (Similar standard terms, suitable for Googling, will be welcomed.)

nzoschke 15y ago

Nice. The state of the art here is to use AWS CloudFormation to bring up the instance. With CloudFormation you can specify the base AMI and other resources, such as an Elastic IP, and AWS will either bring up the stack or rollback the entire transaction if something goes wrong. Then, you specify the script to set up the AMI as a userdata script, and that will be run as root when the instance boots. The downside here is that the CloudFormation syntax is very funky, and 100% AWS specific. This might justify sticking with your strategy of simple fabric scripts.

bmelton 15y ago
I like dotcloud for this, but before I fell in love with dotcloud, this is exactly the sort of solution I was looking for.

ridruejo 15y ago

Not exactly the same, but BitNami provides ready-made Django AMIs http://bitnami.org/stack/djangostack http://bitnami.org/stack/djangostack

jokull 15y ago

I maintain a CLI tool that does something similar but for WSGI apps, and with a slightly different stack (uWSGI and daemontools). It uses libcloud to create instances so rackspace and linode are also supported among others. https://github.com/jokull/kraftwerk https://github.com/jokull/kraftwerk

ronnix 15y ago

Cool. Fabric rocks! People may also want to check out Ian Bicking's Silver Lining tool: http://cloudsilverlining.org/ http://cloudsilverlining.org/

alonswartz 15y ago

A little late to the conversation, but none the less... You could use the TurnKey Hub for easy deployment of a pre-configured Django appliance to Amazon EC2. If you prefer the command line you can use the Hub's API or CLI, for example: $ hub-launch django [-options] If you've enabled TKLBAM (encrypted backups of changes to files, databases, package management state, even users and groups), you can launch a new cloud server which will be automatically restored upon boot: $ hub-launch $BACKUP_ID Take that a step further and say you're developing your Django app in a local VM. When you're ready to deploy to the cloud you just trigger a backup, then launch it in the cloud.