5 ms·

Some more tips to maintain your sanity once you are past the beginner level: * Use South before you start deploying * Use an automated deployment tool like Fa

by arocks 14y ago

Some more tips to maintain your sanity once you are past the beginner level: * Use South before you start deploying * Use an automated deployment tool like Fabric or more powerful ones like Chef/Puppet * Get a better debugger and other niceties with django-command-extensions Of course, a lot more apps can be considered must have depending on what you are working on.

hdra 14y ago
I have been trying to get a grasp on your second point, but to this day, I still haven't been able to really come to terms with it. Is there any total noobs guide to automated deployment? Everything I have seen to this day assumes some knowledge of sysops and the whole ecosystem of that specific deployment tool (e.g. like it assumes I knew what is a 'cookbook', or where to find them, etc). It would be great if there is a guide for people like me that doesn't assume any knowledge from the reader aside from development / basic local deployment.
- raverbashing 14y ago
  Try fabric It's a lightweight management tool, like writing a shell script in python, but easier and targeted towards system deployment, really you only have to get around its unintuitive calling convention (not inside the script - it's the way you use the cmd line tool) but otherwise it's a breeze Chef/Puppet are harder (as in, you really have to fit their paradigms)
  - hdra 14y ago
    I haven't really looked into it, but how does that differ from using say, a shell script file?
    - deleted 14y ago
      [deleted]
    - DrJ 14y ago
      I might be saying this incorrectly, but... with Fabric you are writing shell scripts in python, so you can have people who doesn't know how to read complex(?) shell scripts be able to read/write/modify your deploy scripts.
    - btipling 14y ago
      It's similar to using a shell script except you're using Python and have access to all of Python's modules and fabric is written with roles and remote management in mind so it makes tasks easier. I love it and even use it to deploy my jekyl blog.
    - pekk 14y ago
      It makes it really easy to target ssh hosts and tell them to do things remotely, while also doing things locally, in one or more scripts. Writing the functionality of Fabric in shell would take you a long time and give you something which was buggy and inflexible. Someone else wrote it in Python for you. There's no good reason to write it in shell.
  - reinhardt 14y ago
    This doesn't really answer the grandparent's question. "Try fabric" is good advice for someone that knows (a) what he wants to do and (b) how to do it using a bunch of shell scripts, but prefers Python instead. That's quite far from a "total noob" starting point.
- helfire 14y ago
  This article[1] is what made chef 'click' for me. Try it out with vagrant or just solo on a vm to start. Try to forget everything you already learned and just do this example the build off that. [1] http://www.opinionatedprogrammer.com/2011/06/chef-solo-tutorial-managing-a-single-server-with-chef/ http://www.opinionatedprogrammer.com/2011/06/chef-solo-tutor...
- arocks 14y ago
  I think I should explain my point a little more. Chef and Puppet are a different category of tools compared to Fabric. It is comparable to the difference between a Makefile and a shell script. Chef/Puppet are like Makefiles, it would rebuild the targets if they don't exist. If it is already in the desired state, running the script performs nothing. This is not the case with Fabric. Fabric scripts would usually always perform the required action (unless you have written checks explicitly). Fabric is a low hanging fruit so I would suggest starting there until you really have the need to manage multiple servers and then use Chef/Puppet.
  - St-Clock 14y ago
    I would have a hard time recommending Chef and Puppet unless you wrap your deployed unit in a .deb or .rpm file with pre and post install scripts. Puppet is not really great at executing a sequence of steps because it is declarative: you can always express sequence as dependencies (e.g., reload nginx when file X has changed), but for fine grained steps (e.g., create a softlink for the latest version of your code), it's less convenient. It's also slower than a Fabric script. If I want to revert to a previous version, I just do "fab revert myapp". With puppet I would have to rerun the puppet agent/change a manifest/change the version in my rpm repository/... We heavily use puppet to manage our servers, but we rely on Fabric for deployment tasks.
- mkhattab 14y ago
  > Is there any total noobs guide to automated deployment? Everything I have seen to this day assumes some knowledge of sysops and the whole ecosystem of that specific deployment tool (e.g. like it assumes I knew what is a 'cookbook', or where to find them, etc). You need to have some knowledge of sysops to use automated deployment tools. I guess it's why they were created in the first place, i.e. not for beginners but for those with enough knowledge/skill to automate tedious sys admin tasks. Perhaps the best solution for you is to use a PaaS like Heroku, EngineYard, Google App Engine, etc.
  - hdra 14y ago
    I am currently using Heroku for live apps, and it is indeed simple enough to use without having to do all these things, but one thing that I intend to use is to automate my VMs setup. I am currently using Vagrant to automate some system configuration such as security keys and port forwarding, but I still install all my packages manually. I am not looking to be so proficient that I can handle a production server, but I do wish that I can automate some of my development workflow.
- beambot 14y ago
  For the true beginner, it's tough to beat Heroku. Their documentation on how to deploy is pretty sound. It's as simple as a "git push" to deploy your application. Once you're comfortable with that, it's not too bad to slowly move stuff over to EC2 using your own tools (eg. Fabric).
  - hdra 14y ago
    That is indeed how I work (play) today. So, using these automated deployment tools have little benefit in a PAAS ecosystem?
    - beambot 14y ago
      That's my understanding. Basically... these tools give you more fine-grained control at the expense of configuration & maintenance burdens. Classic PAAS tradeoffs.
- decadentcactus 14y ago
  Try out fabric, start really, really, really small. I'll assume you use Git (if not, replace with your workflow). A super simple fabric script might simply be a `git push`. Instead of typing "git push origin master" in order to push to github/bitbucket/wherever, you'd run "fab deploy". Not going to go into the fabric specifics, but deploy() would pretty much just call git push. Anyway, that may have confused you even more. My point is, start with something incredibly simple. After you've added git push, you can add to your fabfile.py to ssh into the server, and run git pull. Then add in something to run south migrations. Then collectstatic. Eventually you'll build to something that you can simply run, and everything will be updated and all good.
- e12e 14y ago
  If you're working with python, I would recommend looking at buildout first. The best part of working with plone lately has been learning bootstrap. For working with Django and buildoout see eg: http://www.slideshare.net/zerok/django-buildout-en-presentation http://www.slideshare.net/zerok/django-buildout-en-presentat... http://www.sophomoredev.com/2012/06/how-i-use-django-virtualenv-and-buildout-together/ http://www.sophomoredev.com/2012/06/how-i-use-django-virtual... As for managing deployments and general devops, I am currently looking at ansible.
btipling 14y ago
I'd say fabric is more powerful than Chef or Puppet. Fabric basically lets you do anything and Chef and Puppet are constrained to do certain things in certain ways. Not sure why that would be considered more powerful. Also I think Chef and Puppet require production servers to do things other than production things, like check for updates. I'd rather just have production servers do production things and not unintentionally update or do things when someone didn't intentionally instruct the server to do those things (as part of an automated process of course).
- pekk 14y ago
  Fabric does let you do pretty much anything. But what it actually does is limited to certain basic tools. Tools like Chef and Puppet don't let you do just anything, but they also do a lot more. Some of the things they do would be pretty insane to do on top of Fabric.
pessimism 14y ago
Even at the beginner level, Travis CI is great. Granted, it requires knowledge of git. It’s a really effortless way of automated testing, and you can add more and more specificity to it, as you develop your project. And the merge validation is invaluable. (Assuming you can get it to work, which hasn’t always been a cake walk with weird, inexplicable failures every now and then.)
mladenkovacevic 14y ago
I can't speak for Chef/Puppet but I tried in the past to incorporate Fabric into my workflow but I couldn't find a fabfile that worked out of the box with my setup so I found myself having to learn the specifics of all the deployment steps - initially for the purpose of creating my own Fabric setup. Once I got through to the other side I realized that my process really isn't that complicated to begin with, and now I really appreciate knowing exactly how my website works. Just for the record I run a single blog using Django, nginx, uwsgi and supervisord on Webfaction. I use git and virtualenvwrapper to run development on local computers and staging and production environments on webfaction. Once I get into using the same setup for many sites, I believe I will appreciate solutions like fabric or chef, but for right now I can't stress enough the value of writing your own conf files, syncing and migrating your model changes, learning to use git, and stopping and restarting processes manually. TL;DR While Fabric is a great tool to help you simplify your process, I wouldn't recommend it to beginners of Django. Instead make a few passes at going through deployment manually and establish a process so you can write your own fabric configurations later on.
- mkhattab 14y ago
  > I can't speak for Chef/Puppet but I tried in the past to incorporate Fabric into my workflow but I couldn't find a fabfile that worked out of the box [...] You should think of a fabfile as a glorified shell script (or Makefile). So, instead of writing your own shell script that uploads your assets to your remote box, restarts Apache, etc. and then adjusting your script to run on multiple boxes, you can just write a simple Fabric task or two that does the same thing (with minimal configuration). Chef/Puppet are completely different tools than Fabric. In fact, the two are complementary[1]. I totally recommend Fabric to beginners because it builds on what you should know, i.e. how to write simple shell scripts, and it makes life easier. [1] http://www.youtube.com/watch?v=sdkAXM36C7M http://www.youtube.com/watch?v=sdkAXM36C7M (DevOps for Python: Fabric + Chef)
- StavrosK 14y ago
  Don't use Fabric, Fabric isn't for provisioning. Use Salt/Ansible (my current favorite)/Chef/Puppet/etc.

by arocks 14y ago

hdra 14y ago

I have been trying to get a grasp on your second point, but to this day, I still haven't been able to really come to terms with it. Is there any total noobs guide to automated deployment? Everything I have seen to this day assumes some knowledge of sysops and the whole ecosystem of that specific deployment tool (e.g. like it assumes I knew what is a 'cookbook', or where to find them, etc). It would be great if there is a guide for people like me that doesn't assume any knowledge from the reader aside from development / basic local deployment.

raverbashing 14y ago
Try fabric It's a lightweight management tool, like writing a shell script in python, but easier and targeted towards system deployment, really you only have to get around its unintuitive calling convention (not inside the script - it's the way you use the cmd line tool) but otherwise it's a breeze Chef/Puppet are harder (as in, you really have to fit their paradigms)
- hdra 14y ago
  I haven't really looked into it, but how does that differ from using say, a shell script file?
  - deleted 14y ago
    [deleted]
  - DrJ 14y ago
    I might be saying this incorrectly, but... with Fabric you are writing shell scripts in python, so you can have people who doesn't know how to read complex(?) shell scripts be able to read/write/modify your deploy scripts.
  - btipling 14y ago
    It's similar to using a shell script except you're using Python and have access to all of Python's modules and fabric is written with roles and remote management in mind so it makes tasks easier. I love it and even use it to deploy my jekyl blog.
  - pekk 14y ago
    It makes it really easy to target ssh hosts and tell them to do things remotely, while also doing things locally, in one or more scripts. Writing the functionality of Fabric in shell would take you a long time and give you something which was buggy and inflexible. Someone else wrote it in Python for you. There's no good reason to write it in shell.
- reinhardt 14y ago
  This doesn't really answer the grandparent's question. "Try fabric" is good advice for someone that knows (a) what he wants to do and (b) how to do it using a bunch of shell scripts, but prefers Python instead. That's quite far from a "total noob" starting point.
helfire 14y ago
This article[1] is what made chef 'click' for me. Try it out with vagrant or just solo on a vm to start. Try to forget everything you already learned and just do this example the build off that. [1] http://www.opinionatedprogrammer.com/2011/06/chef-solo-tutorial-managing-a-single-server-with-chef/ http://www.opinionatedprogrammer.com/2011/06/chef-solo-tutor...
arocks 14y ago
I think I should explain my point a little more. Chef and Puppet are a different category of tools compared to Fabric. It is comparable to the difference between a Makefile and a shell script. Chef/Puppet are like Makefiles, it would rebuild the targets if they don't exist. If it is already in the desired state, running the script performs nothing. This is not the case with Fabric. Fabric scripts would usually always perform the required action (unless you have written checks explicitly). Fabric is a low hanging fruit so I would suggest starting there until you really have the need to manage multiple servers and then use Chef/Puppet.
- St-Clock 14y ago
  I would have a hard time recommending Chef and Puppet unless you wrap your deployed unit in a .deb or .rpm file with pre and post install scripts. Puppet is not really great at executing a sequence of steps because it is declarative: you can always express sequence as dependencies (e.g., reload nginx when file X has changed), but for fine grained steps (e.g., create a softlink for the latest version of your code), it's less convenient. It's also slower than a Fabric script. If I want to revert to a previous version, I just do "fab revert myapp". With puppet I would have to rerun the puppet agent/change a manifest/change the version in my rpm repository/... We heavily use puppet to manage our servers, but we rely on Fabric for deployment tasks.
mkhattab 14y ago
> Is there any total noobs guide to automated deployment? Everything I have seen to this day assumes some knowledge of sysops and the whole ecosystem of that specific deployment tool (e.g. like it assumes I knew what is a 'cookbook', or where to find them, etc). You need to have some knowledge of sysops to use automated deployment tools. I guess it's why they were created in the first place, i.e. not for beginners but for those with enough knowledge/skill to automate tedious sys admin tasks. Perhaps the best solution for you is to use a PaaS like Heroku, EngineYard, Google App Engine, etc.
- hdra 14y ago
  I am currently using Heroku for live apps, and it is indeed simple enough to use without having to do all these things, but one thing that I intend to use is to automate my VMs setup. I am currently using Vagrant to automate some system configuration such as security keys and port forwarding, but I still install all my packages manually. I am not looking to be so proficient that I can handle a production server, but I do wish that I can automate some of my development workflow.
beambot 14y ago
For the true beginner, it's tough to beat Heroku. Their documentation on how to deploy is pretty sound. It's as simple as a "git push" to deploy your application. Once you're comfortable with that, it's not too bad to slowly move stuff over to EC2 using your own tools (eg. Fabric).
- hdra 14y ago
  That is indeed how I work (play) today. So, using these automated deployment tools have little benefit in a PAAS ecosystem?
  - beambot 14y ago
    That's my understanding. Basically... these tools give you more fine-grained control at the expense of configuration & maintenance burdens. Classic PAAS tradeoffs.
decadentcactus 14y ago
Try out fabric, start really, really, really small. I'll assume you use Git (if not, replace with your workflow). A super simple fabric script might simply be a `git push`. Instead of typing "git push origin master" in order to push to github/bitbucket/wherever, you'd run "fab deploy". Not going to go into the fabric specifics, but deploy() would pretty much just call git push. Anyway, that may have confused you even more. My point is, start with something incredibly simple. After you've added git push, you can add to your fabfile.py to ssh into the server, and run git pull. Then add in something to run south migrations. Then collectstatic. Eventually you'll build to something that you can simply run, and everything will be updated and all good.
e12e 14y ago
If you're working with python, I would recommend looking at buildout first. The best part of working with plone lately has been learning bootstrap. For working with Django and buildoout see eg: http://www.slideshare.net/zerok/django-buildout-en-presentation http://www.slideshare.net/zerok/django-buildout-en-presentat... http://www.sophomoredev.com/2012/06/how-i-use-django-virtualenv-and-buildout-together/ http://www.sophomoredev.com/2012/06/how-i-use-django-virtual... As for managing deployments and general devops, I am currently looking at ansible.

btipling 14y ago

I'd say fabric is more powerful than Chef or Puppet. Fabric basically lets you do anything and Chef and Puppet are constrained to do certain things in certain ways. Not sure why that would be considered more powerful. Also I think Chef and Puppet require production servers to do things other than production things, like check for updates. I'd rather just have production servers do production things and not unintentionally update or do things when someone didn't intentionally instruct the server to do those things (as part of an automated process of course).

pekk 14y ago
Fabric does let you do pretty much anything. But what it actually does is limited to certain basic tools. Tools like Chef and Puppet don't let you do just anything, but they also do a lot more. Some of the things they do would be pretty insane to do on top of Fabric.

pessimism 14y ago

Even at the beginner level, Travis CI is great. Granted, it requires knowledge of git. It’s a really effortless way of automated testing, and you can add more and more specificity to it, as you develop your project. And the merge validation is invaluable. (Assuming you can get it to work, which hasn’t always been a cake walk with weird, inexplicable failures every now and then.)

mladenkovacevic 14y ago

I can't speak for Chef/Puppet but I tried in the past to incorporate Fabric into my workflow but I couldn't find a fabfile that worked out of the box with my setup so I found myself having to learn the specifics of all the deployment steps - initially for the purpose of creating my own Fabric setup. Once I got through to the other side I realized that my process really isn't that complicated to begin with, and now I really appreciate knowing exactly how my website works. Just for the record I run a single blog using Django, nginx, uwsgi and supervisord on Webfaction. I use git and virtualenvwrapper to run development on local computers and staging and production environments on webfaction. Once I get into using the same setup for many sites, I believe I will appreciate solutions like fabric or chef, but for right now I can't stress enough the value of writing your own conf files, syncing and migrating your model changes, learning to use git, and stopping and restarting processes manually. TL;DR While Fabric is a great tool to help you simplify your process, I wouldn't recommend it to beginners of Django. Instead make a few passes at going through deployment manually and establish a process so you can write your own fabric configurations later on.

mkhattab 14y ago
> I can't speak for Chef/Puppet but I tried in the past to incorporate Fabric into my workflow but I couldn't find a fabfile that worked out of the box [...] You should think of a fabfile as a glorified shell script (or Makefile). So, instead of writing your own shell script that uploads your assets to your remote box, restarts Apache, etc. and then adjusting your script to run on multiple boxes, you can just write a simple Fabric task or two that does the same thing (with minimal configuration). Chef/Puppet are completely different tools than Fabric. In fact, the two are complementary[1]. I totally recommend Fabric to beginners because it builds on what you should know, i.e. how to write simple shell scripts, and it makes life easier. [1] http://www.youtube.com/watch?v=sdkAXM36C7M http://www.youtube.com/watch?v=sdkAXM36C7M (DevOps for Python: Fabric + Chef)
StavrosK 14y ago
Don't use Fabric, Fabric isn't for provisioning. Use Salt/Ansible (my current favorite)/Chef/Puppet/etc.