4 ms·

1. package your python code as a sdist (except setuptools is too hairy so nevermind), keep your deployment scripts & configs in a separate repo or orphan branch

1. package your python code as a sdist (except setuptools is too hairy so nevermind), keep your deployment scripts & configs in a separate repo or orphan branch 3. use nginx to reverse-proxy WSGI to Gunicorn over a unix socket 6. don't have per-environment config in your app, use the same config in all environments and configure each host with aliases/proxies to consume the appropriate resources (use local_settings.py for local development on your workstation & exclude it from your package) 7. use native operating system service management (systemd, upstart, launchd) 10. use munin if your app can withstand a CPU spike every 5 minutes, use ganglia/graphite/collectd otherwise

glenjamin 14y ago
I have to disagree strongly with your point 6. I'm working on a large app at the moment where all hostnames are the same regardless of environment, and the only way to switch envs is via a proxy. It's a massive pain. Keep environments mostly similar, but always make sure you can configure locations of the app itself and its various external resources separately on staging, test and development platforms.
- falsedan 14y ago
  > I'm working on a large app at the moment where all hostnames are the same regardless of environment, and the only way to switch envs is via a proxy. It's a massive pain. One of our projects works like this, it sucks. For per-host app location, we set X-HTTP-Script-Name in the nginx location block and extracted it in WSGI middleware like http://flask.pocoo.org/snippets/35/ http://flask.pocoo.org/snippets/35/ Everything else is hardcoded to use 'http://localhost/service_name/foo http://localhost/service_name/foo & nginx proxies to the appropriate resource. Obviously, this doesn't work for non-HTTP services!
hcarvalhoalves 14y ago
> 1. package your python code as a sdist (except setuptools is too hairy so nevermind), keep your deployment scripts & configs in a separate repo or orphan branch This. I'm tired of Django projects not being packaged/released correctly, and deploy scripts that simply git clone in production (yiikes).
- hoov 14y ago
  I'd take it one step further. Why do they have to be packaged as sdists at all? Why not bdist_egg?
  - hcarvalhoalves 14y ago
    bdist works too, but it adds one more level of annoyance (you need the same environment as the target). sdist is a good balance because you get a pip-installable package, so your deploy can just consist of creating a virtualenv in /opt/<your project> and installing a determined version of your project there.
    - hoov 14y ago
      I was under the impression that Django projects couldn't be deployed when they were bdists (because Django doesn't use pkg_resources). I agree the annoyance wrt needing the same environment as the target. We tend to have two platforms that are supported: CentOS 5.x and MacOS. Keeping the build machine on the same platform as the deployment machines is simple. Creating the eggs for MacOS developers is more difficult, but still not too bad. That might seem odd, since we could just use pypi.python.org, but we have an internal PyPi server so that we can easily share internal libraries. Adding a line to a project's setup.cfg makes this trivial for the application developers. There's another annoyance with sdists. I don't want to compile during deployments. So, I build everything that can possibly be built as an egg as one, and fall back to sdist for everything else. I push those to the internal PyPi server. At deploy time, I create a virtualenv and easy_install the appropriate artifacts. I know the correct artifacts because I `pip freeze` the requirements at build time. We're also extra paranoid, so our stage and production VPCs are on different AWS accounts. We have one PyPi server per VPC and flow artifacts forward as needed.
- subleq 14y ago
  Could you go into some more detail about what being 'packaged correctly' means? What's wrong with a git clone deployment, why is deploying as a package better, and how does one implement it correctly?

glenjamin 14y ago

I have to disagree strongly with your point 6. I'm working on a large app at the moment where all hostnames are the same regardless of environment, and the only way to switch envs is via a proxy. It's a massive pain. Keep environments mostly similar, but always make sure you can configure locations of the app itself and its various external resources separately on staging, test and development platforms.

falsedan 14y ago
> I'm working on a large app at the moment where all hostnames are the same regardless of environment, and the only way to switch envs is via a proxy. It's a massive pain. One of our projects works like this, it sucks. For per-host app location, we set X-HTTP-Script-Name in the nginx location block and extracted it in WSGI middleware like http://flask.pocoo.org/snippets/35/ http://flask.pocoo.org/snippets/35/ Everything else is hardcoded to use 'http://localhost/service_name/foo http://localhost/service_name/foo & nginx proxies to the appropriate resource. Obviously, this doesn't work for non-HTTP services!

hcarvalhoalves 14y ago

> 1. package your python code as a sdist (except setuptools is too hairy so nevermind), keep your deployment scripts & configs in a separate repo or orphan branch This. I'm tired of Django projects not being packaged/released correctly, and deploy scripts that simply git clone in production (yiikes).

hoov 14y ago
I'd take it one step further. Why do they have to be packaged as sdists at all? Why not bdist_egg?
- hcarvalhoalves 14y ago
  bdist works too, but it adds one more level of annoyance (you need the same environment as the target). sdist is a good balance because you get a pip-installable package, so your deploy can just consist of creating a virtualenv in /opt/<your project> and installing a determined version of your project there.
  - hoov 14y ago
    I was under the impression that Django projects couldn't be deployed when they were bdists (because Django doesn't use pkg_resources). I agree the annoyance wrt needing the same environment as the target. We tend to have two platforms that are supported: CentOS 5.x and MacOS. Keeping the build machine on the same platform as the deployment machines is simple. Creating the eggs for MacOS developers is more difficult, but still not too bad. That might seem odd, since we could just use pypi.python.org, but we have an internal PyPi server so that we can easily share internal libraries. Adding a line to a project's setup.cfg makes this trivial for the application developers. There's another annoyance with sdists. I don't want to compile during deployments. So, I build everything that can possibly be built as an egg as one, and fall back to sdist for everything else. I push those to the internal PyPi server. At deploy time, I create a virtualenv and easy_install the appropriate artifacts. I know the correct artifacts because I `pip freeze` the requirements at build time. We're also extra paranoid, so our stage and production VPCs are on different AWS accounts. We have one PyPi server per VPC and flow artifacts forward as needed.
subleq 14y ago
Could you go into some more detail about what being 'packaged correctly' means? What's wrong with a git clone deployment, why is deploying as a package better, and how does one implement it correctly?