3 ms·
It's worth noting that Elastic Beanstalk just released support for multiple docker containers per instance. This takes EBS from being a novelty to being product
by philip1209 12y ago
It's worth noting that Elastic Beanstalk just released support for multiple docker containers per instance. This takes EBS from being a novelty to being production-grade Docker orchestration. Previously, it ran a single docker container per instance which - while good for stage/prod parity and easy rollbacks - doesn't really take advantage of the core Docker advantages over a VM. Now, you can orchestrate multiple containers per host, including nice-to-haves like zero-downtime deploys, automatic load balancing, monitoring, and log aggregation.
We're using EBS at StaffJoy and in the process of switching to multiple containers per EBS host. This means that a staging environment can have cheap redundancy (e.g. for zero-downtime deploys) by utilizing the same EC2 hosts, but also that queue workers (sporadic, processor-intensive scheduling operations) can split a processor with the more memory-intensive web app.
Getting EBS to work with the private docker registry was a bit of a pain because documentation was lacking - if you're having trouble with this, feel free to ping me. I'm considering a blog post on our deployment system, including easy rollbacks.
The problems we haven't figured out yet on EBS are streaming logs from hosts to another service (e.g. loggly/logentries/papertrail for realtime production error monitoring) and monitoring deploys (specifically to send a Slack notification that changes are fully deployed and to flush the Cloudflare cache).
(Note - here EBS = elastic beanstalk, not elastic block storage)
Edit: A final note - we ended up doing a Jenkins install instead of using CircleCI for build promotion purposes. We build a container, push to the docker registry, then deploy to a stage. We have a separate job that lets us deploy to prod. However, there's no rebuilding - it's just a button that takes a build number and tells EBS to deploy that existing build to the master cluster. For rollbacks - you just enter a prior build number and a prior build immediately starts rolling back We couldn't replicate this stage->master promotion/rollback in CircleCI.
Edit2: We chose to build the image ourself and register with the private registry instead of having AWS build it because we run our unit tests within the built container, so we can guarantee that the image that passes tests is the exact same as the one deployed to production. There was some concern that the image hash could differ - e.g. due to a pip download failure - if AWS built the image separately.
- coenhyde 12y agoYou should do a blog post. I had no idea Elastic Beanstalk allows multiple docker containers per instance. This changes EVERYTHING! I would love to hear how you got this all tied together.
- philip1209 12y agoHere's the announcement - it's a little over a week old: https://aws.amazon.com/about-aws/whats-new/2015/03/aws-elastic-beanstalk-supports-multi-container-docker-environments/ https://aws.amazon.com/about-aws/whats-new/2015/03/aws-elast...
- rdoherty 12y agoI think he meant a blog post about your entire architecture and build/deployment would be great. And I agree :)
- kudu 12y ago> (Note - here EBS = elastic beanstalk, not elastic block storage) I, really, really don't recommend using EBS as an abbreviation for Elastic Beanstalk, since Amazon uses the actual abbreviation canonically to refer to Elastic Block Storage. At worst, just call it Beanstalk, when the AWS context is clear.
- jdub 12y agoThe AWS abbreviation for Elastic Beanstalk is EB.
- philip1209 12y agoWe use beanstalkd queue too (SQS gets expensive at scale!), so "beanstalk" is unfortunately also equivocal in our infrastructure. I think it's just bad naming for AWS Elastic Beanstalk overall.