5 ms·

I'll repost my question from the blog here (And thanks again Lyst team!): Really enjoyed the write-up, Lyst team! Thanks for the info. As a member of another

I'll repost my question from the blog here (And thanks again Lyst team!): Really enjoyed the write-up, Lyst team! Thanks for the info. As a member of another large-ish Python dev shop with a number of dependencies, one of things that have kept us from moving fully in the direction of Docker is total deployment time. Since Docker is a 1-process/1-container model, our dependency sizes can be large since we run a number of processes on a single host. This means that our network into a newly launched host would be theoretically very large. How did Lyst deal with that? Or was the flexibility and slight slowness just preferable to something like shipping pre-built AMIs?

boffbowsh 12y ago
We (hi, I work at Lyst too!) see Docker images as fairly similar to pre-built AMIs conceptually, just a lot more lightweight. We bake our Docker host AMIs relatively regularly and use them in the ASG that runs the Dockerized website. Building AMIs (and booting new instances from them on each deploy) takes a lot longer than building a new Docker image and launching containers from it. I'm not entirely sure I understand what you're asking though. We've so far only Dockerized the main Python application, and not many of the supporting applications. We don't think we'd go down the route of running Postgres, ElasticSearch, Redis etc under Docker. To give you an example of times taken, Pull Requests take about 10 minutes to test, a full master build (including uploading build artifacts) takes around 15 minutes, and a deployment to 8 Dockerized web hosts is about 5 minutes, including a smoke test on each host. EDIT: Just realised what you're asking. If you're running many processes on each host then yes it can take a while to deploy. We deploy our Celery workers under separate Docker containers and this can take a while to stop and start. For the website though we run uWSGIs in forked mode, so it's only one container per host. This will probably change in the future, we're evaluating Mesos + Marathon for a more PaaS-like deployment where we can say "14 web processes please!" Hope that answers your question.
- fndrplayer13 12y ago
  Thanks so much for the answer!

boffbowsh 12y ago

We (hi, I work at Lyst too!) see Docker images as fairly similar to pre-built AMIs conceptually, just a lot more lightweight. We bake our Docker host AMIs relatively regularly and use them in the ASG that runs the Dockerized website. Building AMIs (and booting new instances from them on each deploy) takes a lot longer than building a new Docker image and launching containers from it. I'm not entirely sure I understand what you're asking though. We've so far only Dockerized the main Python application, and not many of the supporting applications. We don't think we'd go down the route of running Postgres, ElasticSearch, Redis etc under Docker. To give you an example of times taken, Pull Requests take about 10 minutes to test, a full master build (including uploading build artifacts) takes around 15 minutes, and a deployment to 8 Dockerized web hosts is about 5 minutes, including a smoke test on each host. EDIT: Just realised what you're asking. If you're running many processes on each host then yes it can take a while to deploy. We deploy our Celery workers under separate Docker containers and this can take a while to stop and start. For the website though we run uWSGIs in forked mode, so it's only one container per host. This will probably change in the future, we're evaluating Mesos + Marathon for a more PaaS-like deployment where we can say "14 web processes please!" Hope that answers your question.

fndrplayer13 12y ago
Thanks so much for the answer!