5 ms·
We have a stateless worker process that previously required a separate EC2 instance for each worker. Even using small instances, this meant a pretty cumbersome
by BrianEatWorld 12y ago
We have a stateless worker process that previously required a separate EC2 instance for each worker. Even using small instances, this meant a pretty cumbersome fleet in AWS with lags for spin up and excess costs due to workers that were brought online and finished their jobs well before the use hour was complete.
Using Docker, we can have several of these workers on a single box with near instantaneous spin up. This means that we are able to use fewer, larger instances instead of several small ones. In turn, this makes the fleet easier to manage, quicker to scale and less costly because we aren't over paying for large portions of AWS hours that go under utilized.
I am not entirely sure that Docker was a necessity in building this as I sort of inherited the technology. I originally was pushing for a switch to pure LXC, which would have fit the build system that was in place better. However, given the fervour over Docker there is a lot of information out on the web and so changing the build and deployment systems has been relatively easy and quick. I bring this up because I think some tasks are better suited to pure LXC, but people seem to be defaulting to Docker due to its popularity.