4 ms·

Hey former Guardian employee here. The Guardian has hundreds of servers running, pretty much all EC2 instances. EC2 images are baked and derived from official

Hey former Guardian employee here. The Guardian has hundreds of servers running, pretty much all EC2 instances. EC2 images are baked and derived from official images, similarly to the way you bake a docker image. We built tools before docker became the de facto standard, so we could easily keep the EC2 images up to date. We integrated pretty well with AWS so that the basic constructs of autoscaling and load balancer were well understood by everyone. The stack is mostly JVM based so the benefits of running docker locally weren't really significant. We've evaluated moving to a docker solution a few times and always reached the conclusion that the cost of doing so wouldn't be worth the benefits. Now for a company that starts today I don't think I'd recommend that, it just so happen that The Guardian invested early on the right tooling so that's pretty much an exception.

sparsely 5y ago
> We integrated pretty well with AWS so that the basic constructs of autoscaling and load balancer were well understood by everyone. This is an underappreciated point I think sometimes. Once you have a team which is familiar with your current, working setup, the benefits of moving away have to be pretty huge for it to be worthwhile.
rr808 5y ago
> Now for a company that starts today I don't think I'd recommend that Any reason why? It sounds pretty good.
weego 5y ago
Also ex employee. Riff Raff is absolutely still an excellent mod for build and deploy. At the time I was there it the initial stack build via handwritten cloudformation script that was the friction and pain point.
- alex_duf 5y ago
  Honestly I miss riff-raff :D For anyone who doesn't know about it: https://github.com/guardian/riff-raff https://github.com/guardian/riff-raff
speleding 5y ago
> Now for a company that starts today I don't think I'd recommend that I think for a company starting out that AMIs (Amazon Machine Images), which from your description is probably what you're using, is actually a much better way to go than docker containers, because you get a large part of the orchestration for free with the AWS EC2 auto-scaling and health detection without most of docker complexity. (I would suggest using something like Terraform to set it up in a reproducible way though)
bscanlan 5y ago
Intercom is pretty similar. We use EC2 hosts and no containers (other than for development/test environments and some niche third-party software that is distributed as Docker containers). Autoscaling groups are our unit of scalability, pretty much one per workload, and we treat the EC2 hosts as immutable cattle. We do a scheduled AMI build every week and replace every host. We use an internally developed software tool to deploy buildpacks to hosts - buildpacks are pre-Docker technology from Heroku that solves most of the problems containers do. I wouldn't necessarily recommend building this from scratch today, it was largely put in place around 8 years ago, and there are few compelling reasons for us to switch.

sparsely 5y ago

> We integrated pretty well with AWS so that the basic constructs of autoscaling and load balancer were well understood by everyone. This is an underappreciated point I think sometimes. Once you have a team which is familiar with your current, working setup, the benefits of moving away have to be pretty huge for it to be worthwhile.

rr808 5y ago

> Now for a company that starts today I don't think I'd recommend that Any reason why? It sounds pretty good.

weego 5y ago

Also ex employee. Riff Raff is absolutely still an excellent mod for build and deploy. At the time I was there it the initial stack build via handwritten cloudformation script that was the friction and pain point.

alex_duf 5y ago
Honestly I miss riff-raff :D For anyone who doesn't know about it: https://github.com/guardian/riff-raff https://github.com/guardian/riff-raff

speleding 5y ago

> Now for a company that starts today I don't think I'd recommend that I think for a company starting out that AMIs (Amazon Machine Images), which from your description is probably what you're using, is actually a much better way to go than docker containers, because you get a large part of the orchestration for free with the AWS EC2 auto-scaling and health detection without most of docker complexity. (I would suggest using something like Terraform to set it up in a reproducible way though)

bscanlan 5y ago

Intercom is pretty similar. We use EC2 hosts and no containers (other than for development/test environments and some niche third-party software that is distributed as Docker containers). Autoscaling groups are our unit of scalability, pretty much one per workload, and we treat the EC2 hosts as immutable cattle. We do a scheduled AMI build every week and replace every host. We use an internally developed software tool to deploy buildpacks to hosts - buildpacks are pre-Docker technology from Heroku that solves most of the problems containers do. I wouldn't necessarily recommend building this from scratch today, it was largely put in place around 8 years ago, and there are few compelling reasons for us to switch.