4 ms·
At Notion we run our GitHub Actions jobs on ECS, and use auto-scaling to add and remove hosts from the ECS cluster as demand fluctuates throughout the day. We
by jitl 3y ago
At Notion we run our GitHub Actions jobs on ECS, and use auto-scaling to add and remove hosts from the ECS cluster as demand fluctuates throughout the day. We also age out and terminate hosts although they usually live for a few days to a week. I guess we had to pay some one time setup costs around configuring the ECS cluster and fiddling runner tags, but it seems to work pretty well. We have our own cache action although it’s not as fancy as depot’s, just a tarball in s3.
Overall it’s pretty simple terraform setup plus a couple dockerfiles. And we get to run in the same region as the rest of our infra that’s close to most of our devs (us-west-2).
ECS might sound more complicated than “just use ec2” but we don’t have to screw around with lambdas and the terraform is pretty simple, much simpler then the Philips-labs one. It’s about 1400 lines of Terraform across 2 files since ECS has so much stuff built in and integrates with auto scale groups well.
- liamfd 3y agoThis sounds like a good setup - if you don't mind me asking, what do you use for your auto scaling metric? Also curious how much y'all isolate it from your other infra. I've thought about this but I've been torn on whether I'd set up a separate vpc for it.
- vicio 3y agoThanks for sharing - Do you have any blogpost or YouTube video where you go deep into the details of the implementation? that would be a good one
- throwaway384638 3y agoCan you post the TF file?