2 ms·
can't access the article, but I'm big on testability. We used to have to watch each change like a hawk via monitoring and manual checks. We automated with SLO m
by sethammons 3y ago
can't access the article, but I'm big on testability. We used to have to watch each change like a hawk via monitoring and manual checks. We automated with SLO monitoring for prod and heavy use of gated builds protected behind integration and acceptance tests.
My favorite system designed and used so far:
PR -> gated integration build with docker-compose -> merge master -> async master/main re-run same test to prevent merge regression -> promote to staging -> semi-optional user test suite -> rolling deploy that will revert if error thresholds trip. This could deploy out to thousands of nodes in like 10 minutes from merge, though sometimes it would be closer to 20.
The whole team had huge confidence that anything deployed would not immediately explode. Unlocked a lot of velocity. Each team could do multiple deploys per day.
- mgbmgb 3y agoThis is the way to go. With the recent push towards "more nimble teams" and high output / high velocity there is no place for QA teams to take days to review the releases, so you'd have to bake testability in. Recent example how fairly small team in Meta built the Threads app https://engineering.fb.com/2023/09/07/culture/threads-inside-story-metas-newest-social-app/ https://engineering.fb.com/2023/09/07/culture/threads-inside...