4 ms·
Getting the time from “everyone agrees the change looks good” to “it’s in front of users” down is so satisfying. Work that goes into reducing this tends to have
by rtpg 3y ago
Getting the time from “everyone agrees the change looks good” to “it’s in front of users” down is so satisfying. Work that goes into reducing this tends to have wonderful knock on effects all over, from faster builds, quicker rollbacks, smaller CI bills (yes it’s possible!). The biggest difficulty I’ve seen in practice is going from “somebody hits the button” to “thing happens”.
But even just making sure that “somebody hits the button” is a 1 button thing instead of a 20 button thing can be a major win! Totally valuable work
On the other side of the coin: having good observability into prod (talking mainly about APM stuff and tracing) gives devs way more confidence in the changes they’re making as well. Make it easy to deploy stuff, but also make it easy for systems to log things, and make it easy for developers to poke into those logs (my favorite flavor of this right now is honeycomb, but mainly cuz you can do perf analysis and answer questions from PMs about usage quite easily)
The nice thing about all of this if you’re a mid sized team is that you can totally outsource this with the right kind of person. Somebody who comes in, has the right to write some automation code (maybe it’s Zapier even!), and is not part of office infighting about process. The mandate being “make this pipeline smoother”. Probably among the easier problems to fix compared to deeper business or product problems. And it’s also super visible to other teams (“wow the bug is already fixed?”)
EDIT: one thing I’ve seen done that “never works” (I have not seen everything) is splitting up a repo into smaller repos. If you find yourself thinking “with smaller repos things could be smoother” it’s likely that what you want is not multiple repos but a build tool like Bazel or Buck, that will let you describe multiple projects in one repo, without causing inter-repo friction come review time. Integrating these tools can save you loads of money on CI and enforce decent test suite separation. The unfortunate flip side is that at least Bazel is very idiosyncratic, and most other tools are based on Bazel. I have heard good things about Buck 2 but have yet to see it in prod
- kridsdale1 3y agoAt Apple, we did indeed use many small repos. Some were even in different SCM systems! It was indeed a nightmare to integrate with “everything on main is broken, hold off on pulling” being a common multi day experience. At Google we use Blaze which is Bazel and it works very well, as you say. At Meta (where I was before Google) we used Buck2 and it was the best. Much faster and more laptop-friendly than Blaze. It was like Christmas when they switched us over from Buck1. Meta is very very good at “20,000 contributors to this repo”. Google is about 85% as good. Getting work done (especially on mobile apps that take 7 hours to build from scratch) at that scale without these tools would make me jump off the building.