3 ms·
> The next deploy took our service down. How would multi-repo change this? A dependency updated, and code broke, and the new version was broken—but you update
by klodolph 4y ago
> The next deploy took our service down.
How would multi-repo change this? A dependency updated, and code broke, and the new version was broken—but you update dependencies in multi-repo anyway, and deployments can be broken anyway. I don’t see how multi-repo mitigates this.
> It encourages poor API contracts because it lets anyone import any code in any service arbitrarily.
This has nothing at all to do with monorepos. Google’s own software is built with a tool called Bazel, and Meta has something similar called Buck. These tools let you build the same kind of fine-grained boundaries that you would expect from packaged libraries. In fact, I’d say that the boundaries and API contracts are better when you use tools like Bazel or Buck—instead of just being stuck with something like a private/public distinction, you basically have the freedom to define ACLs on your packages. This is often way too much power for common use cases but it is nice to have it around when you need it, and it’s very easy to work with.
A common way to use this—suppose you have a service. The service code is private, you can’t depend on it. The client library is public, you can import it. The client library may have some internal code which has an ACL so it can only be imported from the client library front-end.
Here’s how we updated services—first add new functionality to the service. Then make the corresponding changes to the client. Finally, push any changes downstream. The service may have to work with multiple versions of the client library at any time, so you have to test with old client libraries. But we also have a “build horizon”—binaries older than some threshold, like 90 days or 180 days or something, are not permitted in production. Because of the build horizon, we know that we only have to support versions of the client library made within the last 90 or 180 days or whatever.
This is for services with “thick clients”—you could cut out the client library and just make RPCs directly, if that was appropriate for your service.
> It encourages a ton of code churn with very low signal.
The places I worked at that had monorepos, you might filter out the automated code changes there to do automated migrations to new APIs. One PR per week sounds pretty manageable, when spread across a team.
Then again, I’ve also worked at places where I had a high meeting load, and barely enough time to get my work done, so maybe one PR per week is burdensome if your are scheduled to death in meetings.
- lopkeny12ko 4y ago> How would multi-repo change this? A dependency updated, and code broke, and the new version was broken—but you update dependencies in multi-repo anyway, and deployments can be broken anyway. I don’t see how multi-repo mitigates this. In a multi-repo world, I control the repo for my own service. For a business-critical service in maintenance mode (with no active feature development), there's no reason for me to upgrade the dependencies. Code changes are the #1 cause of incidents; why fix something that isn't broken? We would have avoided this problem had we not migrated to the monorepo simply because, well, we would have never pulled in the dependency upgrade in the first place. > In fact, I’d say that the boundaries and API contracts are better when you use tools like Bazel or Buck I'm familiar with both of these tools, and I agree with this point. However, you are making an implicit assumption that 1. the monorepo in question is built with a tool like Bazel that can enforce code visibility, and 2. that there exists a team or group of volunteers to maintain such a build system across the entire repo. I suspect both of these are not true for the vast majority of codebases outside of FAANG. > The places I worked at that had monorepos, you might filter out the automated code changes there to do automated migrations to new APIs Sure, this solves a logistical problem, but not the underlying technical problem of low-signal PRs. I would argue that doing this is an antipattern because it desensitizes service owners from reviewing PRs.
- brianwawok 4y agoNot updating old libraries is how you end up getting known security vulns years after they are patched.
- lopkeny12ko 4y agoYou should ask your colleagues who work in critical industries like banking and healthcare how much of their software stack depends on things that haven't been patched in more than 20 years ;)
- precommunicator 4y agoHealthcare developer here, developing for German market, we've used Java preview features and unstable React versions many times before. And we literally have two different roles on our team for upgrading vulnerable dependencies whenever we get an alert.