3 ms·

I think this is roughly true, but at a BigCo it’s not really feasible/easy unless you have a monorepo or otherwise extremely good build/integration tooling to d

• scubbo 3y ago
  > it’s not really feasible/easy unless you have a monorepo or otherwise extremely good build/integration tooling to deal with many repos[...] The issue is coordinating changes [...] across repos is often an utter nightmare with multiple PR/merge builds I'm coming up on a year out of $BIG_TECH_JOB (where the idea of a monorepo was horrifying) and transferred to $WAY_SMALLER_NON_TECH_COMPANY_WHO_USES_TECH_JOB (where the enthusiastically use a monorepo), and have been really confused by repeated claims like this. I really feel like I'm missing something here, because lots of obviously-smart-and-experienced folks repeat it. I'd really appreciate it if you could check my understanding and see what I'm missing. (To be clear, here I'm assuming that a monorepo is a single repository which contains conceptually-distinct-but-related projects - things which _could_ justifiably be their own repos, but which are kept in one repo for reasons of maintenance and managements - and wherein the build system is such that every sub-project within the repo uses the same package dependency tree, i.e. if ProjectA and ProjectB in the monorepo depend on LibraryZ, then the versions of Z that A and B depend on must be identical for any given commit/build. If I've misunderstood that - if that's just straight-up not what a monorepo is, or if it _is_ but with some extra nuance or sauce - then I guess we can short-circuit the response pretty quickly :P ) Here's how a release of a breaking change of a library would work in a polyrepo world: * I publish v2.0.0 of my library * Consumers of that library are notified that a new version exists (via automated email notification, Dependabot, whatever) \ * Anyone who _wants_ to update can do so (independently and at their own rate) - anyone who's comfortable staying on the old version can do so ** If we really want to make things easy for consumers, we can make automated PRs against their repos (this would be the "extremely good build/integration tooling to deal with many repos" you refer to, I suspect? Something like Spotify's FleetShift[0]) to make the change - though in practice this is probably way more trouble than its worth, I've only seen it done for serious security vulnerabilities where a) everyone in the whole damn company has to b) upgrade RIGHT THE FUCK NOW. * Time goes by * v1.x gets deprecated * Anyone still using v1.x gets notified that they are using a deprecated version and strongly encouraged to update * A little more time goes by (not much!) * Consequences Occur for people still using the old version - this could be a visit from your friendly InfoSec enforcer, or automatically failing builds, or the removal of v1.x from the package repository (which will indirectly cause failing builds), or... Conversely, with a monorepo, the situation seems to be: * I publish v2.0.0 of my library * Every single team whose code is in the monorepo must coordinate to make the changes to consume version 2. This change, by definition, proceeds at the pace of the slowest team - if one of them is underwater on oncall, or has their only competent engineer on PTO, or has some quirk of implementation (or dependency on old feature) which means they can't update for two months, then _the whole dang monorepo_ is staying on version 1 for two months; no ifs, ands, or buts ** But, yes, if you want to a wide-ranging refactoring to change all the in-monorepo consumers to use the new method, then yeah, modern IDEs are going to have _some_ functionality built-in for that within a single repo. But - by virtue of being a breaking change, it's pretty likely that the change-at-call-site is going to be more complex than simply changing the type or name of the method being called. Maybe the returned object needs different methods called on it. Maybe the method requires an extra parameter (which refactoring IDEs can add to the actual callsite, but cannot implement for you _fetching and providing_ that parameter). At this point, either humans are going to have to comb through the changes to finalize them (at which point, you lose the claimed advantages of being in a monorepo where changes can be done by tooling), or you're going to have to implement some sort of code-parsing system to make correct changes throughout (which, again, is approximately equivalent to the automation work required in the polyrepo case) So: while, yes, it _would_ be a hassle to "coordinate changes across repos" (though _significantly_ less to coordinate across a single repo-to-repo boundary, especially if both are owned by the same team, than it would to coordinate a whole monorepo's worth of interactions), the joy of a polyrepo situation is that _you don't have to_. Consumers can update when they want to, at their own pace - no coordination necessary! So - yes, there will be a greater _volume_ of PRs required in a polyrepo situation, but a) each of them will be way simpler and the total volume of work will be , b) they are independent (so teams who don't want to update do not hold back those who do). But - people keep saying "it's easier to make wide-ranging changes in a monorepo", so I _must_ be missing something. What is it? [0] https://engineering.atspotify.com/2023/05/fleet-management-at-spotify-part-3-fleet-wide-refactoring/ https://engineering.atspotify.com/2023/05/fleet-management-a...
  • evntdrvn 3y ago
    What I’ve heard from friends working at places where the monorepo model has worked well, is that it also involved a culture shift such that the burden of upgrading the consuming services is put on the shoulders of the _providers_ of the library, rather than the consumers. This implicitly brings along some benefits, like making the providers not cause excessive/unnecessary version churn and ensuring easy upgrade paths, because they experience the pain rather than it being externalized onto the consumers :) Also, good tooling helps—and I think that this model also encourages investing in that.
    • scubbo 3y ago
      Wow, that is certainly a _huge_ culture shift. It certainly explains why it's not sitting right with me - while I do think that "providers not caus[ing] excessive/unnecessary version churn and ensuring easy upgrade paths" is a good thing in isolation, it seems like it would be massively outweighed by the downsides of forcing a library provider to maintain familiarity with all their consumers' codebases and business areas. Seems to me like that would make it impractical to make any changes to a library that is consumed by more than a few other teams - which runs totally contrary to the intention of extracting _commonly_-depended-upon logic. The ideal situation would be to create a library that is _so_ popular that maintaining that level of familiarity is literally impossible. There are already incentives in place to ensure that a library provider is doing right by their consumers (building features that they want, making upgrades not too arduous), because if they make a hard-to-use library then people won't use it which should show up in however their team's success is judged. Seems to me that going the extra mile from "you have to provide a good desirable library" to "and you must also be familiar enough with your consumers' services to do the integration/upgrade work _for_ them" gives no new upside but all downside - service teams _already_ know their domain area, why should someone else have to? (I recognize that you're just reporting what you've heard, I'm not arguing _with you_, I'm trying to reason out the arguments in my head to get them straight) Really appreciate the insight, thank you!
      • evntdrvn 3y ago
        Cheers! I’m curious about this aspect that you mentioned: > forcing a library provider to maintain familiarity with all their consumers' codebases and business areas In my experience, this hasn’t been an issue—the library authors are inherently intimately familiar with how to accomplish transparent upgrades of any of their library’s API changes in a consuming codebase. Or if there are breaking API changes that make it impossible/infeasible to maintain the current consumer behavior, that’s a very useful signal that it’s time for the library producer to go back to the drawing board because the proposed library update has issues—before it’s “too late” to rethink because the library update was formally released :) Or at least, helps trigger a conversation between the producer and consumers to see if the breakage can be accommodated without too much heartburn, or if it’s a non-starter and a pun upstream solution that works for both parties needs to be discussed/implemented. Overall, it really reduces the “library producer in their ivory tower releases changes without fully considering the downstream impacts, causing a crapload of burden/schedule slip for XX teams” issue. And when you multiply that by N library producers in a company of a given scale, it’s easy for app/service dev teams to have a lot of their time non-productively taken up just trying to keep up with the constant treadmill of poorly handled upstream changes. So removing the externality effect by having the producers have some “skin in the game” can really help improve the experience for the consuming teams. hth