6 ms·
You list three items here: “- library developers can easily see the impact their changes will have on their consumers - library changes that introduce regress
by mlthoughts2018 8y ago
You list three items here:
“- library developers can easily see the impact their changes will have on their consumers
- library changes that introduce regressions in their consumers can be caught pre-merge given good test coverage
- dependency version updates between packages cause less mayhem because they are performed atomically and only merged when green”
But I think these are actually signs of big failure modes of monorepos, and each one has analog for solving it with polyrepos and/or versioned artifacts that’s actually much safer and more practical.
Firstly, wishing to see how library changes will cause issues in downstream consumers is often a sign of very deep problems, because the library designer should be free to make changes or enhancements as they deem necessary to solve the problems they need to solve, which may require validating new changes don’t introduce unexpected problems for consumers, but, critically, also may not involve that, and in fact the usual case should be that it doesn’t involve that.
Instead, it’s up to consumers to consume versioned artifacts of the library (more on this in a moment), so that consumers are totally in control of “opting in” to new changes. If the consumer doesn’t want to opt in, they should not be forced to (e.g. in monorepos where a successful merge of library X is a silent, de facto upgrade for all consumers required to consume library X from the same commit, etc.)
Instead, the consumer should be the one creating an experimental branch with the intended version upgrade, and running it through the consumer’s validation tests to see if opting to upgrade the version will work.
If it fails, they can open a bug report, and the library maintainer will decide if that regression is now necessary because of a constraint in the new version’s design, or if it can or should be patched.
To be clear, this all should be happening inside a monorepo or outside of one, truly doesn’t matter. The point is to get rid of the horrible practice of forcing de facto software upgrades on one section of the code merely via a successful merge in a another section. That idea is terrible in concept, a very misguided thing that shouldn’t be desired.
Instead, when you merge library changes in one part of the monorepo, CI or other tooling should produce a fixed artifact from it, like a Python wheel, jar file, Docker container, shared object binaries, whatever, and automatically upload to an artifact store like artifactory, with versioned identifiers.
The code in some other section of the monorepo then can just happily keep plugging along, not blindsided by new changes, hit by errors uncaught in testing, etc., and the maintainer of that code can plan for their version upgrade according to their own timeline and testing.
In particular, this is what actually reduces “mayhem” as you put it in dependency upgrades, because the standard of doing it implicitly when there is a green merge is just a form of putting your head in the sand and acting like the library author can trust green merge status as a reliable indicator that downstream consumers can auto-accept new changes, when really only those downstream consumer team devs actually know if that’s true or desirable.
- aidenn0 8y ago> Firstly, wishing to see how library changes will cause issues in downstream consumers is often a sign of very deep problems, because the library designer should be free to make changes or enhancements as they deem necessary to solve the problems they need to solve, which may require validating new changes don’t introduce unexpected problems for consumers, but, critically, also may not involve that, and in fact the usual case should be that it doesn’t involve that. ... > In particular, this is what actually reduces “mayhem” as you put it in dependency upgrades, because the standard of doing it implicitly when there is a green merge is just a form of putting your head in the sand and acting like the library author can trust green merge status as a reliable indicator that downstream consumers can auto-accept new changes, when really only those downstream consumer team devs actually know if that’s true or desirable. The only reason for a library's existence is to help solve problems for consumers. I would say that if the usual case does not involve worrying about issues for consumers then whether or not you choose polyrepo or monorepo is the least of your issues.
- mlthoughts2018 8y ago> “The only reason for a library's existence is to help solve problems for consumers.“ I agree and this is the whole line of thinking behind my comment. In order to help consumers with what they need to solve, I might be required to break backward compatibility, delete old code to reduce complexity, refactor/redesign in ways that require consumers to change usage patterns, etc. If I cannot do these necessary things because every change I make is beholden to every regression test for every consumer for every legacy behavior, then it means I am failing to help my consumers, and the library is failing to give them what’s needed. Instead, if they can isolate their dependence on the library away from possibly breaking changes needed to reach new solutions in the newest version, then the library is still healthy and capable of solving consumer requirements.
- Shish2k 8y ago> I might be required to break backward compatibility > ... > I cannot do these necessary things because every change I make is beholden to every regression test for every consumer Given that backward-incompatible changes are going to be a fact of life, then at some point, somebody will have to deal with them. We can either write one commit which deals with the library and the consumers at the same time; or we can modify the library in isolation and 3 months later the consumer tries to upgrade and is surprised to find out that the API has changed and the author of that change got hit by a bus. It's the same amount of code either way, but I find small incremental atomic changes are easier to manage than large batches of non-atomic changes.