4 ms·

Cheers! I’m curious about this aspect that you mentioned: > forcing a library provider to maintain familiarity with all their consumers' codebases and busines

Cheers! I’m curious about this aspect that you mentioned: > forcing a library provider to maintain familiarity with all their consumers' codebases and business areas In my experience, this hasn’t been an issue—the library authors are inherently intimately familiar with how to accomplish transparent upgrades of any of their library’s API changes in a consuming codebase. Or if there are breaking API changes that make it impossible/infeasible to maintain the current consumer behavior, that’s a very useful signal that it’s time for the library producer to go back to the drawing board because the proposed library update has issues—before it’s “too late” to rethink because the library update was formally released :) Or at least, helps trigger a conversation between the producer and consumers to see if the breakage can be accommodated without too much heartburn, or if it’s a non-starter and a pun upstream solution that works for both parties needs to be discussed/implemented. Overall, it really reduces the “library producer in their ivory tower releases changes without fully considering the downstream impacts, causing a crapload of burden/schedule slip for XX teams” issue. And when you multiply that by N library producers in a company of a given scale, it’s easy for app/service dev teams to have a lot of their time non-productively taken up just trying to keep up with the constant treadmill of poorly handled upstream changes. So removing the externality effect by having the producers have some “skin in the game” can really help improve the experience for the consuming teams. hth

scubbo 3y ago
Helpful indeed, thanks! > I’m curious about this aspect that you mentioned: >> forcing a library provider to maintain familiarity with all their consumers' codebases and business areas > In my experience, this hasn’t been an issue—the library authors are inherently intimately familiar with how to [make changes] in a consuming codebase (See my comments below where I'm not sure what a "transparent upgrade" means) Wow - that's genuinely surprising (and impressive!) to me. This implies that, for all N consumers of the library, the library team are already familiar (without having to look it up - otherwise, that's added ramp-up burden) with the consumers' preferred code style, testing expectations, commit and branching structure, personal development environment setup, running integration tests against personal environments, how to announce PRs (if at all), and so on - all the "other stuff" that goes "around" making the actual code change. In an ideal world, a lot of those things _should_ be trivially clear/provided (code-style provided by auto-linters, testing carried out automatically during PR, etc.) - but practically speaking they very rarely are. This seems like a _lot_ of extra knowledge which is (in my opinion - though clearly not in a monorepo-mindset opinion) outside of their area of ownership and expertise that they have to carry around on a day-to-day basis, on the off-chance that they make a breaking change to their library (which, as I think we both agree, should hopefully be rare). Even in the best-reasonable case, where all of those things are well-documented (and they _should_ be, because an outside contributor is indistinguishable from a New Hire), that's still a fair bit of reading, ramp-up, and workspace-setup that the library team needs to do - vs. simply telling the consuming team "here's the general shape of change that you need to make, apply this as appropriate to your own situation - ask us if anything's unclear" OK, on to the direct reply: --- So, to be clear about terminology, I think we're only really concerned with breaking changes, here - i.e. those for which, if the consumer moves from consuming v-<previous> to v-<new>, they will also need to change something about how their code calls the library. The most obvious ways I can think of for this to be necessary are: * The type signature changes in a non-backwards compatible way - i.e. adding required parameters (including making previously-optional parameters required), changing the type of a parameter, or changing the type of the return value to something incompatible with the previous type ("incompatible" is hand-waving because different type systems think about this differently, but I think my meaning is clear in general) * The name of the method itself is changed, or the entire call pattern is changed (e.g. instead of `foo.execute(command)`, you now have to call `foo.prepareCommand(command); foo.execute()`) For any library change which _isn't_ a breaking change (i.e. one for which the consumers could keep their call-site code exactly the same, and expect the same behaviour) I believe this whole discussion is moot - both perspectives would agree that nothing would need to be done in the consuming code. My perspective would say that the consuming team has nothing to do, and the monorepo perspective would say that the library team has nothing to do - but, either way, `0 === 0`, even JavaScript agrees on that :) So, then, I might need a bit of elaboration on what a "transparent upgrade" is, since my initial naïve interpretation (a response to a non-breaking change - one in which the call-site code can remain the same) cannot be the thing at issue, since - well, yeah, of course a library team knows how to make a no-op change to their consumers ;) might "transparent upgrade" instead mean "a change where all the data required to make the library call is available at the call-site, but it needs to be reshaped, retyped, or otherwise reorganized before being passed to the library function in v-<new>"? If so, then...yeah, I can see it being _nice_ if the library owners were to publish some deterministic code-diff tool which would transform all their consumers' call-sites to use the newly-reshuffled data, but honestly I'd expect them instead to do that "reshuffling" _internally_ to their function and not even publish the new "breaking but with reorganized data" version in the first place. Maybe there's a good case for this "data-reshuffling" that I'm missing, though. Maybe a pseudocode example of a "transparent upgrade" would help me understand what it is? --- > if there are breaking API changes that make it impossible/infeasible to maintain the current consumer behavior, that’s a very useful signal that it’s time for the library producer to go back to the drawing board because the proposed library update has issues[...]Or at least, helps trigger a conversation between the producer and consumers to see if the breakage can be accommodated without too much heartburn Yeah, I think we're generally on the same page, here; just assigning differing weights, priorities, and likelihoods to breaking changes. Wherever possible, for sure, aim for non-breaking changes; and where breaking changes are necessary, make sure you have had a full discussion with consumers to see whether the churn you're going to create is justified (and maybe consider providing a long period of support for the previous version so that consumers have a long period to upgrade and to amortize that upgrade-churn) - but, if a breaking change _is_ truly justified (is worth more than the upgrade-burden it will cost), don't shy away from it! > Overall, it really reduces the “library producer in their ivory tower releases changes without fully considering the downstream impacts, causing a crapload of burden/schedule slip for XX teams” issue. Maybe I've just been very lucky to have barely run into that problem! :) In ~10 years in my previous role, I'm struggling to think of a handful of times where a major version upgrade was any more than a trivial process - and the few times that it _was_ a meatier change, I was perfectly content with being given a guide or walkthrough on how to migrate. But, yeah - if cavalier library teams making profligate and unjustified breaking changes _is_ a problem being faced, then I can see that that culture of "you break it, you fix it" would rein them in. --- FWIW, this has _already_ really helped me to understand where a coworker was coming from in an (unrelated) proposal which, to me and all other colleagues, initially looked nonsensical. After chewing on it a bit, I realized he's coming at this from the same perspective - "people who make changes to things that other people depend on, should bear the pain of updating those other people's things to work with the new dependency-thing". Gotta say I am still extremely-unconvinced (I'm still on the side of "ceteris paribus, an owner making a change is more efficient than mandating that someone else make the change; so I'd prefer to find other ways of forcing library owners to confront the externalities of disruptive breaking changes that don't introduce that inefficiency"), but it does very much help to understand where he's coming from - thanks!

scubbo 3y ago

Helpful indeed, thanks! > I’m curious about this aspect that you mentioned: >> forcing a library provider to maintain familiarity with all their consumers' codebases and business areas > In my experience, this hasn’t been an issue—the library authors are inherently intimately familiar with how to [make changes] in a consuming codebase (See my comments below where I'm not sure what a "transparent upgrade" means) Wow - that's genuinely surprising (and impressive!) to me. This implies that, for all N consumers of the library, the library team are already familiar (without having to look it up - otherwise, that's added ramp-up burden) with the consumers' preferred code style, testing expectations, commit and branching structure, personal development environment setup, running integration tests against personal environments, how to announce PRs (if at all), and so on - all the "other stuff" that goes "around" making the actual code change. In an ideal world, a lot of those things _should_ be trivially clear/provided (code-style provided by auto-linters, testing carried out automatically during PR, etc.) - but practically speaking they very rarely are. This seems like a _lot_ of extra knowledge which is (in my opinion - though clearly not in a monorepo-mindset opinion) outside of their area of ownership and expertise that they have to carry around on a day-to-day basis, on the off-chance that they make a breaking change to their library (which, as I think we both agree, should hopefully be rare). Even in the best-reasonable case, where all of those things are well-documented (and they _should_ be, because an outside contributor is indistinguishable from a New Hire), that's still a fair bit of reading, ramp-up, and workspace-setup that the library team needs to do - vs. simply telling the consuming team "here's the general shape of change that you need to make, apply this as appropriate to your own situation - ask us if anything's unclear" OK, on to the direct reply: --- So, to be clear about terminology, I think we're only really concerned with breaking changes, here - i.e. those for which, if the consumer moves from consuming v-<previous> to v-<new>, they will also need to change something about how their code calls the library. The most obvious ways I can think of for this to be necessary are: * The type signature changes in a non-backwards compatible way - i.e. adding required parameters (including making previously-optional parameters required), changing the type of a parameter, or changing the type of the return value to something incompatible with the previous type ("incompatible" is hand-waving because different type systems think about this differently, but I think my meaning is clear in general) * The name of the method itself is changed, or the entire call pattern is changed (e.g. instead of `foo.execute(command)`, you now have to call `foo.prepareCommand(command); foo.execute()`) For any library change which _isn't_ a breaking change (i.e. one for which the consumers could keep their call-site code exactly the same, and expect the same behaviour) I believe this whole discussion is moot - both perspectives would agree that nothing would need to be done in the consuming code. My perspective would say that the consuming team has nothing to do, and the monorepo perspective would say that the library team has nothing to do - but, either way, `0 === 0`, even JavaScript agrees on that :) So, then, I might need a bit of elaboration on what a "transparent upgrade" is, since my initial naïve interpretation (a response to a non-breaking change - one in which the call-site code can remain the same) cannot be the thing at issue, since - well, yeah, of course a library team knows how to make a no-op change to their consumers ;) might "transparent upgrade" instead mean "a change where all the data required to make the library call is available at the call-site, but it needs to be reshaped, retyped, or otherwise reorganized before being passed to the library function in v-<new>"? If so, then...yeah, I can see it being _nice_ if the library owners were to publish some deterministic code-diff tool which would transform all their consumers' call-sites to use the newly-reshuffled data, but honestly I'd expect them instead to do that "reshuffling" _internally_ to their function and not even publish the new "breaking but with reorganized data" version in the first place. Maybe there's a good case for this "data-reshuffling" that I'm missing, though. Maybe a pseudocode example of a "transparent upgrade" would help me understand what it is? --- > if there are breaking API changes that make it impossible/infeasible to maintain the current consumer behavior, that’s a very useful signal that it’s time for the library producer to go back to the drawing board because the proposed library update has issues[...]Or at least, helps trigger a conversation between the producer and consumers to see if the breakage can be accommodated without too much heartburn Yeah, I think we're generally on the same page, here; just assigning differing weights, priorities, and likelihoods to breaking changes. Wherever possible, for sure, aim for non-breaking changes; and where breaking changes are necessary, make sure you have had a full discussion with consumers to see whether the churn you're going to create is justified (and maybe consider providing a long period of support for the previous version so that consumers have a long period to upgrade and to amortize that upgrade-churn) - but, if a breaking change _is_ truly justified (is worth more than the upgrade-burden it will cost), don't shy away from it! > Overall, it really reduces the “library producer in their ivory tower releases changes without fully considering the downstream impacts, causing a crapload of burden/schedule slip for XX teams” issue. Maybe I've just been very lucky to have barely run into that problem! :) In ~10 years in my previous role, I'm struggling to think of a handful of times where a major version upgrade was any more than a trivial process - and the few times that it _was_ a meatier change, I was perfectly content with being given a guide or walkthrough on how to migrate. But, yeah - if cavalier library teams making profligate and unjustified breaking changes _is_ a problem being faced, then I can see that that culture of "you break it, you fix it" would rein them in. --- FWIW, this has _already_ really helped me to understand where a coworker was coming from in an (unrelated) proposal which, to me and all other colleagues, initially looked nonsensical. After chewing on it a bit, I realized he's coming at this from the same perspective - "people who make changes to things that other people depend on, should bear the pain of updating those other people's things to work with the new dependency-thing". Gotta say I am still extremely-unconvinced (I'm still on the side of "ceteris paribus, an owner making a change is more efficient than mandating that someone else make the change; so I'd prefer to find other ways of forcing library owners to confront the externalities of disruptive breaking changes that don't introduce that inefficiency"), but it does very much help to understand where he's coming from - thanks!