11 ms·

This might be unpopular, but I don't think libraries should ever break backwards compatibility, at least not with enough regularity to require all of this. Sur

This might be unpopular, but I don't think libraries should ever break backwards compatibility, at least not with enough regularity to require all of this. Sure if there is a major security thing, or something outside of your control then you might have to. But if you just want to change a function's signature, or return value, because you think it will help "clean things up" could you please just make a new function, or make it work both ways if your language allows it. I understand the desire to tear out a messy bit of code and make it beautiful, but it always leads to a bunch of headaches for little gain.

Mathnerd314 5y ago
There's no way to entirely avoid breaking compatibility. If you allow small API breakages then at least all of the team is aware of the possibility and designs their processes to handle backwards incompatible changes. If you design your processes assuming backwards compatibility then you'll run into the 1-in-1000 case where it's not possible and have to stop everything to handle the major update.
- jnxx 5y ago
  > There's no way to entirely avoid breaking compatibility. Why?
  - masklinn 5y ago
    Because you've got no idea how your code's being used in the wild an one person's bug is an other's feature. So any observable change in behaviour is a possibly breaking change. And then people go and muck with your internals even if you protected them, because that's the only way they found to do what they needed to do, and they were much more interested in doing the thing than following your guidelines. And that's before even mentioning broken features which requires changes which will invariably break other stuff. The only way to know for sure you're never breaking compatibility is to never release any update. Otherwise you're playing the odds.
    - dmoy 5y ago
      https://www.hyrumslaw.com/ https://www.hyrumslaw.com/ Also more facetiously, xkcd/1172
      - jnxx 5y ago
        That might be a real issue for people which strive for a high level of compatibility, such as engineers at Google, but I don't think it applies to less than 0.1% of compatibility-breaking changes. Also, such things are fixable if they are important - just provide the old behavior by default and add a flag or keyword argument which selects the updated behavior. And it also helps to make clear that the documented behavior is what you promise, nothing else. I think API breakages are usually just caused by developers which failed to define consistent and thought-out interfaces from the start, and library developers which do not want to do any extra work to continue to provide APIs with the old semantics. I know that it can be made to work, because I have worked in industrial automation and PLC libraries, and while there are not always and everywhere geniuses at work, in this field legacy interfaces are just never scrapped because they need to continue to work. What you publish needs to continue to work for 20+ years, because nobody wants to scrap a six-figure printing machine just because of an update to the PLC libraries. So, it does not require geniuses to do that. It is more a matter of what the consumers of a library tolerate. And the same is valid for language evolution. Common Lisp for example, or C are extremely stable. Python users have come to tolerate a far higher amount of breakage and instability, but I do not buy that this is in any way technically required. The breakage might not matter that much if the code is only used in fast-moving start-ups of which 99% will be bust anyway in four years time. But there are domains where the costs of this are just too high, such as science projects, industrial automation, enterprise systems, and many more, because systems are too complex and expensive to update.
      - jnxx 5y ago
        Also, Hyrums law are about unforeseen breakage because people did not relay only on the documented API specification. This is undesirable, and in fact hard to avoid completely. But most breaking changes such as in semantic versioning major number changes do intentionally break client code, which is an entirely different thing. There are a few rules one can follow which make it pretty improbable that such foreseeable breakage will happen. And this will constitute more than 99.9% of API breakage - even more if you stipulate that undocumented features must not be relied upon, and client developers follow that rule. Developers relying on undocumented behavior is usually due to a lack of clear documentation, so it is largely avoidable, too.
- dec0dedab0de 5y ago
  There's no way to entirely avoid breaking compatibility. If you allow small API breakages then at least all of the team is aware of the possibility and designs their processes to handle backwards incompatible changes. If you design your processes assuming backwards compatibility then you'll run into the 1-in-1000 case where it's not possible and have to stop everything to handle the major update. Are you saying that it is better to have to deal with a bunch of tiny changes, so that when you have to deal with a big change you'll be used to it? If so, I strongly disagree. I would much rather have to deal with a big annoying change every 10 years, than a small annoying change every 6 months (which would still likely need the big one every 10 years).
ebiester 5y ago
/orders /orders2 /orders3 /orders4 /neworders /getordersbutonlythistype
- earthboundkid 5y ago
  If you're changing the endpoint that same amount but keeping the URL, you've just imposed a ton of churn on everyone downstream. It's a tradeoff, like everything. Try to get the API right the first time. Try to avoid introducing new APIs. Try to avoid breaking the old API. There's no silver bullet.
  - ebiester 5y ago
    After all, your business never changes and neither do the constraints.
    - earthboundkid 5y ago
      Who are you arguing with?
- underwater 5y ago
  And /orders uses an old data layer in a way that is incompatible with the new one, so the team can't deprecate the internal DB abstractions. And they can't introduce a new caching layer without making it work across the new and old abstractions. Etc, etc.
samsquire 5y ago
I agree with you. Churn for churn's sake to make things marginally better. One man's refactoring is another man's refucktoring. Depending on a library could also insert a record on the library's CI system to indicate the dependency. The library CI will ping a URL with a version number to indicate that "you need to re-test with this version". It would certainly make software more robust and evolvable. I've thought about it before and called it library mesh: https://github.com/samsquire/ideas2#10-library-mesh https://github.com/samsquire/ideas2#10-library-mesh
zachrip 5y ago
People don't understand how their code can mess up others in unexpected ways. A few weeks ago my youtube just stopped working. Turns out, a change made to the apollo-graphql extension caused window.process to be readonly and youtube's code was not expecting that: https://github.com/apollographql/apollo-client-devtools/issues/455 https://github.com/apollographql/apollo-client-devtools/issu... I know this isn't a dependency issue, but it's a great example of why things like semver don't work perfectly in practice. It's hard to figure out what the effects of your code really are in the end. You might assume a change is minor when it really is major. I wish programming languages could figure out what the version change should be for a given code change.
- dec0dedab0de 5y ago
  Sure, but that's just a bug in an application introduced by accident, and fixed when they found out about it. That is going to happen no matter what. The same thing goes for libraries. That is the best part of this tool, since they control the library and all the clients they can automatically run their tests across everything. Which would help you catch more of these types of problems, but it will never be perfect. My point is that if a user of your library is failing after a change, you should treat that as a bug in the library. Backwards incompatible changes should only be on purpose, and only when absolutely necessary. Of course, it's only their own code they're affecting here, so it doesn't really matter as much as a library for users you don't know. Plus it is a pretty cool solution, I just don't think it should be necessary.
- jesseryoung 5y ago
  "Don't change the function signature" and semver are things that are absolutely hammered into devs who work on library code. I'd be hard pressed to find somebody even tangentially involved in software development who doesn't understand this. Semver doesn't cover changes in behavior. What a "breaking change" is when it comes to behavior changes is up to the person who implemented it.
  - jnxx 5y ago
    >Semver doesn't cover changes in behavior. It does: https://semver.org/ https://semver.org/ "Software using Semantic Versioning MUST declare a public API. This API could be declared in the code itself or exist strictly in documentation. However it is done, it SHOULD be precise and comprehensive." If the API changes semantics between different releases, it is a breaking change. It does not matter whether the types in the function signature is identical. If printReport(int invoice_number) prints a report in version 1.0, and deletes the database or launches nuclear weapons in version 1.1, it does not follow semantic versioning. But the core problem with semantic versioning is that it somehow legitimizes to make breaking changes all the time. It is not OK to beat somebody up in a bar. It is also not OK to tell somebody that he is going to be beaten up, and then beat him up. Now, semantic versioning holds the idea that it is OK to make breaking changes, if only you tell people before. And this is a lousy idea. Breaking changes break other people's code, especially if they use it in deep graphs of dependencies. And they can't necessarily do anything about it, because they might have two dependencies which are depending on your library, only that the first one now requires version 2.0 at least, while the second one will not work with anything newer than version 1.99.17. And thus the whole project is broken. Actually, requiring the new version 2.0 in the first dependency is a breaking change, too, because it breaks backwards-compatibility. And this is something which the Python devs just did not comprehend - that this kind of breakage trickles up dependency graphs and makes it extremely difficult for users with a limited budget to make upgrades, even if most of their features are highly desired.
jnxx 5y ago
I've come to the same conclusion. Infrastructure and libraries breaking backward compatibility should not happen, it is not a sign of technical quality. It is almost always possible to provide wrapper interfaces which contain the old behavior and function signature as a special case. And if that does not work, it is better to release a different library. There is also one interesting insight I had when looking at error codes and exceptions: Normally, backwards compatibility is broken when you take things away from an interface - functions from a library, values from an enumeration type, and so on (Rich Hickey has made a brilliant talk about this). However, the general thing is that backward-compatible change must never narrow pre-conditions, nor widen post-conditions. This means that it is not OK to just add extra error codes to the return values of a function interface, or to add new classes of exceptions. This breaks backward compatibility. You can of course add enumeration values, optional arguments, keyword arguments and such to function call arguments, and these are great ways to make interface changes backwards-compatible. Another insight is that backward-incompatible changes tend to cascade up larger dependency graphs. The broader and deeper the dependency graphs becomes, the more probable it is that a backward-compatible change (e.g. requiring a newer dependency) becomes itself a breaking change up the dependency chain. The philosophy in this age seems to be that of coourse the other library authors should "just fix their stuff", they get the responsibility of upgrading to the new version of the dependency. I do not agree to this at all - I think if something changes in a component, and this breaks the software, the responsibility is always within that component, and nowhere else. If some change in the component causes breakage, this is a breaking change, by definition. And this is also true if it is an upgrade from Qt4 to Qt5, or OpenCV, or imageio-py2 to imageio-py3 or such. One could increment the mayor version number but this does not fix the problem. What fixes the problem is to not break stuff in the first place. I am quite convinced that as in the future, ever more programming will be done in libraries and components, there will be ecosystems which do not accept that kind of breakage. Linux is already a good example, and it is wildly successful.
- asdfasgasdgasdg 5y ago
  I think widening post conditions may have to be possible in some cases. In my experience, programs should only depend on three possible error states: ok, eagain, and everything else. I guess it's fine for library authors to promise more, but I'd only do so in the case where I think that calling code can respond with more specificity to other conditions, which I think is pretty rare.
lumost 5y ago
At big companies this attitude eventually yields a stagnant product. No one is confidant that any impactful change wouldn’t break something and no one knows if that something even matters to customers. Even worse there may be teams that will make 0 changes as they’ve been marked as KTLO. Having tooling to acknowledge that in fact no one will be affected by a change is a huge productivity boost.
- dec0dedab0de 5y ago
  Having tooling to acknowledge that in fact no one will be affected by a change is a huge productivity boost. Yes I do agree with that part
- nradov 5y ago
  Years ago I was blown away when I saw a presentation by a Google Java library developer. He was able to make changes to his library, then automatically build every Java application throughout the enterprise which used that library to check for automated test failures. This allowed the library team to move fast with confidence.
  - wikibob 5y ago
    And you can too! http://Bazel.build http://Bazel.build
- underwater 5y ago
  KTLO = Keeping the lights on, for anyone else who hasn't heard the acronym before.
- jnxx 5y ago
  > At big companies this attitude eventually yields a stagnant product. Such as Linux?
simonw 5y ago
Expecting library authors to get their design entirely correct the first time round isn't reasonable. When libraries make backwards-incompatible changes it's usually because the original design was flawed in a way that made continuing to support it, while adding new features, infeasible. You're welcome to evaluate and select libraries with the best possible track record for avoiding backwards incompatible changes - I do that myself. But, especially for libraries that are relatively early in their development process, having a hard "no breakages" policy isn't realistic. I still think semver is the best fix for this. If you make a backwards-incompatible change, you bump the major version number and you extensively document the change. If you're consuming libraries, the best way to address this problem is with solid test coverage. That way any time there's a new library version you can lean on your test suite to help you understand if it's going to break anything for you or not.
- wahern 5y ago
  > Expecting library authors to get their design entirely correct the first time round isn't reasonable. People shouldn't be publishing such libraries, nor should people be using them. With the exception of my first few FOSS projects, all of my open source libraries represent the 2nd, 3rd, or even 4th incarnation of a concept. That's just being responsible--don't knowingly litter the ecosystem with junk. I take a similar view at work. People are too quick to break out components into modules and libraries, long before they have a firm notion of what a proper interface should look like. If you can't commit to a stable API, then often times it's better to just copy whole functions and files between projects and let them evolve naturally. It takes time to see where that evolution goes, and which commonalities emerge in the implementation and surrounding application. But what about sharing bug and security fixes, you ask? Well, without a stable API you'll invariably end up with different projects stuck using different versions, anyhow. That's why people invented Docker :( What you end up with isn't an actually shared library, just the pretense of sharing, plus a bunch of unnecessary build complexity.
  - simonw 5y ago
    > People shouldn't be publishing such libraries, nor should people be using them. That's why we have pre-1.0 version numbers (and alphas and betas) - they let us release code early for people to provide feedback before we've committed to a non-breaking API. Even with a stable 1.0 release things change. People using the library may come up with logical feature requests that the original design didn't consider, and which require backwards incompatible changes. That's when you bump to a 2.0 release. My dream project would have a version number of something like 1.237 - meaning there have been 237 new features releases since the 1.0 release without backwards compatibility breaks. My hat is off to any project that pulls that off though, it's a rare accomplishment!
failwhaleshark 5y ago
Semver is one way, but yes. It annoys me when libraries "clean up" (churn) their codebase when they should've thought more carefully about designing the interface (the contract) before publishing it. Code churn often creates more pointless work for N library users. I swear that most code churn is to make a project look "alive" or to give people something to do / job security. Deprecate slowly and have semi-automated user code fixups and notes.
m33k44 5y ago
I remember reading an article on MSDN(a developer info system - I call it a system because it was also available on CDs as internet was not available everywhere) by a Microsoft employee. The article was titled some what on the lines of "How the marketing teams won over the engineering teams!" The article mentioned a tussle within Microsoft between the marketing and the engineering teams. The engineers were pushing for providing backwards compatibility which slowed them down and took time in providing new things/features that were the next sought after tech. The marketing teams on the otherhand were very frustrated that new sought after tech was not coming out faster so that they can then market it to the world. That tugh-off war was ultimately won by the marketing teams at Microsoft as per the article.
- matheusmoreira 5y ago
  Reminds me of this article: https://www.joelonsoftware.com/2004/06/13/how-microsoft-lost-the-api-war/ https://www.joelonsoftware.com/2004/06/13/how-microsoft-lost...
renewiltord 5y ago
I actually genuinely think library authors should choose whatever trade-offs they want, and consumers should choose between those libraries based on the trade-offs they're comfortable accepting. The "always backwards compatibility" and "never backwards compatibility" schools of thought might just as well be "no programmer should ever use a garbage collector" or "all programmers should only ever use garbage collectors". Tools like OP help people who want to accept backwards incompatibility.
_carbyau_ 5y ago
I don't do modern coding with "bunch of dependencies downloaded at compile time" so now I am having fears about being "like so out of touch man". Just reply and tell me I am out of touch if I am so far off base here. If the updated library breaks your stuff then can't you just... not update? And if you want to update to the new library for XYZ new cool stuff it does, then you are already putting in effort to utilise XYZ. In which case you can sort out the new ABC fix too. It is extra work but it is part of having updated to the new coolness.

Mathnerd314 5y ago

There's no way to entirely avoid breaking compatibility. If you allow small API breakages then at least all of the team is aware of the possibility and designs their processes to handle backwards incompatible changes. If you design your processes assuming backwards compatibility then you'll run into the 1-in-1000 case where it's not possible and have to stop everything to handle the major update.

jnxx 5y ago
> There's no way to entirely avoid breaking compatibility. Why?
- masklinn 5y ago
  Because you've got no idea how your code's being used in the wild an one person's bug is an other's feature. So any observable change in behaviour is a possibly breaking change. And then people go and muck with your internals even if you protected them, because that's the only way they found to do what they needed to do, and they were much more interested in doing the thing than following your guidelines. And that's before even mentioning broken features which requires changes which will invariably break other stuff. The only way to know for sure you're never breaking compatibility is to never release any update. Otherwise you're playing the odds.
  - dmoy 5y ago
    https://www.hyrumslaw.com/ https://www.hyrumslaw.com/ Also more facetiously, xkcd/1172
    - jnxx 5y ago
      That might be a real issue for people which strive for a high level of compatibility, such as engineers at Google, but I don't think it applies to less than 0.1% of compatibility-breaking changes. Also, such things are fixable if they are important - just provide the old behavior by default and add a flag or keyword argument which selects the updated behavior. And it also helps to make clear that the documented behavior is what you promise, nothing else. I think API breakages are usually just caused by developers which failed to define consistent and thought-out interfaces from the start, and library developers which do not want to do any extra work to continue to provide APIs with the old semantics. I know that it can be made to work, because I have worked in industrial automation and PLC libraries, and while there are not always and everywhere geniuses at work, in this field legacy interfaces are just never scrapped because they need to continue to work. What you publish needs to continue to work for 20+ years, because nobody wants to scrap a six-figure printing machine just because of an update to the PLC libraries. So, it does not require geniuses to do that. It is more a matter of what the consumers of a library tolerate. And the same is valid for language evolution. Common Lisp for example, or C are extremely stable. Python users have come to tolerate a far higher amount of breakage and instability, but I do not buy that this is in any way technically required. The breakage might not matter that much if the code is only used in fast-moving start-ups of which 99% will be bust anyway in four years time. But there are domains where the costs of this are just too high, such as science projects, industrial automation, enterprise systems, and many more, because systems are too complex and expensive to update.
    - jnxx 5y ago
      Also, Hyrums law are about unforeseen breakage because people did not relay only on the documented API specification. This is undesirable, and in fact hard to avoid completely. But most breaking changes such as in semantic versioning major number changes do intentionally break client code, which is an entirely different thing. There are a few rules one can follow which make it pretty improbable that such foreseeable breakage will happen. And this will constitute more than 99.9% of API breakage - even more if you stipulate that undocumented features must not be relied upon, and client developers follow that rule. Developers relying on undocumented behavior is usually due to a lack of clear documentation, so it is largely avoidable, too.
dec0dedab0de 5y ago
There's no way to entirely avoid breaking compatibility. If you allow small API breakages then at least all of the team is aware of the possibility and designs their processes to handle backwards incompatible changes. If you design your processes assuming backwards compatibility then you'll run into the 1-in-1000 case where it's not possible and have to stop everything to handle the major update. Are you saying that it is better to have to deal with a bunch of tiny changes, so that when you have to deal with a big change you'll be used to it? If so, I strongly disagree. I would much rather have to deal with a big annoying change every 10 years, than a small annoying change every 6 months (which would still likely need the big one every 10 years).

ebiester 5y ago

/orders /orders2 /orders3 /orders4 /neworders /getordersbutonlythistype

earthboundkid 5y ago
If you're changing the endpoint that same amount but keeping the URL, you've just imposed a ton of churn on everyone downstream. It's a tradeoff, like everything. Try to get the API right the first time. Try to avoid introducing new APIs. Try to avoid breaking the old API. There's no silver bullet.
- ebiester 5y ago
  After all, your business never changes and neither do the constraints.
  - earthboundkid 5y ago
    Who are you arguing with?
underwater 5y ago
And /orders uses an old data layer in a way that is incompatible with the new one, so the team can't deprecate the internal DB abstractions. And they can't introduce a new caching layer without making it work across the new and old abstractions. Etc, etc.

samsquire 5y ago

I agree with you. Churn for churn's sake to make things marginally better. One man's refactoring is another man's refucktoring. Depending on a library could also insert a record on the library's CI system to indicate the dependency. The library CI will ping a URL with a version number to indicate that "you need to re-test with this version". It would certainly make software more robust and evolvable. I've thought about it before and called it library mesh: https://github.com/samsquire/ideas2#10-library-mesh https://github.com/samsquire/ideas2#10-library-mesh

zachrip 5y ago

People don't understand how their code can mess up others in unexpected ways. A few weeks ago my youtube just stopped working. Turns out, a change made to the apollo-graphql extension caused window.process to be readonly and youtube's code was not expecting that: https://github.com/apollographql/apollo-client-devtools/issues/455 https://github.com/apollographql/apollo-client-devtools/issu... I know this isn't a dependency issue, but it's a great example of why things like semver don't work perfectly in practice. It's hard to figure out what the effects of your code really are in the end. You might assume a change is minor when it really is major. I wish programming languages could figure out what the version change should be for a given code change.

dec0dedab0de 5y ago
Sure, but that's just a bug in an application introduced by accident, and fixed when they found out about it. That is going to happen no matter what. The same thing goes for libraries. That is the best part of this tool, since they control the library and all the clients they can automatically run their tests across everything. Which would help you catch more of these types of problems, but it will never be perfect. My point is that if a user of your library is failing after a change, you should treat that as a bug in the library. Backwards incompatible changes should only be on purpose, and only when absolutely necessary. Of course, it's only their own code they're affecting here, so it doesn't really matter as much as a library for users you don't know. Plus it is a pretty cool solution, I just don't think it should be necessary.
jesseryoung 5y ago
"Don't change the function signature" and semver are things that are absolutely hammered into devs who work on library code. I'd be hard pressed to find somebody even tangentially involved in software development who doesn't understand this. Semver doesn't cover changes in behavior. What a "breaking change" is when it comes to behavior changes is up to the person who implemented it.
- jnxx 5y ago
  >Semver doesn't cover changes in behavior. It does: https://semver.org/ https://semver.org/ "Software using Semantic Versioning MUST declare a public API. This API could be declared in the code itself or exist strictly in documentation. However it is done, it SHOULD be precise and comprehensive." If the API changes semantics between different releases, it is a breaking change. It does not matter whether the types in the function signature is identical. If printReport(int invoice_number) prints a report in version 1.0, and deletes the database or launches nuclear weapons in version 1.1, it does not follow semantic versioning. But the core problem with semantic versioning is that it somehow legitimizes to make breaking changes all the time. It is not OK to beat somebody up in a bar. It is also not OK to tell somebody that he is going to be beaten up, and then beat him up. Now, semantic versioning holds the idea that it is OK to make breaking changes, if only you tell people before. And this is a lousy idea. Breaking changes break other people's code, especially if they use it in deep graphs of dependencies. And they can't necessarily do anything about it, because they might have two dependencies which are depending on your library, only that the first one now requires version 2.0 at least, while the second one will not work with anything newer than version 1.99.17. And thus the whole project is broken. Actually, requiring the new version 2.0 in the first dependency is a breaking change, too, because it breaks backwards-compatibility. And this is something which the Python devs just did not comprehend - that this kind of breakage trickles up dependency graphs and makes it extremely difficult for users with a limited budget to make upgrades, even if most of their features are highly desired.

jnxx 5y ago

I've come to the same conclusion. Infrastructure and libraries breaking backward compatibility should not happen, it is not a sign of technical quality. It is almost always possible to provide wrapper interfaces which contain the old behavior and function signature as a special case. And if that does not work, it is better to release a different library. There is also one interesting insight I had when looking at error codes and exceptions: Normally, backwards compatibility is broken when you take things away from an interface - functions from a library, values from an enumeration type, and so on (Rich Hickey has made a brilliant talk about this). However, the general thing is that backward-compatible change must never narrow pre-conditions, nor widen post-conditions. This means that it is not OK to just add extra error codes to the return values of a function interface, or to add new classes of exceptions. This breaks backward compatibility. You can of course add enumeration values, optional arguments, keyword arguments and such to function call arguments, and these are great ways to make interface changes backwards-compatible. Another insight is that backward-incompatible changes tend to cascade up larger dependency graphs. The broader and deeper the dependency graphs becomes, the more probable it is that a backward-compatible change (e.g. requiring a newer dependency) becomes itself a breaking change up the dependency chain. The philosophy in this age seems to be that of coourse the other library authors should "just fix their stuff", they get the responsibility of upgrading to the new version of the dependency. I do not agree to this at all - I think if something changes in a component, and this breaks the software, the responsibility is always within that component, and nowhere else. If some change in the component causes breakage, this is a breaking change, by definition. And this is also true if it is an upgrade from Qt4 to Qt5, or OpenCV, or imageio-py2 to imageio-py3 or such. One could increment the mayor version number but this does not fix the problem. What fixes the problem is to not break stuff in the first place. I am quite convinced that as in the future, ever more programming will be done in libraries and components, there will be ecosystems which do not accept that kind of breakage. Linux is already a good example, and it is wildly successful.

asdfasgasdgasdg 5y ago
I think widening post conditions may have to be possible in some cases. In my experience, programs should only depend on three possible error states: ok, eagain, and everything else. I guess it's fine for library authors to promise more, but I'd only do so in the case where I think that calling code can respond with more specificity to other conditions, which I think is pretty rare.

lumost 5y ago

At big companies this attitude eventually yields a stagnant product. No one is confidant that any impactful change wouldn’t break something and no one knows if that something even matters to customers. Even worse there may be teams that will make 0 changes as they’ve been marked as KTLO. Having tooling to acknowledge that in fact no one will be affected by a change is a huge productivity boost.

dec0dedab0de 5y ago
Having tooling to acknowledge that in fact no one will be affected by a change is a huge productivity boost. Yes I do agree with that part
nradov 5y ago
Years ago I was blown away when I saw a presentation by a Google Java library developer. He was able to make changes to his library, then automatically build every Java application throughout the enterprise which used that library to check for automated test failures. This allowed the library team to move fast with confidence.
- wikibob 5y ago
  And you can too! http://Bazel.build http://Bazel.build
underwater 5y ago
KTLO = Keeping the lights on, for anyone else who hasn't heard the acronym before.
jnxx 5y ago
> At big companies this attitude eventually yields a stagnant product. Such as Linux?

simonw 5y ago

Expecting library authors to get their design entirely correct the first time round isn't reasonable. When libraries make backwards-incompatible changes it's usually because the original design was flawed in a way that made continuing to support it, while adding new features, infeasible. You're welcome to evaluate and select libraries with the best possible track record for avoiding backwards incompatible changes - I do that myself. But, especially for libraries that are relatively early in their development process, having a hard "no breakages" policy isn't realistic. I still think semver is the best fix for this. If you make a backwards-incompatible change, you bump the major version number and you extensively document the change. If you're consuming libraries, the best way to address this problem is with solid test coverage. That way any time there's a new library version you can lean on your test suite to help you understand if it's going to break anything for you or not.

wahern 5y ago
> Expecting library authors to get their design entirely correct the first time round isn't reasonable. People shouldn't be publishing such libraries, nor should people be using them. With the exception of my first few FOSS projects, all of my open source libraries represent the 2nd, 3rd, or even 4th incarnation of a concept. That's just being responsible--don't knowingly litter the ecosystem with junk. I take a similar view at work. People are too quick to break out components into modules and libraries, long before they have a firm notion of what a proper interface should look like. If you can't commit to a stable API, then often times it's better to just copy whole functions and files between projects and let them evolve naturally. It takes time to see where that evolution goes, and which commonalities emerge in the implementation and surrounding application. But what about sharing bug and security fixes, you ask? Well, without a stable API you'll invariably end up with different projects stuck using different versions, anyhow. That's why people invented Docker :( What you end up with isn't an actually shared library, just the pretense of sharing, plus a bunch of unnecessary build complexity.
- simonw 5y ago
  > People shouldn't be publishing such libraries, nor should people be using them. That's why we have pre-1.0 version numbers (and alphas and betas) - they let us release code early for people to provide feedback before we've committed to a non-breaking API. Even with a stable 1.0 release things change. People using the library may come up with logical feature requests that the original design didn't consider, and which require backwards incompatible changes. That's when you bump to a 2.0 release. My dream project would have a version number of something like 1.237 - meaning there have been 237 new features releases since the 1.0 release without backwards compatibility breaks. My hat is off to any project that pulls that off though, it's a rare accomplishment!

failwhaleshark 5y ago

Semver is one way, but yes. It annoys me when libraries "clean up" (churn) their codebase when they should've thought more carefully about designing the interface (the contract) before publishing it. Code churn often creates more pointless work for N library users. I swear that most code churn is to make a project look "alive" or to give people something to do / job security. Deprecate slowly and have semi-automated user code fixups and notes.

m33k44 5y ago

I remember reading an article on MSDN(a developer info system - I call it a system because it was also available on CDs as internet was not available everywhere) by a Microsoft employee. The article was titled some what on the lines of "How the marketing teams won over the engineering teams!" The article mentioned a tussle within Microsoft between the marketing and the engineering teams. The engineers were pushing for providing backwards compatibility which slowed them down and took time in providing new things/features that were the next sought after tech. The marketing teams on the otherhand were very frustrated that new sought after tech was not coming out faster so that they can then market it to the world. That tugh-off war was ultimately won by the marketing teams at Microsoft as per the article.

matheusmoreira 5y ago
Reminds me of this article: https://www.joelonsoftware.com/2004/06/13/how-microsoft-lost-the-api-war/ https://www.joelonsoftware.com/2004/06/13/how-microsoft-lost...

renewiltord 5y ago

I actually genuinely think library authors should choose whatever trade-offs they want, and consumers should choose between those libraries based on the trade-offs they're comfortable accepting. The "always backwards compatibility" and "never backwards compatibility" schools of thought might just as well be "no programmer should ever use a garbage collector" or "all programmers should only ever use garbage collectors". Tools like OP help people who want to accept backwards incompatibility.

_carbyau_ 5y ago

I don't do modern coding with "bunch of dependencies downloaded at compile time" so now I am having fears about being "like so out of touch man". Just reply and tell me I am out of touch if I am so far off base here. If the updated library breaks your stuff then can't you just... not update? And if you want to update to the new library for XYZ new cool stuff it does, then you are already putting in effort to utilise XYZ. In which case you can sort out the new ABC fix too. It is extra work but it is part of having updated to the new coolness.