6 ms·

Rich made some interesting points on developing libraries in such a manner that it doesn't introduce breaking changes (for the calling code). Does anyone here a

by grease 8y ago

Rich made some interesting points on developing libraries in such a manner that it doesn't introduce breaking changes (for the calling code). Does anyone here agree (or have counterpoints) to his suggested approach?

bpicolo 8y ago
This is essentially the model the Go community is aiming to do with versioning standards. It's also the philosophy the linux kernel takes to e.g. syscalls. It's something a lot of libraries could learn from. Most users want to build software for the long term and tend to avoid upgrading libraries too often. Fwiw, this is a large part of Rich's Spec-ulation [0] talk. [0] https://www.youtube.com/watch?v=oyLBGkS5ICk https://www.youtube.com/watch?v=oyLBGkS5ICk
- sethev 8y ago
  I don't know the specifics of the Go standards but semantic versioning doesn't add anything if you don't break your consumers.
tosh 8y ago
Around 14:50 in Rich talks about breaking changes and mentions a library he worked on (codec) to track function changes (checksums, not semantically but still) instead of looking at a library as 'changed' as a whole. This makes a ton of sense. Knowing a set of functions I rely on did not change (at all) provides more peace of mind to me than just knowing the function signature is still compatible. Versioning (or at least understanding changes) on the function level in addition to the library level sounds so powerful to me that I wonder why this concept isn't more mainstream yet. Are there other examples? [edit: I found codec: https://github.com/Datomic/codeq#codeq https://github.com/Datomic/codeq#codeq]
mkobit 8y ago
It sounds like the philosophy is "don't break your consumers", which I agree with. Somebody else already linked his Spec-ulation talk, but I think somewhere in there he compares the 10 minutes a library author spends on deprecating or removing some piece of code versus the 10 minutes each and every one of your consumers spends trying to determine why a dependency they put trust in broke their application. I do think it can be very difficult to know if you, as a producer, are breaking any consumers. I'd love to see some sort of OSS build tool and CI system that automatically rebuilds and tests all consumers of your library to give you, the library author, more information about the changes you make before releasing.
- guipsp 8y ago
  Rust (core) uses the cargo ecosystem as tests
  - puredanger 8y ago
    Similar idea in ClojureScript: https://github.com/cljs-oss/canary https://github.com/cljs-oss/canary
Naomarik 8y ago
Clojure has been the most stable ecosystem I've ever dealt with. Once you get into it it's not uncommon to see libraries that are years old that function perfectly. I'm not even scared to update clojure to the latest alpha builds because things just always work. There's also this to look at on the subject: https://lkml.org/lkml/2018/8/3/621 https://lkml.org/lkml/2018/8/3/621 Reddit discussion: https://www.reddit.com/r/linux/comments/95b1hf/linus_torvalds_on_regressions/ https://www.reddit.com/r/linux/comments/95b1hf/linus_torvald...
- kamaal 8y ago
  I imagine in case of a Kernel that should be like a rule beyond any scope of discussion. And may be for any piece of software that is at the bottom layers of any stack. You just can't break backwards compatibility. This was a big mistake Python 3 made.
  - iwintermute 8y ago
    That's why it's called Python 3 and not Python 2.x Think about it like totally new language
    - deleted 8y ago
      [deleted]
    - decebalus1 8y ago
      If it's like totally a new language, it should be called TotallyNewython 1.0
      - iwintermute 8y ago
        AlmostTotallyNewthon would be better, correct. Would shift discussion from 'upgrade' to 'migration' without all the usual cries. Other than that it's understandable that some architecture decisions from 1991 haven't aged well.
      - Roboprog 8y ago
girishso 8y ago
Elm goes a step further, it enforces semantic versioning. If the package has any breaking interface changes, it forces major version change.
- jeremiep 8y ago
  That's not a step further, you lose the ability to use the latest version without code changes.
  - girishso 8y ago
    Which in any case you'd have to do in other languages. What Elm provides is a way for library users to decide whether to upgrade the library or not and more importantly not automatically upgrade to a library which needs code changes.
- puredanger 8y ago
  If you just don't have breaking interface changes you never need a major version change.
natbobc 8y ago
I agree with the principle that you shouldn't arbitrarily change API's. I disagree if it is extended to being an unbreakable contract. There are specific scenarios where people should consider breaking changes most of which fall under some for of improving usability of the library; 1. improving/providing secure defaults. 2. reducing the API surface so that usage is more evident. 3. refactoring that eases maintenance. 4. improving performance where appropriate. 5. probably others I haven't thought of. In RFC style I would say it's a SHOULD rule rather than a MUST.
- puredanger 8y ago
  In all of these cases you can add new functions without removing (and breaking) the old ones.
  - natbobc 8y ago
    Agree in many cases you can do this however with point 2 the removal of a function(s) is literally the aim. The accretion of functions benefits legacy systems however its tradeoff is potential harm to users new and unfamiliar with the library. Accretion creates a cognitive overhead (even if only minor) for both the maintainer and new users. Maintainers when they return to code to update and modify behaviour. New users when they seek to understand the library through documentation, examples, and usage. I don't think it's coincidence that a number of languages and libraries acknowledge this by having "one correct way to do X". Using a concrete example relating to security; libressl maintained much of the API surface that OpenSSL provides. In essence they aimed to provide a "drop-in" replacement. However there were whole families of algorithms and functions which they deemed "unsafe/unfit" for purpose (e.g. FIPS algorithms). I think that's a perfectly valid exception to the rule. It acts as a canary in the coal mine and you have options; fix your code or defer upgrading. I would advocate for thoughtful deprecation cycles over ossification of poor APIs and algorithms.
filoeleven 8y ago
He’s been thinking some really interesting thoughts about dependency management and libraries. It sounds like if he had his way, library versions might disappear entirely. From the interview: “But there are really two very distinct kinds of change - there are breaking changes, where your expectations have been violated, and there are accretions, where there's just some more stuff where there wasn't stuff before. And in general, accretion is not breaking.” To paraphrase stuff I’ve heard him talk about elsewhere, your library should never change a function to require more input or provide less output. If you decide to do that, call it something else and keep the old function around. He’s talked about function-level versioning too, touches on it here a bit. If I use library functions x y and z and the maintainers only broke (the contract for) functions a b and c, then I probably don’t care and can grab the latest version for its other new goodies. But that decision still requires me to read the library source or at least a changelog, even though it’s simple enough for the right tool to be able to report it to me. Spec looks like a step in that direction.

by grease 8y ago

bpicolo 8y ago

This is essentially the model the Go community is aiming to do with versioning standards. It's also the philosophy the linux kernel takes to e.g. syscalls. It's something a lot of libraries could learn from. Most users want to build software for the long term and tend to avoid upgrading libraries too often. Fwiw, this is a large part of Rich's Spec-ulation [0] talk. [0] https://www.youtube.com/watch?v=oyLBGkS5ICk https://www.youtube.com/watch?v=oyLBGkS5ICk

sethev 8y ago
I don't know the specifics of the Go standards but semantic versioning doesn't add anything if you don't break your consumers.

tosh 8y ago

Around 14:50 in Rich talks about breaking changes and mentions a library he worked on (codec) to track function changes (checksums, not semantically but still) instead of looking at a library as 'changed' as a whole. This makes a ton of sense. Knowing a set of functions I rely on did not change (at all) provides more peace of mind to me than just knowing the function signature is still compatible. Versioning (or at least understanding changes) on the function level in addition to the library level sounds so powerful to me that I wonder why this concept isn't more mainstream yet. Are there other examples? [edit: I found codec: https://github.com/Datomic/codeq#codeq https://github.com/Datomic/codeq#codeq]

mkobit 8y ago

It sounds like the philosophy is "don't break your consumers", which I agree with. Somebody else already linked his Spec-ulation talk, but I think somewhere in there he compares the 10 minutes a library author spends on deprecating or removing some piece of code versus the 10 minutes each and every one of your consumers spends trying to determine why a dependency they put trust in broke their application. I do think it can be very difficult to know if you, as a producer, are breaking any consumers. I'd love to see some sort of OSS build tool and CI system that automatically rebuilds and tests all consumers of your library to give you, the library author, more information about the changes you make before releasing.

guipsp 8y ago
Rust (core) uses the cargo ecosystem as tests
- puredanger 8y ago
  Similar idea in ClojureScript: https://github.com/cljs-oss/canary https://github.com/cljs-oss/canary

Naomarik 8y ago

Clojure has been the most stable ecosystem I've ever dealt with. Once you get into it it's not uncommon to see libraries that are years old that function perfectly. I'm not even scared to update clojure to the latest alpha builds because things just always work. There's also this to look at on the subject: https://lkml.org/lkml/2018/8/3/621 https://lkml.org/lkml/2018/8/3/621 Reddit discussion: https://www.reddit.com/r/linux/comments/95b1hf/linus_torvalds_on_regressions/ https://www.reddit.com/r/linux/comments/95b1hf/linus_torvald...

kamaal 8y ago
I imagine in case of a Kernel that should be like a rule beyond any scope of discussion. And may be for any piece of software that is at the bottom layers of any stack. You just can't break backwards compatibility. This was a big mistake Python 3 made.
- iwintermute 8y ago
  That's why it's called Python 3 and not Python 2.x Think about it like totally new language
  - deleted 8y ago
    [deleted]
  - decebalus1 8y ago
    If it's like totally a new language, it should be called TotallyNewython 1.0
    - iwintermute 8y ago
      AlmostTotallyNewthon would be better, correct. Would shift discussion from 'upgrade' to 'migration' without all the usual cries. Other than that it's understandable that some architecture decisions from 1991 haven't aged well.
    - Roboprog 8y ago

girishso 8y ago

Elm goes a step further, it enforces semantic versioning. If the package has any breaking interface changes, it forces major version change.

jeremiep 8y ago
That's not a step further, you lose the ability to use the latest version without code changes.
- girishso 8y ago
  Which in any case you'd have to do in other languages. What Elm provides is a way for library users to decide whether to upgrade the library or not and more importantly not automatically upgrade to a library which needs code changes.
puredanger 8y ago
If you just don't have breaking interface changes you never need a major version change.

natbobc 8y ago

I agree with the principle that you shouldn't arbitrarily change API's. I disagree if it is extended to being an unbreakable contract. There are specific scenarios where people should consider breaking changes most of which fall under some for of improving usability of the library; 1. improving/providing secure defaults. 2. reducing the API surface so that usage is more evident. 3. refactoring that eases maintenance. 4. improving performance where appropriate. 5. probably others I haven't thought of. In RFC style I would say it's a SHOULD rule rather than a MUST.

puredanger 8y ago
In all of these cases you can add new functions without removing (and breaking) the old ones.
- natbobc 8y ago
  Agree in many cases you can do this however with point 2 the removal of a function(s) is literally the aim. The accretion of functions benefits legacy systems however its tradeoff is potential harm to users new and unfamiliar with the library. Accretion creates a cognitive overhead (even if only minor) for both the maintainer and new users. Maintainers when they return to code to update and modify behaviour. New users when they seek to understand the library through documentation, examples, and usage. I don't think it's coincidence that a number of languages and libraries acknowledge this by having "one correct way to do X". Using a concrete example relating to security; libressl maintained much of the API surface that OpenSSL provides. In essence they aimed to provide a "drop-in" replacement. However there were whole families of algorithms and functions which they deemed "unsafe/unfit" for purpose (e.g. FIPS algorithms). I think that's a perfectly valid exception to the rule. It acts as a canary in the coal mine and you have options; fix your code or defer upgrading. I would advocate for thoughtful deprecation cycles over ossification of poor APIs and algorithms.

filoeleven 8y ago

He’s been thinking some really interesting thoughts about dependency management and libraries. It sounds like if he had his way, library versions might disappear entirely. From the interview: “But there are really two very distinct kinds of change - there are breaking changes, where your expectations have been violated, and there are accretions, where there's just some more stuff where there wasn't stuff before. And in general, accretion is not breaking.” To paraphrase stuff I’ve heard him talk about elsewhere, your library should never change a function to require more input or provide less output. If you decide to do that, call it something else and keep the old function around. He’s talked about function-level versioning too, touches on it here a bit. If I use library functions x y and z and the maintainers only broke (the contract for) functions a b and c, then I probably don’t care and can grab the latest version for its other new goodies. But that decision still requires me to read the library source or at least a changelog, even though it’s simple enough for the right tool to be able to report it to me. Spec looks like a step in that direction.