11 ms·

Mercurial, made by another (in my opinion) much more well-spoken kernel hacker, is what really introduced me to the concept that you do not break interface for

Mercurial, made by another (in my opinion) much more well-spoken kernel hacker, is what really introduced me to the concept that you do not break interface for downstream users, no matter how wrongly you may think they are using the interface. It’s an attitude that is difficult to convey because software developers always want to have the freedom to “improve” their own software, even at the possible cost of breaking something for some users (and worse, even telling users that they shouldn’t have been doing that in the first place). I keep going back to this blog post which I wish more people agreed with (Steve Losh is another person influenced by Mercurial): http://stevelosh.com/blog/2012/04/volatile-software/ http://stevelosh.com/blog/2012/04/volatile-software/

_Codemonkeyism 9y ago
For various reasons I would not go back to Java, but that was wonderful, your very old code still worked with newer versions, nothing broke during upgrades (even when behind deprecation one version).
- pjmlp 9y ago
  We do enjoy breaking changes every now and then. :) http://www.oracle.com/technetwork/java/javase/jdk-relnotes-index-2162236.html http://www.oracle.com/technetwork/java/javase/jdk-relnotes-i... Of course, they are never at the level of a Python 3 release.
  - _Codemonkeyism 9y ago
    Yes, but compared to e.g. Scala breaking changes in Java are heaven, thanks :-)
- cgdub 9y ago
  There's a great talk about backwards compatibility in Java by Brian Goetz: https://www.youtube.com/watch?v=2y5Pv4yN0b0 https://www.youtube.com/watch?v=2y5Pv4yN0b0
  - _Codemonkeyism 9y ago
    Watched, great talk, thanks!
- dilap 9y ago
  Go is really good about this too. It's great.
  - andrestc 9y ago
    It's not as strong as the kernel. Go does not break APIs but may break undocumented behavior.
pjmlp 9y ago
Rich Hickey has a wonderful talk about it. "Spec-ulation" - https://www.youtube.com/watch?v=oyLBGkS5ICk https://www.youtube.com/watch?v=oyLBGkS5ICk Which I fully agree with, basically there are no major, minor, patch level changes, in reality every change is a possible breaking changes, because we don't control what downstream is doing with our software.
- tome 9y ago
  Oh well, he should try a language with types ... /me runs ...
  - spraak 9y ago
    What are you trying to say? There are already a lot of explanations for why Clojure is dynamically typed and how it fills its role nicely that way.
    - tome 9y ago
      I know, I know. We've been having some interesting discussions about Clojure, Haskell and dynamic and static typing on these stories https://news.ycombinator.com/item?id=15602658 https://news.ycombinator.com/item?id=15602658 https://news.ycombinator.com/item?id=15593069 https://news.ycombinator.com/item?id=15593069 My comment was intended to be a wry one referencing those very interesting discussions and hinting that if you have types then you can more easily make breaking changes internally that aren't visible externally.
      - nickbauman 9y ago
        Dubious. The maintenance argument is weak. https://www.youtube.com/watch?v=2V1FtfBDsLU&feature=youtu.be&t=1h8m15s https://www.youtube.com/watch?v=2V1FtfBDsLU&feature=youtu.be...
        tome 9y ago
        I'm not sure what you mean. I thought it was self-evident that if your interface is an abstract type then you can change the implementation without breaking consumers. If you don't think it's true then please can you explain how? I thought the Clojure argument is not that types don't provide beneficial encapsulation, but that it's not worth paying the cost for them. Again if you know any different I'd be pleased to hear.
fdej 9y ago
If only compiler developers had this attitude.
- rkangel 9y ago
  I think Rust has done this quite well, managing to have the best of both worlds. Unstable features are only available on the nightly build and need specifically enabling (and their interface may change). Once a feature is considered stable it is carefully maintained. They do have some breaking changes, but they're not common and for unusual edge cases: https://killercup.github.io/bitrust/ https://killercup.github.io/bitrust/ This explains the Rust approach: https://blog.rust-lang.org/2014/10/30/Stability.html https://blog.rust-lang.org/2014/10/30/Stability.html
  - snuxoll 9y ago
    Rust also has yet to develop a stable ABI, I know they’re hard at work on so many things so I don’t say this as some snide remark - but until this is the case and Rust supports proper dynamic linking there’s whole domains of tasks I’m not comfortable using it for.
    - steveklabnik 9y ago
      I can assure you we’re not actively working on a stable ABI.
      - majewsky 9y ago
        This is a thing I've been wondering about. Is dynamic linking (in the way that C/C++ programs use it all the time) not a usecase that Rust wants to cover?
        steveklabnik 9y ago
        You can do dynamic linking, you just can't do it across compiler versions. This is how all of the Linux distros are doing it, for example. It's a spectrum: Rust: recompile the world when you update the compiler C++: recompile the world when you change ABIs or when the ABI changes (My understanding is that MSVC++ changes the ABI every ~2 years? I could be wrong.) C: never recompile the world It's not that we don't want the benefits of a stable ABI, but it's a monumental task, and there are more important things for now. Don't forget that you can get a stable ABI today by using the C ABI. You just lose out on the fancier features Rust has.
acobster 9y ago
Minor point, but he puts Python in with software that "almost never breaks" on update. Really? Am I missing something? Maybe he just hasn't tried running Python 3 yet...
- orf 9y ago
  You're missing that Python 3 is clearly labeled as a breaking change? He is talking about changes that are not labeled as breaking changes but do infact break things.
- mikeash 9y ago
  Python 3 is a similar language with a similar name to Python 2. It shouldn’t be considered an update, nor applied as one.
  - magicalhippo 9y ago
    Yet the Python 3 executable has the same name as the Python 2 executable...
    - vetinari 9y ago
      Only on Arch and Windows... All other Linux distribution have python3 and macOS binary is also python3.
    - computerfriend 9y ago
      No they don't. At least one of them has a number after it.
      - magicalhippo 9y ago
        I got both Python 2 and Python 3 installed on my machine, from the official distribution, and both have "python.exe", no numbers.
  - majewsky 9y ago
    Yet a huge number of Python modules are both valid Python 2 code and Python 3 code.
crdoconnor 9y ago
>Mercurial, made by another (in my opinion) much more well-spoken kernel hacker, is what really introduced me to the concept that you do not break interface for downstream users, no matter how wrongly you may think they are using the interface. It matters less, IMHO, the further up the stack you get. Being right at the bottom of every stack it matters most of all for the kernel, which is why Linus needs to be such a stickler. The amount of damage you can do by breaking userspace is vastly greater than the damage you do removing an hg command.
rambojazz 9y ago
> you do not break interface for downstream users, no matter how wrongly you may think they are using the interface How does Linux manage to never break its API? For example in the event of a refactoring.
- emodendroket 9y ago
  I mean, is the answer really that hard? If the "refactor" would break the interface, you can't do it. Maybe you deprecate the old method and make a new one.
  - chrisseaton 9y ago
    How do they handle bug fixing though? What if one application wants a bug fixed, but another application is depending on the bug inadvertently?
    - grigjd3 9y ago
      You trivialize the broken usage.
      - deleted 9y ago
        [deleted]
    - magnetic 9y ago
      The bug they fix is the bug that keeps the API contract intact. The first application gets its bug fixed, and the second application breaks because it was relying on a bug that didn't match the API contract. You program to an API, not to an implementation. The second application did the latter, and will therefore pay the price.
      - emodendroket 9y ago
        Well, sometimes, but the Windows philosophy is probably to introduce a shim so that an app relying on the broken functionality still works.
EGreg 9y ago
Why not simply bake these contracts and tests into the actual software APIs so they are impossible to break? That way the app would be "backward compatible certified" if it used this layer. The layer would only run during deployment. I guess an approximation is the Passing indicator on Travis or GitHub.
- majewsky 9y ago
  Some contracts are so complicated that we need 100-page RFCs to express them. What makes you think we can express these in a regular type system? (Unless you use Idris or Coq or something like that.)
deleted 9y ago
[deleted]
smcl 9y ago
I was about to post that it was funny this URL popped up when I read the Lobste.rs discussion until I saw your username :)
pdkl95 9y ago
> software developers always want to have the freedom to “improve” their own software Developer can still have that freedom in most[1] situations. When you have a new idea for a better/faster implementation that is, unfortunately, incompatible with the existing interface, you can usually provide compatibility with a shim that implements the old interface using the new interface. Sometimes this is little more than some #define renames and a handful of very-thin wrapper functions. Occasionally it will be a bit heavier, which might make the legacy interface slower, but this should be balanced (approximately) the faster underlying implementation. If the compatibility shim is significantly slower (or has other unavoidable runtime issues), an upgrade that is slower for legacy uses that still works is always better than breaking existing code. For an example and opinionated discussion, see JWZ's post[2] about his OpenGL 1.3 -> OpenGL ES 1.1 header. [1] Exceptions are cases where there is a *fundamentally unavoidable cost to providing the interface (e.g. where N is the total number of interface entry points, something in the critical path unavoidably MUST take >O(N) time or space, or N is limited externally to a small value ("we only have silicon for at most N instructions")) edit: [1b] Another exception would be when the legacy interface itself is a serious security problem. If your legacy interface unfortunately forgot to include a way to supply credentials and simply accepts writes from anyone, a prompt fix that breaks that API is necessary and appropriate. Next time you're designing an interface, be sure to remember that security has to be baked in from the very beginning! [2] https://www.jwz.org/blog/2012/06/i-have-ported-xscreensaver-to-the-iphone/ https://www.jwz.org/blog/2012/06/i-have-ported-xscreensaver-... (you might want to copy/paste that URL if your browser betrays where you came from with a Referer header)
- jordigh 9y ago
  > Sometimes this is little more than some #define renames and a handful of very-thin wrapper functions. That's more or less the thesis of Steve's blog post too.
emerged 9y ago
There are cases where a major change to a software project caused a handful of minor regressions but added new functionality, vastly improved performance in the majority of cases, etc. In these cases, a Zero Regressions policy is a bit silly. You can flip things around and look at the new code as baseline and from that perspective the old code has many more regressions relative to it. There seems to be a pretty stark divide between these mindsets.

_Codemonkeyism 9y ago

For various reasons I would not go back to Java, but that was wonderful, your very old code still worked with newer versions, nothing broke during upgrades (even when behind deprecation one version).

pjmlp 9y ago
We do enjoy breaking changes every now and then. :) http://www.oracle.com/technetwork/java/javase/jdk-relnotes-index-2162236.html http://www.oracle.com/technetwork/java/javase/jdk-relnotes-i... Of course, they are never at the level of a Python 3 release.
- _Codemonkeyism 9y ago
  Yes, but compared to e.g. Scala breaking changes in Java are heaven, thanks :-)
cgdub 9y ago
There's a great talk about backwards compatibility in Java by Brian Goetz: https://www.youtube.com/watch?v=2y5Pv4yN0b0 https://www.youtube.com/watch?v=2y5Pv4yN0b0
- _Codemonkeyism 9y ago
  Watched, great talk, thanks!
dilap 9y ago
Go is really good about this too. It's great.
- andrestc 9y ago
  It's not as strong as the kernel. Go does not break APIs but may break undocumented behavior.

pjmlp 9y ago

Rich Hickey has a wonderful talk about it. "Spec-ulation" - https://www.youtube.com/watch?v=oyLBGkS5ICk https://www.youtube.com/watch?v=oyLBGkS5ICk Which I fully agree with, basically there are no major, minor, patch level changes, in reality every change is a possible breaking changes, because we don't control what downstream is doing with our software.

tome 9y ago
Oh well, he should try a language with types ... /me runs ...
- spraak 9y ago
  What are you trying to say? There are already a lot of explanations for why Clojure is dynamically typed and how it fills its role nicely that way.
  - tome 9y ago
    I know, I know. We've been having some interesting discussions about Clojure, Haskell and dynamic and static typing on these stories https://news.ycombinator.com/item?id=15602658 https://news.ycombinator.com/item?id=15602658 https://news.ycombinator.com/item?id=15593069 https://news.ycombinator.com/item?id=15593069 My comment was intended to be a wry one referencing those very interesting discussions and hinting that if you have types then you can more easily make breaking changes internally that aren't visible externally.
    - nickbauman 9y ago
      Dubious. The maintenance argument is weak. https://www.youtube.com/watch?v=2V1FtfBDsLU&feature=youtu.be&t=1h8m15s https://www.youtube.com/watch?v=2V1FtfBDsLU&feature=youtu.be...
      - tome 9y ago
        I'm not sure what you mean. I thought it was self-evident that if your interface is an abstract type then you can change the implementation without breaking consumers. If you don't think it's true then please can you explain how? I thought the Clojure argument is not that types don't provide beneficial encapsulation, but that it's not worth paying the cost for them. Again if you know any different I'd be pleased to hear.

fdej 9y ago

If only compiler developers had this attitude.

rkangel 9y ago
I think Rust has done this quite well, managing to have the best of both worlds. Unstable features are only available on the nightly build and need specifically enabling (and their interface may change). Once a feature is considered stable it is carefully maintained. They do have some breaking changes, but they're not common and for unusual edge cases: https://killercup.github.io/bitrust/ https://killercup.github.io/bitrust/ This explains the Rust approach: https://blog.rust-lang.org/2014/10/30/Stability.html https://blog.rust-lang.org/2014/10/30/Stability.html
- snuxoll 9y ago
  Rust also has yet to develop a stable ABI, I know they’re hard at work on so many things so I don’t say this as some snide remark - but until this is the case and Rust supports proper dynamic linking there’s whole domains of tasks I’m not comfortable using it for.
  - steveklabnik 9y ago
    I can assure you we’re not actively working on a stable ABI.
    - majewsky 9y ago
      This is a thing I've been wondering about. Is dynamic linking (in the way that C/C++ programs use it all the time) not a usecase that Rust wants to cover?
      - steveklabnik 9y ago
        You can do dynamic linking, you just can't do it across compiler versions. This is how all of the Linux distros are doing it, for example. It's a spectrum: Rust: recompile the world when you update the compiler C++: recompile the world when you change ABIs or when the ABI changes (My understanding is that MSVC++ changes the ABI every ~2 years? I could be wrong.) C: never recompile the world It's not that we don't want the benefits of a stable ABI, but it's a monumental task, and there are more important things for now. Don't forget that you can get a stable ABI today by using the C ABI. You just lose out on the fancier features Rust has.

acobster 9y ago

Minor point, but he puts Python in with software that "almost never breaks" on update. Really? Am I missing something? Maybe he just hasn't tried running Python 3 yet...

orf 9y ago
You're missing that Python 3 is clearly labeled as a breaking change? He is talking about changes that are not labeled as breaking changes but do infact break things.
mikeash 9y ago
Python 3 is a similar language with a similar name to Python 2. It shouldn’t be considered an update, nor applied as one.
- magicalhippo 9y ago
  Yet the Python 3 executable has the same name as the Python 2 executable...
  - vetinari 9y ago
    Only on Arch and Windows... All other Linux distribution have python3 and macOS binary is also python3.
  - computerfriend 9y ago
    No they don't. At least one of them has a number after it.
    - magicalhippo 9y ago
      I got both Python 2 and Python 3 installed on my machine, from the official distribution, and both have "python.exe", no numbers.
- majewsky 9y ago
  Yet a huge number of Python modules are both valid Python 2 code and Python 3 code.

crdoconnor 9y ago

>Mercurial, made by another (in my opinion) much more well-spoken kernel hacker, is what really introduced me to the concept that you do not break interface for downstream users, no matter how wrongly you may think they are using the interface. It matters less, IMHO, the further up the stack you get. Being right at the bottom of every stack it matters most of all for the kernel, which is why Linus needs to be such a stickler. The amount of damage you can do by breaking userspace is vastly greater than the damage you do removing an hg command.

rambojazz 9y ago

> you do not break interface for downstream users, no matter how wrongly you may think they are using the interface How does Linux manage to never break its API? For example in the event of a refactoring.

emodendroket 9y ago
I mean, is the answer really that hard? If the "refactor" would break the interface, you can't do it. Maybe you deprecate the old method and make a new one.
- chrisseaton 9y ago
  How do they handle bug fixing though? What if one application wants a bug fixed, but another application is depending on the bug inadvertently?
  - grigjd3 9y ago
    You trivialize the broken usage.
    - deleted 9y ago
      [deleted]
  - magnetic 9y ago
    The bug they fix is the bug that keeps the API contract intact. The first application gets its bug fixed, and the second application breaks because it was relying on a bug that didn't match the API contract. You program to an API, not to an implementation. The second application did the latter, and will therefore pay the price.
    - emodendroket 9y ago
      Well, sometimes, but the Windows philosophy is probably to introduce a shim so that an app relying on the broken functionality still works.

EGreg 9y ago

Why not simply bake these contracts and tests into the actual software APIs so they are impossible to break? That way the app would be "backward compatible certified" if it used this layer. The layer would only run during deployment. I guess an approximation is the Passing indicator on Travis or GitHub.

majewsky 9y ago
Some contracts are so complicated that we need 100-page RFCs to express them. What makes you think we can express these in a regular type system? (Unless you use Idris or Coq or something like that.)

deleted 9y ago

[deleted]

smcl 9y ago

I was about to post that it was funny this URL popped up when I read the Lobste.rs discussion until I saw your username :)

pdkl95 9y ago

> software developers always want to have the freedom to “improve” their own software Developer can still have that freedom in most[1] situations. When you have a new idea for a better/faster implementation that is, unfortunately, incompatible with the existing interface, you can usually provide compatibility with a shim that implements the old interface using the new interface. Sometimes this is little more than some #define renames and a handful of very-thin wrapper functions. Occasionally it will be a bit heavier, which might make the legacy interface slower, but this should be balanced (approximately) the faster underlying implementation. If the compatibility shim is significantly slower (or has other unavoidable runtime issues), an upgrade that is slower for legacy uses that still works is always better than breaking existing code. For an example and opinionated discussion, see JWZ's post[2] about his OpenGL 1.3 -> OpenGL ES 1.1 header. [1] Exceptions are cases where there is a *fundamentally unavoidable cost to providing the interface (e.g. where N is the total number of interface entry points, something in the critical path unavoidably MUST take >O(N) time or space, or N is limited externally to a small value ("we only have silicon for at most N instructions")) edit: [1b] Another exception would be when the legacy interface itself is a serious security problem. If your legacy interface unfortunately forgot to include a way to supply credentials and simply accepts writes from anyone, a prompt fix that breaks that API is necessary and appropriate. Next time you're designing an interface, be sure to remember that security has to be baked in from the very beginning! [2] https://www.jwz.org/blog/2012/06/i-have-ported-xscreensaver-to-the-iphone/ https://www.jwz.org/blog/2012/06/i-have-ported-xscreensaver-... (you might want to copy/paste that URL if your browser betrays where you came from with a Referer header)

jordigh 9y ago
> Sometimes this is little more than some #define renames and a handful of very-thin wrapper functions. That's more or less the thesis of Steve's blog post too.

emerged 9y ago

There are cases where a major change to a software project caused a handful of minor regressions but added new functionality, vastly improved performance in the majority of cases, etc. In these cases, a Zero Regressions policy is a bit silly. You can flip things around and look at the new code as baseline and from that perspective the old code has many more regressions relative to it. There seems to be a pretty stark divide between these mindsets.