15 ms·

I claim Rich Hickey is wrong about non-null arguments to functions (2020)

77 points by Capricorn2481 3y ago

patrickthebold 3y ago
> You are changing the behavior of the function. That is a breaking change. That makes almost everything breaking.
- nimih 3y ago
  Well, yes. Any time you change a function’s behavior, you’re risking breaking the behavior of its callers. That’s just the nature of programming and doesn’t seem like particularly controversial statement, honestly.
  - Tcepsa 3y ago
    One of the big benefits of using functions is encapsulation: the idea that you (as the caller) do not need to understand how a function does what it does, you just need to know what it does (perhaps along with some performance guarantees so you know it’s not going to do e.g. an O(n^2) operation). Beyond that, I don’t want to care and should not generally have to care about what the function is doing. As long as I get out (including side effects) what I expect based on what I put in, the internal behavior of the function can change any which way and I would not consider that breaking, because my code—using that function—would still run just fine.
    - nimih 3y ago
      Yes, I agree. I probably should've been more clear that when I said "behavior," I meant "behavior observable by the caller." Going back to the original article, it seems reasonable to me that the following qualifies as such a user-observable change: foo(null) # throws InvalidArgumentException updated_foo(null) # the same as foo(0)
      - Tcepsa 3y ago
        Thank you for the clarification; that's a much more compelling argument, and I think I am inclined to agree. I believe, however, that that is not what Hickey is saying. He's not saying "previously you could pass null and it would throw an exception, and now you can pass null and it won't throw an exception". He's saying "Previously if you tried to pass null you'd get a compiler error, but then I figured out that the method as written can handle nulls, so I changed the type signature to allow null to be passed in without the compiler throwing an error about it. See the updated documentation to understand what it does if you pass in a null value." This isn't a breaking change because up until that point any code that was written to use that method already wasn't passing null (because it couldn't, because it wouldn't compile). The method's behavior hasn't changed, just its type signature, and so for any of the arguments that the existing code might possibly pass to it, it will still handle all of those exactly the same as it would have before (because, again, the implementation did not change). Therefore, it is not a breaking change.
  - tshaddox 3y ago
    Then why talk about “breaking” changes? Just say “changes.” Of course, the reason we use the phrase “breaking changes” is that we are making a distinction between certain types of changes.
    - hansvm 3y ago
      Whichever line in the sand you draw as an API boundary, if the break doesn't propagate across it then the change isn't breaking in that context. Suppose (as a crude example to simply illustrate the point), a codebase exposes some CRUD operations. Internally it relies on sorting methods and comparator methods for that sorting. If you reverse the order that sort works and also reverse all the comparators then internally you've made a bunch of breaking changes, but externally you can describe the change as non-breaking (probably -- performance regressions can be tricky). The change itself was both breaking and non-breaking, and the choice of description depends on who you're describing it to.
      - tshaddox 3y ago
        > externally you can describe the change as non-breaking (probably -- performance regressions can be tricky) Yes, in most programming languages/environments it is always possible to write your application in such a way that any change of a dependency is I’ll break your app. Heck, you could throw an exception if someLib.version != “1.0.2” and then complain that a patch to 1.0.3 is actually a breaking change. Or your app could test that some API method does not exist, then complain when that method gets added later. Or you could complain when performance improvements reveal race conditions in your app (or break your usage of your computer as a space heater).
    - nimih 3y ago
      You can improve performance without changing the function's interface. Or remove a bug which causes an unrecoverable error. Or fix a memory leak. Or improve the error message on an internal assertion. Or add new methods to an object's interface. These would not change the extant observable behavior for most problem domains (although you could probably reasonably argue that any changes in performance characteristics are potentially breaking changes in e.g. game development), and are thus not "breaking", but are nevertheless useful changes.
- kazinator 3y ago
  A breaking change occurs if the function (before the proposed change) can be called in a way that doesn't crash (stop the program, or render it intoperable, or blow up at compile time or basically bring the show to a halt), and after the change, that call yields a different result or has a different effect or output. We can divide those breaking changes into two: 1. The call obeys the specification. 2. The call, though apparently successful, circumvents the specification, relying on undocumented behavior. The question is how much we care about 2, and there is no 100% answer. There can be situation in which some kinds of 2 breakages are such that we care about them more than 1 breakages. Suppose there is a certain correct, documented way of using the API, but almost nobody out there uses it that way. And suppose there is some undocumented way of using the API, which millions of installations use, thousands of times per second. Suppose we need to implement something new, or even ore importantly, fix a critical bug, and suppose that the work boils down to either breaking one, or the other. It may be better to break the former to keep the latter working (and possibly elevate the latter to documented status). In other words, in a perfect world we'd like to say that breakages of type 1 are non-negotiable, whereas 2 can be debated. But we can't even do that.
draw_down 3y ago
No, he’s right. he’s speaking only of what is required and what is provided, all else being equal. The what and the why are different. If the function started to provide different results for the same arguments, that’s different from what he’s talking about and would be a breaking change. Worrying about the internals of the function is a violation of encapsulation. We care what we provide and what we receive back.
- one-more-minute 3y ago
  I mean, the article's logic is effectively that you can break code without the type system helping or warning you. I don't think Hickey would disagree with this – it's arguably a point in his favour.
- joeatwork 3y ago
  I think the poster means that adding null is a big, substantial change to the interface - if you have a function that used to not make sense with a null argument, and then it suddenly does make sense with a null, then the function has changed in a way observable to callers.
  - sanderjd 3y ago
    > the function has changed in a way observable to callers. Observable to new callers who might want to pass None, but not to the existing callers who don't pass None as it is because it was not possible prior to the change.
  - draw_down 3y ago
    [dead]
XVincentX 3y ago
The article is wrong. Rich Hickey is talking about the surface of the function, interface. The implementation details do not matter in this case.
- riffraff 3y ago
  The article is claiming that the surface and the implementation are related. If client code passed a null value before, an error would have occurred. Now it is handled with a default that the original code was not accounting for, and this might be bad. Thus the change in interface _is_ a change in behavior.
  - saulpw 3y ago
    Which no client should/would have been using (given the error they would have received). Why would this new implementation be presumed incompatible, when any reasonable person would think this is an addition to the interface, and not a difference to the existing interface?
    - omnicognate 3y ago
      > no client should/would No client could, even. He's talking about static types so it wouldn't have been possible even to express a caller not providing a value.
  - wk_end 3y ago
    If the code passed in a null value before, it was violating the contract the function provided. A component isn't responsible for what happens when you operate it out of spec. If I were a hardware designer and some changes to an IC I'm working on are going to make the next batch, say, work properly in hotter temperatures than it could before, and one of my coworkers comes and says, "Don't make that change, what if someone using that IC is deliberately operating it out of spec expecting it to fail and now the behaviour is going to change!" I'm going to start polishing up my resume, because I'm working for an organization that employs lunatics. Happily, that wouldn't happen in the hardware world; unhappily, I'm a software developer rather than a hardware designer.
GrumpySloth 3y ago
I agree with Rich Hickey here and I think this is a flaw of sum types compared to union types. If Option<T> was a union type, T would be a subtype of Option<T> and those changes wouldn't be breaking. When writing Rust, I find myself repeatedly writing implementations of the From trait for enums, just because most of the time I actually want union types, not sum types. I think sum types should be built on top of union types by combining union types with a "lexically-scoped newtype", if that makes sense (i.e. they wouldn't be built in, they would be the result of two orthogonal features, while allowing for other combinations as well). Edit: another nice consequence of this design would be None being its own type, which can be transparently converted to an Option<T> for any T, allowing for type inference to go in only one direction, while still avoiding the boilerplate of Option<T>::None. Unidirectional type inference would make for more intelligible compiler errors. Full Hindley-Milner can get confusing.
- mjburgess 3y ago
  Whenever i design a lang, i just give option semantics to types of the form `x|none`
- kibwen 3y ago
  > which can be transparently converted to an Option<T> for any T, allowing for type inference to go in only one direction, while still avoiding the boilerplate of Option<T>::None You don't need to qualify the None, the following works: fn foo<T>(x: T) -> Option<T> { None }
  - yakubin 3y ago
    Yes. That's thanks to Hindley-Milner type inference which makes the compiler infer which None you mean. But in longer code snippets full Hindley-Milner type inference can cause hard-to-understand type errors. Here specifically the compiler infers the type of a value in your function body based on the type of the function it belongs to, which is in the opposite direction than what a programmer typically thinks in. It leads to spooky actions at a distance.
    - 3y ago
jhardy54 3y ago
Counter-point: consider the case where the body of the function is only `return value`.
- codebje 3y ago
  Isn't that an argument that the function implementation might break, not that callers might? Assuming implicit conversion for union types, you could either widen argument or narrow return without changing that implementation, but not both. A different implementation may not be able to handle either modification, though.
fiddlerwoaroof 3y ago
The definition of breaking change here completely destroys any idea of abstraction or bug fixes. If calling a function requires knowing how the function is implemented, then the function should not exist in the first place: the whole point of an abstraction is that, if the use-site fulfills the precondition, it can assume that the postcondition holds after it used the abstraction. This means that relaxing the precondition is, by definition, not a breaking change because the old inputs are a subset of the new ones. Similarly, strengthening the postcondition isn’t a breaking change because the new outputs are a subset of the old ones. (If and he use-site assumes that it gets a specific member of the function’s codomain for specific input values, you have an abstraction leak and the use-site has a bug [or the function should be inlined because it’s a false abstraction])
- _madmax_ 3y ago
  I agree with you wholeheartedly
- sanderjd 3y ago
  My first thought was, "I wonder what the author considers a non-breaking change..."
  - squirtlebonflow 3y ago
    Comments, updating dependencies, or documentation
    - sanderjd 3y ago
      Updating dependencies could way more easily be a breaking change than what we're discussing... And comments and documentation aren't code changes at all.
- hn_throwaway_99 3y ago
  I really wish I had the ability to downvote articles. Agree with you 100%, and I feel like I lost braincells from reading this sentence: > You are changing the behavior of the function. That is a breaking change. So this guy is declaring someone else "wrong" because he unilaterally decides to redefine what a "breaking change" means? This is one of the dumbest "all Internet arguments are over semantics" examples I can think of. As an aside, I've seen a number of recent instances (here's another, https://news.ycombinator.com/item?id=36826111 https://news.ycombinator.com/item?id=36826111) where a flat out ridiculous blog post makes it to the front page, only for the top comments to all point out how dumb and ridiculous it is. To stay within the HN guidelines, I'm going to refrain from speculating why, and I certainly know that HN audience is not some monolithic group, and that people have many different opinions. Would just like to know if others have noticed this or have other specific examples.
easeout 3y ago
It's only considered "breaking" to change the externally observed behavior of at least one previously published use of a code interface. When widening the interface's range of inputs, you can avoid changing existing behavior. We do that all the time when we create a new optional parameter, new CLI tool switch or subcommand, new public function or type in a library, new API endpoint, new web domain, new network protocol.
alexmolas 3y ago
Isn't what Rich Hickey defense the same as the Liskov substitution principle? Or at least very similar.
- codebje 3y ago
  Somewhat related. The subclass relationship isn't very rigorous, you're free to have completely different behaviours out of the same methods in a subclass. Liskov says don't do that: if B is a subclass of A it should behave like an A. When it comes to types, a subtype is one that can be transparently substituted for its super type. If S <: T, a term of type S can be used where a T was expected just fine. For union types, it's a given that T <: T|a for any type a, and a T will behave exactly like a T|a whose value is a T. The Liskov principle states that a subclass should be a subtype.
krick 3y ago
Others already commented on why this complaint doesn't make any sense, but, hey, at least I was pointed to a Rich Hickey talk I've missed. It never occurred to me before that "Maybe" isn't actually a "maybe", but Hickey is right. Seriously, every single talk of his I've seen so far has something revealing in it. I should go search for all his presentations I've missed, I guess.
coldtea 3y ago
>If there wasn't a change in behavior, why are you changing the contract? Because you're restricting or relaxing the types, and you want to type system to know this. This doesn't have to translate to a change in behavior.
Gunax 3y ago
I feel like they are talking past each other, or at least talking about different things. Hickey seems to be talking about breaking the compiler. That is, he is saying that changing the return signature from null|int to int should not break the compilation. In effect, this would create a warning a la 'You are checking for null, but var x cannot be null'. The author seems to be talking about breaking changes in the behaviour/operation sense. Those are different types of breaking.
- scubbo 3y ago
  But there _is_ no "breaking" change in the behaviour, because none of the previously-possible behaviours _can_ break. For every way-of-calling the function that previously existed, those ways-of-calling will continue to operate as before. Some new ways-of-calling will now be legal, but that is not a breakage, because no code that already existed could have been using those ways before, and so no code exists that can be broken. Clients _may_ choose to write new code to take advantage of the new behaviour - but there is nothing that they _must_ do (unlike with a breaking change, where they _must_ take action). It's for the purposes of this categorization, for communicating about client's responsibilities, that the concept of breaking changes is valuable.
  - anonymouskimmer 3y ago
    Hijacking this to respond to a reply you made 40 days ago to a comment I made 45 days ago: https://news.ycombinator.com/item?id=36335405 https://news.ycombinator.com/item?id=36335405 > But that doesn't seem to be what you were, or are, saying. If you issue is specifically with "an epicurean theme park of enlightenment and joy to span the stars", and not with the dramatized reaction to (against) it, then...I must say I still don't understand what the problem is. An experience which is _definitionally_ pleasant, fulfilling, and non-harmful (to self or others) is....well, it's capital-g Good, no? Technological advance is fine on its own. The risk with "an epicurean theme park of enlightenment and joy to span the stars" is our own mental faculties, presuming no one is left behind. A variety of examples of this turning bad exist in fiction, a particularly pertinent one to me is Asimov's "The End of Eternity", and another one is the post-immortality centuries of Niven's "Known Space", but there are many others. Psychologically theme parks are meant to be visited for a break. If we spend our entire lives in them it does stuff to our capacities. In everyday life animal species spend most of their problem solving effort on dealing with other members of their species (and some similar species). We are our own greatest competitors. In part, I think animals do this because members of each species are approximately on equal footing, so our individual problem solving is sufficient to deal with other individuals, and our group problem solving sufficient to deal with other groups of us. But nature itself, and the universe itself, is the biggest player. The biggest risk. I fear that themepark life, as opposed to themepark visit, will either spark ennui and a bunch of interpersonal shenanigans, or will invite people to forget that we're just living in a fragile bubble. If the opportunities are not equal for every person, the situation is even worse. And I like to think that we have a responsibility to our co-inhabitants in the universe, the other species. A themepark life ultimately risks navel-gazing (at the individual or group level) and ignoring these responsibilities as well. No thank you. I want a nice life, and a nice place to live it. But I don't want ennui. I don't want self-involved navel-gazing. I want people paying attention to the real issues in the universe. We spend enough effort already on dealing with other humans. > What is it about the real world that is more important, inherently, than "the experiences that arise from living in it"? If a "better" (I'm hand-waving the complexity of comparison, because it's assumed as part of the discussion) set of experiences can arise - with perfect certainty, no trade-offs, no utilitarian cheats, just "everything is better for everyone" - then, as the other commenter said, it would be abhorrent to deny it 'because of some philosophical quibble about a difference between “simulation” vs “reality”'. Sure. I agree. As long as we take pains to address the inevitable externalities before they impact others (including other species) this is fine. But make sure you know what's "better" for everyone before it is implemented. It's not obvious that even a god could manage this, much less humans (or even the humans/angels/demons of The Good Place). "What is it about the real world that is more important, inherently" - It's everything, and we're just a part of it. "importance" is a value judgement, so is inextricably caught up in the individual so judging, but subtract out the value judgement and it's obvious that a human and a squirrel are more "important" than a single human. Now expand this to the "real world" entire.
gavinhoward 3y ago
I've read the article and watched the video that it is a reply to. I both agree with the author and disagree. I disagree that Rich Hickey is wrong when it comes to whether those changes should be breaking or not. Those curves can be non-breaking, unlike what the author of the article claims. But I agree with the author that the fact that they are breaking doesn't really matter. Rich Hickey mentions this himself in his talk. He says that no one talks about the costs of using Option or Maybe. So then he lays out the costs. And I was...not that impressed. Sure, this change that should be non-breaking is breaking. And your downstream clients will have to change their code. That seems bad. Until you realize that the only change they would be forced to make is to delete what is now dead code. Sure, it's annoying; I don't deny that. But Rich Hickey claims that it increases code maintenance to have to make the change. I whole-heartedly disagree because any time you can safely delete code, you are making maintenance easier. Plus, despite all of his complaints that people don't considers costs, he never really considers the costs of his proposal, the biggest of which is the complexity of the language. I think the industry has rightly come to the conclusion that, all else being equal, and even sometimes when they are not, a more complex language is a worse language. I think proper union types, the kind needed here, would add enormous complexity to the language for little benefit in rare cases. (Because how often do programmers relax constraints? I don't think they do that often.) In my language, I'll keep Option as it is, and make such changes breaking, in order to avoid the complexity of union types.
- pyrale 3y ago
  > I think the industry has rightly come to the conclusion that, all else being equal, and even sometimes when they are not, a more complex language is a worse language. That was a theory behind golang, and the observed result is that you're better off having a little bit more language constructs that may be misused, than having essential complexity in your problem space that you can't easily describe in your solution space, for lack of tools.
  - gavinhoward 3y ago
    That's why I said "all else being equal." Go took it too far.
saghm 3y ago
If I were changing the signature of `do_something` in Rust and wanted to avoid breaking callers when changing the function from accepting `T` to `Option<T>` , I'd do the following: fn do_something(value: impl Into<Option<T>>) { let value = value.into(); // remaining function body using `value` as an `Option<T>` } Existing calls to `do_something` will continue working because `Option<T>` implements `From<T>` (and `Into` implementations are generated in the opposite direction whenever `From` is implemented). Passing in `None` will also work because every type `T` implements `From<T>`, and therefore `Into<T>` as well. I'd even argue that in a lot of cases, this is worth doing even when creating new library functions (where there are no existing callers to worry about). In my experience, it's generally better to keep the API clean from a caller's perspective by taking on extra boilerplate internally in the implementation rather than shift the burden to the callers of the API in order to keep the library's implementation clean.
- skavi 3y ago
  Often better to do the conversion then immediately call a non generic inner function. This prevents the rest of the logic from getting monomorphized. It’s a pattern you’ll notice in std.
- skitter 3y ago
  This is a breaking change too, as it can break type inference: https://play.rust-lang.org/?edition=2021&gist=8d528bcec2b92b73780d3335d936dfcb https://play.rust-lang.org/?edition=2021&gist=8d528bcec2b92b...
  - obi1kenobi 3y ago
    Interestingly, even though it's a breaking change, breaking inference is explicitly defined to not be a semver-major change in Rust: https://predr.ag/blog/some-rust-breaking-changes-do-not-require-major-version/ https://predr.ag/blog/some-rust-breaking-changes-do-not-requ...
chmod600 3y ago
Examples? What problems will be caused if we allow existing callers to keep working as before?
paulddraper 3y ago
Rich Hickey is correct. I prefer sum types. But this is a legitimate point for union types.
singularity2001 3y ago
He isn't wrong, languages that make optionals not backwards comparable are. So swift "string?" is a perfect match for his insight, whereas rust option<string > is a mess.
- secretsatan 3y ago
  Having run into the is recently when our cloud API changed a value we relied on to be optional, it is indeed a breaking change in swift.
MrTortoise 3y ago
I feel like you missed the main point of what you are saying. Whilst existing callers who obey the contract will not be broken by this callers who depend upon this code breaking for null values will no longer do so. so it is a breaking change for them. You have just extended their behaviour. That said if we defined the functionality of code as being valid within a scope of input and not defined anything outside of this we are safer in a sense. That said you do require feedback to show you are out of range. But the point I want to make is that the line of argument that attempts to ascribe changes to behaviour on values that were out of scope as breaking changes is not helpful to anyone who is trying to co-exist via contracts in order to understand what work is expected as a result of change.
ndr 3y ago
Hard to disagree with Rich Hickey re non-breaking changes. See for yourself the "Fig. 3. Clojure codebase—Introduction and retention of code" chart at page 26 of https://download.clojure.org/papers/clojure-hopl-iv-final.pdf https://download.clojure.org/papers/clojure-hopl-iv-final.pd... Rich Hickey is right and has a track record to show.
jbritton 3y ago
I came away with a different interpretation of Rich Hickey’s talk. I think he was advocating against using slots, where a slot would be a struct field. A slot has to have a value and if a value is not available then null is put there and now null must be dealt with. Instead I think he was advocating for using dictionaries. You either have a key or you don’t, but you never use null. However it seems that now you would have a bunch of “key in dict” checks which are less efficient than “value is null” checks. Plus confusion about what exactly has been supplied in a dictionary. Although there may be cases where code can just sequence through what exists without worrying about what doesn’t exist.
- Sakos 3y ago
  I think the fact that it's impossible to know what could be in a dictionary without explicitly checking makes dicts an awful solution. A lot of times I'll encounter functions that take maps, but I'll have to reverse engineer the surrounding code (and if I'm particularly unlucky, a related service) to figure out what goes in there. I'll take explicit types over dicts any day.
darkavenger42 3y ago
This careless approach with nulls kinda works for Clojure because of "nil punning" but is dangerous for other languages I guess
AtNightWeCode 3y ago
Rick is right I believe. But why not just use overloading instead? That is what is done in many OO-langs to avoid making breaking changes.
hansvm 3y ago
Language features like type promotion or automatic casting to default/zero values are likely to change how the new function is called, purely based on the change in type signature. Even if every old input has the same output in the new function, old code can compile to produce a new input and have broken behavior. That's ignoring any metaprogramming or more complicated type shenanigans. Is there something special about Clojure making that change safe?
- layer8 3y ago
  I don’t know about Clojure specifically, but Java and the JVM have bridge methods for such purposes. Meaning, a hidden method with the old type signature would still get generated whose implementation transparently forwards to the new method with the changed type signature, e.g. converting the T value to an Optional<T> value (or whatever).
kazinator 3y ago
The changed return convention is potentially a breaking change. If there are some documented circumstances under which the function returns null, then you can write a program which produces those circumstances and expects the null value. That program will break if there is no more null value. The only way it can be a non-breaking change is if there are no such circumstances; the function never actually returned null, but only threatened to do that in its documentation or the way it was declared, or both. In fact, changing a function that returns a string to one that returns a string or nil, can be a realistically useful non-breaking change. It can be a non-breaking change if the null value is not produced for any client which adheres to the existing API documentation. So that is to say, the circumstances by which the function returns null are entirely new. For instance suppose we have a function like this: number identity(number) It's an identity function, that only works for numbers. We cannot call identity(nil); that is an error. Since it returns number, it will never return nil. This is a totally non-breaking change: any identity(any) the function now just returns its argument, no matter what that is. It will now return nil---but, only if invoked as identity(nil), which was previously erroneous. The conditions are paramount. When we analyze the call, the conditions are easy to think about: is the situation that the program is passing nil, which was previously not allowed, or not? When we analyze the return, it's not so easy: we have to think about: is this returning nil under existing conditions under which it previously could not have done that? Or is it only returning nil under new conditions? The conditions which pertain to the call are always those of the call. The conditions which pertain to the return are also those of the call, plus the semantics of the function!

77 points by Capricorn2481 3y ago

patrickthebold 3y ago

> You are changing the behavior of the function. That is a breaking change. That makes almost everything breaking.

nimih 3y ago
Well, yes. Any time you change a function’s behavior, you’re risking breaking the behavior of its callers. That’s just the nature of programming and doesn’t seem like particularly controversial statement, honestly.
- Tcepsa 3y ago
  One of the big benefits of using functions is encapsulation: the idea that you (as the caller) do not need to understand how a function does what it does, you just need to know what it does (perhaps along with some performance guarantees so you know it’s not going to do e.g. an O(n^2) operation). Beyond that, I don’t want to care and should not generally have to care about what the function is doing. As long as I get out (including side effects) what I expect based on what I put in, the internal behavior of the function can change any which way and I would not consider that breaking, because my code—using that function—would still run just fine.
  - nimih 3y ago
    Yes, I agree. I probably should've been more clear that when I said "behavior," I meant "behavior observable by the caller." Going back to the original article, it seems reasonable to me that the following qualifies as such a user-observable change: foo(null) # throws InvalidArgumentException updated_foo(null) # the same as foo(0)
    - Tcepsa 3y ago
      Thank you for the clarification; that's a much more compelling argument, and I think I am inclined to agree. I believe, however, that that is not what Hickey is saying. He's not saying "previously you could pass null and it would throw an exception, and now you can pass null and it won't throw an exception". He's saying "Previously if you tried to pass null you'd get a compiler error, but then I figured out that the method as written can handle nulls, so I changed the type signature to allow null to be passed in without the compiler throwing an error about it. See the updated documentation to understand what it does if you pass in a null value." This isn't a breaking change because up until that point any code that was written to use that method already wasn't passing null (because it couldn't, because it wouldn't compile). The method's behavior hasn't changed, just its type signature, and so for any of the arguments that the existing code might possibly pass to it, it will still handle all of those exactly the same as it would have before (because, again, the implementation did not change). Therefore, it is not a breaking change.
- tshaddox 3y ago
  Then why talk about “breaking” changes? Just say “changes.” Of course, the reason we use the phrase “breaking changes” is that we are making a distinction between certain types of changes.
  - hansvm 3y ago
    Whichever line in the sand you draw as an API boundary, if the break doesn't propagate across it then the change isn't breaking in that context. Suppose (as a crude example to simply illustrate the point), a codebase exposes some CRUD operations. Internally it relies on sorting methods and comparator methods for that sorting. If you reverse the order that sort works and also reverse all the comparators then internally you've made a bunch of breaking changes, but externally you can describe the change as non-breaking (probably -- performance regressions can be tricky). The change itself was both breaking and non-breaking, and the choice of description depends on who you're describing it to.
    - tshaddox 3y ago
      > externally you can describe the change as non-breaking (probably -- performance regressions can be tricky) Yes, in most programming languages/environments it is always possible to write your application in such a way that any change of a dependency is I’ll break your app. Heck, you could throw an exception if someLib.version != “1.0.2” and then complain that a patch to 1.0.3 is actually a breaking change. Or your app could test that some API method does not exist, then complain when that method gets added later. Or you could complain when performance improvements reveal race conditions in your app (or break your usage of your computer as a space heater).
  - nimih 3y ago
    You can improve performance without changing the function's interface. Or remove a bug which causes an unrecoverable error. Or fix a memory leak. Or improve the error message on an internal assertion. Or add new methods to an object's interface. These would not change the extant observable behavior for most problem domains (although you could probably reasonably argue that any changes in performance characteristics are potentially breaking changes in e.g. game development), and are thus not "breaking", but are nevertheless useful changes.
kazinator 3y ago
A breaking change occurs if the function (before the proposed change) can be called in a way that doesn't crash (stop the program, or render it intoperable, or blow up at compile time or basically bring the show to a halt), and after the change, that call yields a different result or has a different effect or output. We can divide those breaking changes into two: 1. The call obeys the specification. 2. The call, though apparently successful, circumvents the specification, relying on undocumented behavior. The question is how much we care about 2, and there is no 100% answer. There can be situation in which some kinds of 2 breakages are such that we care about them more than 1 breakages. Suppose there is a certain correct, documented way of using the API, but almost nobody out there uses it that way. And suppose there is some undocumented way of using the API, which millions of installations use, thousands of times per second. Suppose we need to implement something new, or even ore importantly, fix a critical bug, and suppose that the work boils down to either breaking one, or the other. It may be better to break the former to keep the latter working (and possibly elevate the latter to documented status). In other words, in a perfect world we'd like to say that breakages of type 1 are non-negotiable, whereas 2 can be debated. But we can't even do that.

draw_down 3y ago

No, he’s right. he’s speaking only of what is required and what is provided, all else being equal. The what and the why are different. If the function started to provide different results for the same arguments, that’s different from what he’s talking about and would be a breaking change. Worrying about the internals of the function is a violation of encapsulation. We care what we provide and what we receive back.

one-more-minute 3y ago
I mean, the article's logic is effectively that you can break code without the type system helping or warning you. I don't think Hickey would disagree with this – it's arguably a point in his favour.
joeatwork 3y ago
I think the poster means that adding null is a big, substantial change to the interface - if you have a function that used to not make sense with a null argument, and then it suddenly does make sense with a null, then the function has changed in a way observable to callers.
- sanderjd 3y ago
  > the function has changed in a way observable to callers. Observable to new callers who might want to pass None, but not to the existing callers who don't pass None as it is because it was not possible prior to the change.
- draw_down 3y ago
  [dead]

XVincentX 3y ago

The article is wrong. Rich Hickey is talking about the surface of the function, interface. The implementation details do not matter in this case.

riffraff 3y ago
The article is claiming that the surface and the implementation are related. If client code passed a null value before, an error would have occurred. Now it is handled with a default that the original code was not accounting for, and this might be bad. Thus the change in interface _is_ a change in behavior.
- saulpw 3y ago
  Which no client should/would have been using (given the error they would have received). Why would this new implementation be presumed incompatible, when any reasonable person would think this is an addition to the interface, and not a difference to the existing interface?
  - omnicognate 3y ago
    > no client should/would No client could, even. He's talking about static types so it wouldn't have been possible even to express a caller not providing a value.
- wk_end 3y ago
  If the code passed in a null value before, it was violating the contract the function provided. A component isn't responsible for what happens when you operate it out of spec. If I were a hardware designer and some changes to an IC I'm working on are going to make the next batch, say, work properly in hotter temperatures than it could before, and one of my coworkers comes and says, "Don't make that change, what if someone using that IC is deliberately operating it out of spec expecting it to fail and now the behaviour is going to change!" I'm going to start polishing up my resume, because I'm working for an organization that employs lunatics. Happily, that wouldn't happen in the hardware world; unhappily, I'm a software developer rather than a hardware designer.

GrumpySloth 3y ago

I agree with Rich Hickey here and I think this is a flaw of sum types compared to union types. If Option<T> was a union type, T would be a subtype of Option<T> and those changes wouldn't be breaking. When writing Rust, I find myself repeatedly writing implementations of the From trait for enums, just because most of the time I actually want union types, not sum types. I think sum types should be built on top of union types by combining union types with a "lexically-scoped newtype", if that makes sense (i.e. they wouldn't be built in, they would be the result of two orthogonal features, while allowing for other combinations as well). Edit: another nice consequence of this design would be None being its own type, which can be transparently converted to an Option<T> for any T, allowing for type inference to go in only one direction, while still avoiding the boilerplate of Option<T>::None. Unidirectional type inference would make for more intelligible compiler errors. Full Hindley-Milner can get confusing.

mjburgess 3y ago
Whenever i design a lang, i just give option semantics to types of the form `x|none`
kibwen 3y ago
> which can be transparently converted to an Option<T> for any T, allowing for type inference to go in only one direction, while still avoiding the boilerplate of Option<T>::None You don't need to qualify the None, the following works: fn foo<T>(x: T) -> Option<T> { None }
- yakubin 3y ago
  Yes. That's thanks to Hindley-Milner type inference which makes the compiler infer which None you mean. But in longer code snippets full Hindley-Milner type inference can cause hard-to-understand type errors. Here specifically the compiler infers the type of a value in your function body based on the type of the function it belongs to, which is in the opposite direction than what a programmer typically thinks in. It leads to spooky actions at a distance.
  - 3y ago

jhardy54 3y ago

Counter-point: consider the case where the body of the function is only `return value`.

codebje 3y ago
Isn't that an argument that the function implementation might break, not that callers might? Assuming implicit conversion for union types, you could either widen argument or narrow return without changing that implementation, but not both. A different implementation may not be able to handle either modification, though.

fiddlerwoaroof 3y ago

The definition of breaking change here completely destroys any idea of abstraction or bug fixes. If calling a function requires knowing how the function is implemented, then the function should not exist in the first place: the whole point of an abstraction is that, if the use-site fulfills the precondition, it can assume that the postcondition holds after it used the abstraction. This means that relaxing the precondition is, by definition, not a breaking change because the old inputs are a subset of the new ones. Similarly, strengthening the postcondition isn’t a breaking change because the new outputs are a subset of the old ones. (If and he use-site assumes that it gets a specific member of the function’s codomain for specific input values, you have an abstraction leak and the use-site has a bug [or the function should be inlined because it’s a false abstraction])

_madmax_ 3y ago
I agree with you wholeheartedly
sanderjd 3y ago
My first thought was, "I wonder what the author considers a non-breaking change..."
- squirtlebonflow 3y ago
  Comments, updating dependencies, or documentation
  - sanderjd 3y ago
    Updating dependencies could way more easily be a breaking change than what we're discussing... And comments and documentation aren't code changes at all.
hn_throwaway_99 3y ago
I really wish I had the ability to downvote articles. Agree with you 100%, and I feel like I lost braincells from reading this sentence: > You are changing the behavior of the function. That is a breaking change. So this guy is declaring someone else "wrong" because he unilaterally decides to redefine what a "breaking change" means? This is one of the dumbest "all Internet arguments are over semantics" examples I can think of. As an aside, I've seen a number of recent instances (here's another, https://news.ycombinator.com/item?id=36826111 https://news.ycombinator.com/item?id=36826111) where a flat out ridiculous blog post makes it to the front page, only for the top comments to all point out how dumb and ridiculous it is. To stay within the HN guidelines, I'm going to refrain from speculating why, and I certainly know that HN audience is not some monolithic group, and that people have many different opinions. Would just like to know if others have noticed this or have other specific examples.

easeout 3y ago

It's only considered "breaking" to change the externally observed behavior of at least one previously published use of a code interface. When widening the interface's range of inputs, you can avoid changing existing behavior. We do that all the time when we create a new optional parameter, new CLI tool switch or subcommand, new public function or type in a library, new API endpoint, new web domain, new network protocol.

alexmolas 3y ago

Isn't what Rich Hickey defense the same as the Liskov substitution principle? Or at least very similar.

codebje 3y ago
Somewhat related. The subclass relationship isn't very rigorous, you're free to have completely different behaviours out of the same methods in a subclass. Liskov says don't do that: if B is a subclass of A it should behave like an A. When it comes to types, a subtype is one that can be transparently substituted for its super type. If S <: T, a term of type S can be used where a T was expected just fine. For union types, it's a given that T <: T|a for any type a, and a T will behave exactly like a T|a whose value is a T. The Liskov principle states that a subclass should be a subtype.

krick 3y ago

Others already commented on why this complaint doesn't make any sense, but, hey, at least I was pointed to a Rich Hickey talk I've missed. It never occurred to me before that "Maybe" isn't actually a "maybe", but Hickey is right. Seriously, every single talk of his I've seen so far has something revealing in it. I should go search for all his presentations I've missed, I guess.

coldtea 3y ago

>If there wasn't a change in behavior, why are you changing the contract? Because you're restricting or relaxing the types, and you want to type system to know this. This doesn't have to translate to a change in behavior.

Gunax 3y ago

I feel like they are talking past each other, or at least talking about different things. Hickey seems to be talking about breaking the compiler. That is, he is saying that changing the return signature from null|int to int should not break the compilation. In effect, this would create a warning a la 'You are checking for null, but var x cannot be null'. The author seems to be talking about breaking changes in the behaviour/operation sense. Those are different types of breaking.

scubbo 3y ago
But there _is_ no "breaking" change in the behaviour, because none of the previously-possible behaviours _can_ break. For every way-of-calling the function that previously existed, those ways-of-calling will continue to operate as before. Some new ways-of-calling will now be legal, but that is not a breakage, because no code that already existed could have been using those ways before, and so no code exists that can be broken. Clients _may_ choose to write new code to take advantage of the new behaviour - but there is nothing that they _must_ do (unlike with a breaking change, where they _must_ take action). It's for the purposes of this categorization, for communicating about client's responsibilities, that the concept of breaking changes is valuable.
- anonymouskimmer 3y ago
  Hijacking this to respond to a reply you made 40 days ago to a comment I made 45 days ago: https://news.ycombinator.com/item?id=36335405 https://news.ycombinator.com/item?id=36335405 > But that doesn't seem to be what you were, or are, saying. If you issue is specifically with "an epicurean theme park of enlightenment and joy to span the stars", and not with the dramatized reaction to (against) it, then...I must say I still don't understand what the problem is. An experience which is _definitionally_ pleasant, fulfilling, and non-harmful (to self or others) is....well, it's capital-g Good, no? Technological advance is fine on its own. The risk with "an epicurean theme park of enlightenment and joy to span the stars" is our own mental faculties, presuming no one is left behind. A variety of examples of this turning bad exist in fiction, a particularly pertinent one to me is Asimov's "The End of Eternity", and another one is the post-immortality centuries of Niven's "Known Space", but there are many others. Psychologically theme parks are meant to be visited for a break. If we spend our entire lives in them it does stuff to our capacities. In everyday life animal species spend most of their problem solving effort on dealing with other members of their species (and some similar species). We are our own greatest competitors. In part, I think animals do this because members of each species are approximately on equal footing, so our individual problem solving is sufficient to deal with other individuals, and our group problem solving sufficient to deal with other groups of us. But nature itself, and the universe itself, is the biggest player. The biggest risk. I fear that themepark life, as opposed to themepark visit, will either spark ennui and a bunch of interpersonal shenanigans, or will invite people to forget that we're just living in a fragile bubble. If the opportunities are not equal for every person, the situation is even worse. And I like to think that we have a responsibility to our co-inhabitants in the universe, the other species. A themepark life ultimately risks navel-gazing (at the individual or group level) and ignoring these responsibilities as well. No thank you. I want a nice life, and a nice place to live it. But I don't want ennui. I don't want self-involved navel-gazing. I want people paying attention to the real issues in the universe. We spend enough effort already on dealing with other humans. > What is it about the real world that is more important, inherently, than "the experiences that arise from living in it"? If a "better" (I'm hand-waving the complexity of comparison, because it's assumed as part of the discussion) set of experiences can arise - with perfect certainty, no trade-offs, no utilitarian cheats, just "everything is better for everyone" - then, as the other commenter said, it would be abhorrent to deny it 'because of some philosophical quibble about a difference between “simulation” vs “reality”'. Sure. I agree. As long as we take pains to address the inevitable externalities before they impact others (including other species) this is fine. But make sure you know what's "better" for everyone before it is implemented. It's not obvious that even a god could manage this, much less humans (or even the humans/angels/demons of The Good Place). "What is it about the real world that is more important, inherently" - It's everything, and we're just a part of it. "importance" is a value judgement, so is inextricably caught up in the individual so judging, but subtract out the value judgement and it's obvious that a human and a squirrel are more "important" than a single human. Now expand this to the "real world" entire.

gavinhoward 3y ago

I've read the article and watched the video that it is a reply to. I both agree with the author and disagree. I disagree that Rich Hickey is wrong when it comes to whether those changes should be breaking or not. Those curves can be non-breaking, unlike what the author of the article claims. But I agree with the author that the fact that they are breaking doesn't really matter. Rich Hickey mentions this himself in his talk. He says that no one talks about the costs of using Option or Maybe. So then he lays out the costs. And I was...not that impressed. Sure, this change that should be non-breaking is breaking. And your downstream clients will have to change their code. That seems bad. Until you realize that the only change they would be forced to make is to delete what is now dead code. Sure, it's annoying; I don't deny that. But Rich Hickey claims that it increases code maintenance to have to make the change. I whole-heartedly disagree because any time you can safely delete code, you are making maintenance easier. Plus, despite all of his complaints that people don't considers costs, he never really considers the costs of his proposal, the biggest of which is the complexity of the language. I think the industry has rightly come to the conclusion that, all else being equal, and even sometimes when they are not, a more complex language is a worse language. I think proper union types, the kind needed here, would add enormous complexity to the language for little benefit in rare cases. (Because how often do programmers relax constraints? I don't think they do that often.) In my language, I'll keep Option as it is, and make such changes breaking, in order to avoid the complexity of union types.

pyrale 3y ago
> I think the industry has rightly come to the conclusion that, all else being equal, and even sometimes when they are not, a more complex language is a worse language. That was a theory behind golang, and the observed result is that you're better off having a little bit more language constructs that may be misused, than having essential complexity in your problem space that you can't easily describe in your solution space, for lack of tools.
- gavinhoward 3y ago
  That's why I said "all else being equal." Go took it too far.

saghm 3y ago

If I were changing the signature of `do_something` in Rust and wanted to avoid breaking callers when changing the function from accepting `T` to `Option<T>` , I'd do the following: fn do_something(value: impl Into<Option<T>>) { let value = value.into(); // remaining function body using `value` as an `Option<T>` } Existing calls to `do_something` will continue working because `Option<T>` implements `From<T>` (and `Into` implementations are generated in the opposite direction whenever `From` is implemented). Passing in `None` will also work because every type `T` implements `From<T>`, and therefore `Into<T>` as well. I'd even argue that in a lot of cases, this is worth doing even when creating new library functions (where there are no existing callers to worry about). In my experience, it's generally better to keep the API clean from a caller's perspective by taking on extra boilerplate internally in the implementation rather than shift the burden to the callers of the API in order to keep the library's implementation clean.

skavi 3y ago
Often better to do the conversion then immediately call a non generic inner function. This prevents the rest of the logic from getting monomorphized. It’s a pattern you’ll notice in std.
skitter 3y ago
This is a breaking change too, as it can break type inference: https://play.rust-lang.org/?edition=2021&gist=8d528bcec2b92b73780d3335d936dfcb https://play.rust-lang.org/?edition=2021&gist=8d528bcec2b92b...
- obi1kenobi 3y ago
  Interestingly, even though it's a breaking change, breaking inference is explicitly defined to not be a semver-major change in Rust: https://predr.ag/blog/some-rust-breaking-changes-do-not-require-major-version/ https://predr.ag/blog/some-rust-breaking-changes-do-not-requ...

chmod600 3y ago

Examples? What problems will be caused if we allow existing callers to keep working as before?

paulddraper 3y ago

Rich Hickey is correct. I prefer sum types. But this is a legitimate point for union types.

singularity2001 3y ago

He isn't wrong, languages that make optionals not backwards comparable are. So swift "string?" is a perfect match for his insight, whereas rust option<string > is a mess.

secretsatan 3y ago
Having run into the is recently when our cloud API changed a value we relied on to be optional, it is indeed a breaking change in swift.

MrTortoise 3y ago

I feel like you missed the main point of what you are saying. Whilst existing callers who obey the contract will not be broken by this callers who depend upon this code breaking for null values will no longer do so. so it is a breaking change for them. You have just extended their behaviour. That said if we defined the functionality of code as being valid within a scope of input and not defined anything outside of this we are safer in a sense. That said you do require feedback to show you are out of range. But the point I want to make is that the line of argument that attempts to ascribe changes to behaviour on values that were out of scope as breaking changes is not helpful to anyone who is trying to co-exist via contracts in order to understand what work is expected as a result of change.

ndr 3y ago

Hard to disagree with Rich Hickey re non-breaking changes. See for yourself the "Fig. 3. Clojure codebase—Introduction and retention of code" chart at page 26 of https://download.clojure.org/papers/clojure-hopl-iv-final.pdf https://download.clojure.org/papers/clojure-hopl-iv-final.pd... Rich Hickey is right and has a track record to show.

jbritton 3y ago

I came away with a different interpretation of Rich Hickey’s talk. I think he was advocating against using slots, where a slot would be a struct field. A slot has to have a value and if a value is not available then null is put there and now null must be dealt with. Instead I think he was advocating for using dictionaries. You either have a key or you don’t, but you never use null. However it seems that now you would have a bunch of “key in dict” checks which are less efficient than “value is null” checks. Plus confusion about what exactly has been supplied in a dictionary. Although there may be cases where code can just sequence through what exists without worrying about what doesn’t exist.

Sakos 3y ago
I think the fact that it's impossible to know what could be in a dictionary without explicitly checking makes dicts an awful solution. A lot of times I'll encounter functions that take maps, but I'll have to reverse engineer the surrounding code (and if I'm particularly unlucky, a related service) to figure out what goes in there. I'll take explicit types over dicts any day.

darkavenger42 3y ago

This careless approach with nulls kinda works for Clojure because of "nil punning" but is dangerous for other languages I guess

AtNightWeCode 3y ago

Rick is right I believe. But why not just use overloading instead? That is what is done in many OO-langs to avoid making breaking changes.

hansvm 3y ago

Language features like type promotion or automatic casting to default/zero values are likely to change how the new function is called, purely based on the change in type signature. Even if every old input has the same output in the new function, old code can compile to produce a new input and have broken behavior. That's ignoring any metaprogramming or more complicated type shenanigans. Is there something special about Clojure making that change safe?

layer8 3y ago
I don’t know about Clojure specifically, but Java and the JVM have bridge methods for such purposes. Meaning, a hidden method with the old type signature would still get generated whose implementation transparently forwards to the new method with the changed type signature, e.g. converting the T value to an Optional<T> value (or whatever).

kazinator 3y ago

The changed return convention is potentially a breaking change. If there are some documented circumstances under which the function returns null, then you can write a program which produces those circumstances and expects the null value. That program will break if there is no more null value. The only way it can be a non-breaking change is if there are no such circumstances; the function never actually returned null, but only threatened to do that in its documentation or the way it was declared, or both. In fact, changing a function that returns a string to one that returns a string or nil, can be a realistically useful non-breaking change. It can be a non-breaking change if the null value is not produced for any client which adheres to the existing API documentation. So that is to say, the circumstances by which the function returns null are entirely new. For instance suppose we have a function like this: number identity(number) It's an identity function, that only works for numbers. We cannot call identity(nil); that is an error. Since it returns number, it will never return nil. This is a totally non-breaking change: any identity(any) the function now just returns its argument, no matter what that is. It will now return nil---but, only if invoked as identity(nil), which was previously erroneous. The conditions are paramount. When we analyze the call, the conditions are easy to think about: is the situation that the program is passing nil, which was previously not allowed, or not? When we analyze the return, it's not so easy: we have to think about: is this returning nil under existing conditions under which it previously could not have done that? Or is it only returning nil under new conditions? The conditions which pertain to the call are always those of the call. The conditions which pertain to the return are also those of the call, plus the semantics of the function!