14 ms·
Equality Is Hard
- lazulicurio 7y agoAs much as null is considered the "billion dollar mistake", personally, I think that Java giving Object a virtual equals() and hashCode() was an even bigger mistake. NPEs blow up loudly in your face and are fairly easy to lint for. By comparison, bad equals() and hashCode() implementations can cause difficult-to-track-down errors. Unfortunately, this design was inherited by C# and other OO languages. C# does have the IEqualityComparer interface, which helps in some cases, but many containers and other framework methods rely on an overridden instance equality method.
- mjevans 7y agoI agree. It might make sense to use something like memcmp() on structs to assert that they are exactly the same, or to have a dedicated comparison function that makes sure they are functionally equivalent for some defined need. However Objects should have only the second and it might also need to be (I think replacing a function pointer might be) an over-loaded function if something has extended a base object type. I'm not even actually positive that could work if there are two extensions to a base type.
- usrnm 7y ago> It might make sense to use something like memcmp() on structs to assert that they are exactly the same No, actually, it doesn't make sense in general. Due to padding, structs may have arbitrary meaningless bytes between the actual data fields, and memcmp will break on those. It works because structs are often filled with zeroes on initialization, but it isn't required or enforced in any way. I was burnt by this problem more than once, not really fun to debug
- Gibbon1 7y agoFun, create a struct on the stack. Set the elements individually. Padding is now filled with garbage. Any elements you forget to set will also be garbage. When you copy the struct the garbage goes with it. I've been burnt too.
- mjevans 7y agoYou've missed the 'functionally equal' check option; which covers cases like you're focusing on. To check if two differently created things happen to be equal, yes, you should prefer the function that checks exactly what matters exactly how it matters. As you rightly point out, the memcmp check would only work on either pre-zeroed memory (depends on environment and what you're doing) or actual literal copies of a specific struct.
- barrkel 7y agomemcmp() on your structs means any fields of a reference type are back to using reference equality. You need to recurse if you're going to be consistent.
- deleted 7y ago[deleted]
- deleted 7y ago[deleted]
- WalterBright 7y agoC's actual billion dollar mistake was losing array bounds with every function call: https://www.digitalmars.com/articles/b44.html https://www.digitalmars.com/articles/b44.html
- hinkley 7y agoThere's something interesting about 'sortBy' semantics which you see in some languages. As the cost of your comparator goes up (especially on real hardware with pointer chasing) the difference between running the comparator m * nlogn times and running the transform m*n times could start to stack up.
- kstenerud 7y agoThe mistake wasn't in giving Object those functions. You get "=" behavior for all types in a given language, and it's nice to be able to control this somehow (i.e. they were actually thinking a little bit about the equality problem here). The problem came when they decided that the default behavior should be a pointer comparison rather than a member-by-member sub-equals. As for hashCode(), it should have thrown an exception as the default behavior, because there's no way a language designer can anticipate what makes a proper hash for class XYZ.
- barrkel 7y agoRecursive by default (which is what member-by-member implies) leads trivially to infinite recursion in reference loops, which your default equals would need to guard against, greatly increasing its complexity. Reference equality is generally fine for mutable types, you probably only want value equality for immutable types. If you use value equality for mutable types, you start getting into trouble when you use them as e.g. hash keys.
- kstenerud 7y agoYou are correct, but it's far better to have expensive-by-default but safe operations vs the counterintuitive mess you end up with when comparing references by default. The point is that you need to counter human nature, which is to not do work you can get away with not doing. If the worst case result of not doing the work (i.e. not creating an equals method) is slow performance, that's better than getting incorrect results. You'll have a much harder time designing a useable language if your understanding of human psychology is poor. Besides, in 90% of cases, you won't have potentially recursive types, and so a smart compiler can optimize for this when generating a default equals method. In fact, many cases could just degenerate into a memcmp.
- renox 7y agoI'm not so sure, when Java was created computers weren't that fast.. Of course, now with computers fast enough that Python and JavaScript are king, sure!
- smallnamespace 7y agoI’m surprised the article discusses extensional equality for functions, then doesn’t mention why languages don’t try to derive it: it’s equivalent to solving the Halting Problem (per Rice’s Theorem).
- JadeNB 7y agoI think Rice's theorem is a big hammer here: it's equivalent to the halting problem because one can just test extensional equality of given M: simulate M; return 1 and given M: return 1 Or is it even true of extensional equality for _total_ functions?
- jerf 7y ago"I think Rice's theorem is a big hammer here" It is certainly a big hammer, but as theoretical computer science tools go, it is pleasingly easy to use. Unfortunately it is pleasingly easy to use in much the same way high explosives are pleasingly easy to blow yourself up with, as it annihilates so many things that we'd rather like to keep. But, still... easy to use.
- guerrilla 7y agoThat's only for Turing complete languages. It's not true for total functions. Total functions can be written in Coq, Agda and Idris among others.
- TheAsprngHacker 7y agoFunnily, in programming languages based on Martin Lof Type Theory, the topic of function extensionality is a whole rabbit hole for reasons other than the halting problem. There is a distinction between intensional Martin Lof type theory and extensional Martin Lof type theory, which differ in their treatments of equality. First, one must distinguish between judgmental equality x = y, which is a statement in the mathematical metalanguage that two terms are equal, and propositional equality, which "internalizes" judgmental equality within the programming language through the equality type Id(A, x, y), the type of proofs that x and y are judgmentally equal. In extensional type theory, judgmental equality also follows from propositional equality, and together with the eta-conversion rule for functions, function extensionality is provable [0]. However, typechecking extensional type theory is undecidable [1]. Alternatively, in Homotopy Type Theory (HoTT), supports function extensionality is provable from the univalence axiom, which states that isomorphic types are equal [2]. Function extensionality is trivially provable in Cubical Agda [3], which gives a computational interpretation of HoTT's notion of equality. [0] https://cstheory.stackexchange.com/questions/46331/extensional-type-theory-and-function-extensionality/46342#46342 https://cstheory.stackexchange.com/questions/46331/extension... [1] https://cs.stackexchange.com/a/112559 https://cs.stackexchange.com/a/112559 [2] https://ncatlab.org/nlab/show/function+extensionality#relation_to_the_univalence_axiom https://ncatlab.org/nlab/show/function+extensionality#relati... [3] https://agda.readthedocs.io/en/v2.6.0.1/language/cubical.html https://agda.readthedocs.io/en/v2.6.0.1/language/cubical.htm...
- earthboundkid 7y agoIf no mainstream programming languages implement equals like math—an obvious and well known example throughout the history of computing—maybe it’s because that’s a bad idea.
- rossdavidh 7y agoOr really hard, when you're communicating with a computer and not another human. Lots of concepts that are easy to communicate to a human, are difficult to communicate to a computer, not just equality. It is a good point, though, that if virtually EVERY mainstream language gets it wrong, when every single one of the people creating those languages knew about the math =, there has to be a reason.
- dajohnson89 7y agoand what might that reason be?
- TheOtherHobbes 7y agoIt depends what the meaning of "is" is.
- earthboundkid 7y agoPhilosophers distinguish three senses of "is": - Identity: "Clark Kent is Superman." Anything true of Superman is true of Clark Kent and vice versa, since these are two names for the same thing. - Predication: "Superman's cape is red." Red is a quality of the cape. It's not reflexive because not all red things are capes. - Existence: "There is no Superman." In English, existence sentences usually begin with "there is" or more poetically, you say "Superman is ~not~." Existence is not a relation. In the case of Clinton, he was being evasive about "is" versus "was" in the time dimension. There was a sexual relationship between him and Lewinski, but he called there "is" no relationship in general.
- moring 7y ago
- cperciva 7y agoTo correct the opening statement: The two hard problems in computer science are cache invalidation, naming things, and off-by-one errors.
- aaron695 7y agoThe [1] is worth a read - https://skeptics.stackexchange.com/questions/19836/has-phil-karlton-ever-said-there-are-only-two-hard-things-in-computer-science https://skeptics.stackexchange.com/questions/19836/has-phil-...
- dvt 7y ago> First, programming languages should make it simple to create types where equality comparison is disabled because it makes no sense This is kind of a silly "solution" as some languages don't even have types, others have "soft" types, and others support arbitrary typecasting. (Not even going to get into duck typing, union types, etc. Type theory is hard.) > Programming languages must make the difference between structural equality and reference equality crystal clear This is probably intractable. Author doesn't even touch cases where you have objects that contain other objects. Are these references or values? How does the equality operator work there? How deep does the rabbit hole go? If you only deal with values, then you're probably crippling performance. There are reasons for these design concessions, it's unfair to assume that people that designed these languages are complete doofuses. > One might look at the length of this post and say, “Wow, equality is really complicated! I’m going to give up coding and become a soybean farmer.” Farming is magnitudes harder than what the average programmer does day-to-day :)
- rossdavidh 7y agoIn fairness, he did also say that being a soybean farmer was harder.
- lmm 7y ago> This is kind of a silly "solution" as some languages don't even have types, others have "soft" types, and others support arbitrary typecasting. Well, more fool those languages. It's a good solution, and if your language can't offer an equivalent, take it up with your language's maintainers. > This is probably intractable. Author doesn't even touch cases where you have objects that contain other objects. Are these references or values? Let's not declare it intractable before we've actually tried. The language would need to draw a clear distinction between (possibly compound) values and objects-with-identity - like the distinction between struct and class in C#, or between case class and class in Scala - and ripple that distinction all the way up; values should only be allowed to contain other values, should be immutable, and should be able to be treated in a generic way (i.e. a record system). Objects-with-identity should be rarer and heavily encapsulated, almost like Erlang actors; the simplest example is a reference cell (i.e. a mutable thing holding a value of a particular type that can be get and set). Doesn't sound like such a bad language design.
- jeffadotio 7y agoI really like this about Rust, implementing equality is just another trait. The same is true of comparison operators. If you don’t want a type to be able to be compared by == you can make it always evaluate to false or even panic and give a message that the type cannot be used in that way. That might aggravate coders who do not test comprehensively but it is an option. Edit: This is a response to the concerns of the article. For general info on this it’s a great introduction to Rust’s awesome documentation. https://doc.rust-lang.org/std/cmp/trait.Eq.html https://doc.rust-lang.org/std/cmp/trait.Eq.html
- tus88 7y agoYou mean there is no compile time way to accomplish that? Doesn't sound very Rust-like.
- jeffadotio 7y ago“Option” is the operative word. A type that does not implement the correct trait cannot be used in a way that requires the trait. If you are writing anything non-trivial that would probably be preferable. It requires nothing, that is the default behavior, so I did not bother to mention it.
- re 7y agoIt is determined at compile time; I'm not sure why GP is suggesting returning false if you don't want types to be compared. By default structs are not comparable. If you opt-in to the standard PartialEq implementation using the `derive` annotation[1], it will only allow a struct to be compared for equality with other structs of the same type. You can choose to add implementations to compare against other types[2]. [1] https://doc.rust-lang.org/book/ch05-02-example-structs.html#adding-useful-functionality-with-derived-traits https://doc.rust-lang.org/book/ch05-02-example-structs.html#... [2] https://doc.rust-lang.org/std/cmp/trait.PartialEq.html#how-can-i-compare-two-different-types https://doc.rust-lang.org/std/cmp/trait.PartialEq.html#how-c...
- artursapek 7y ago
- dnautics 7y agoI believe the erlang VM and the languages on top of it get all four conditions correct (not counting rebinding a variable to a new value, which you can do in some BEAM languages)
- vadiml 7y agoHm... let john = { Name = "John"; Age = 15; Offspring = [] }; let jane = { Name = "Jane"; Age = 47; Offspring = [ john ] } let sue = { Name = "Sue"; Age = 35; Offspring = [ john ] } Not very realistic :)
- ldeangelis 7y ago> Never use == in JavaScript. Use === instead. Or better, know what == [0] and === [1] do by looking at the ECMAScript reference [0]: https://www.ecma-international.org/ecma-262/10.0/index.html#sec-abstract-equality-comparison https://www.ecma-international.org/ecma-262/10.0/index.html#... [1]: https://www.ecma-international.org/ecma-262/10.0/index.html#sec-strict-equality-comparison https://www.ecma-international.org/ecma-262/10.0/index.html#...
- marcus_holmes 7y agoI know that opening line as "there are two hard problems in computer science: cache invalidation, naming things, and off-by-one errors"
- mar77i 7y agoI came here for this, too. You're not mistaken.
- capableweb 7y agoI'm pretty sure the original statement is just with "cache invalidation" and "naming things", and later the "off-by-one errors" was added, as when I first learned about this quote, it was never with the "off-by-one errors" parts but nowadays, seems to always include the three of them.
- bovermyer 7y agoI have to admit that I expected an article on a different type of equality before I clicked on the link.
- AnimalMuppet 7y agoI, uh, equally expected that.
- jerome-jh 7y agoIs reference equality ever useful in itself, in a high level language? Of course reference equality is a straightforward optimization for structural equality, as mentioned in the post. But it is not required to make it visible to the programmer. Reference equally is mandatory for the garbage collector. Again it may not be visible to the programmer. Finally there is the case of two data structures sharing a reference to a single instance of some data. Usually that is a bad idea if the pointed to data is mutable, and often makes the program leaky. The paper reads: "So reference equality is a sensible default in a language which supports mutation.". Is it not a bad reason to support reference equality?