7 ms·
Hyrum's Law
- azhenley 1y agoDiscussion last year: https://news.ycombinator.com/item?id=39401973 https://news.ycombinator.com/item?id=39401973
- dang 1y agoThanks! Macroexpanded: Hyrum’s Law in Golang - https://news.ycombinator.com/item?id=42201892 https://news.ycombinator.com/item?id=42201892 - Nov 2024 (183 comments) Hash Ordering and Hyrum's Law - https://news.ycombinator.com/item?id=41673295 https://news.ycombinator.com/item?id=41673295 - Sept 2024 (41 comments) Hyrum's Law - https://news.ycombinator.com/item?id=39401973 https://news.ycombinator.com/item?id=39401973 - Feb 2024 (66 comments) Git archive generation meets Hyrum's law - https://news.ycombinator.com/item?id=34631275 https://news.ycombinator.com/item?id=34631275 - Feb 2023 (76 comments) Hyrum's Law - https://news.ycombinator.com/item?id=33283849 https://news.ycombinator.com/item?id=33283849 - Oct 2022 (52 comments) Hyrum's Law - https://news.ycombinator.com/item?id=29848295 https://news.ycombinator.com/item?id=29848295 - Jan 2022 (36 comments) Hyrum's Law - https://news.ycombinator.com/item?id=27386818 https://news.ycombinator.com/item?id=27386818 - June 2021 (5 comments) Hyrum's Law: An Observation on Software Engineering - https://news.ycombinator.com/item?id=21515225 https://news.ycombinator.com/item?id=21515225 - Nov 2019 (6 comments) Hyrum's Law - https://news.ycombinator.com/item?id=19249199 https://news.ycombinator.com/item?id=19249199 - Feb 2019 (1 comment)
- AdamH12113 1y agoFrom an API designer's standpoint (especially if that API has paying customers), Hyrum's Law is something that has to be taken into account. But from a user's standpoint, it is engineering malpractice, plain and simple. At the very least, relying on quirks of someone else's implementation is a risk that should be understood and accounted for, and no one has any reasonable grounds for complaint if those quirks suddenly change in a new version.
- kccqzy 1y agoMalpractice? It's usually just plain old bugs unintentionally written that way.
- breppp 1y agoDepends on the product. Sometimes you are completely dependent on an API ecosystem (iOS, Android, Windows) where the only way to achieve something is a quirk
- Ygg2 1y ago> But from a user's standpoint Not true generally. One man's engineering malpractice is another man's clever hack. Users of Windows 95 complained that Windows 95 broke SimCity. What did Windows 95 break? It fixed an obscure allocator bug SimCity was relying on. Users loved Windows 95, for ""fixing"" this. How was it fixed? By introducing an obscure switch to old allocator if it detected SimCity in the app name. https://arstechnica.com/gadgets/2022/10/windows-95-went-the-extra-mile-to-ensure-compatibility-of-simcity-other-games/ https://arstechnica.com/gadgets/2022/10/windows-95-went-the-...
- bigstrat2003 1y agoDifferent users. The users that GP was accusing of malpractice would be the Maxis devs in this case, not the end users who were trying to install SimCity on their Windows 95 machine. Microsoft has a commitment to backwards compatibility that I think is going too far, but I understand why. Raymond Chen has explained that if a user buys the new version of Windows and their programs stop working, they will blame MS regardless because they don't have any way to know it's the program's fault. So MS is incentivized to go out of their way to enable these other programs' bad behavior, because it keeps their (Microsoft's) customers happy.
- Ygg2 1y ago> Different users. But same principle. Everyone loves for someone else to maintain backwards (and forwards compatibility). > The users that GP was accusing of malpractice would be the Maxis devs in this case, not the end users who were trying to install SimCity on their Windows 95 machine And Windows devs aren't to blame for backporting buggy allocators? If SimCity depended on buggy behavior was malpractice, what the hell is backporting bugs? Exporting malpractice?
- nvader 1y agoIt seems to me that there's some advantages to undertaking "Freedom of Navigation Operations" by randomizing implementations from time to time to discourage any dependence on internal behaviors. For instance, traversal order of maps in Go is always randomized, to prevent subtle bugs caused by depending on the order. As AI generated code becomes cheaper, it may be worthwhile to change some subset of your internal behaviors from release to release, so that users don't become too complacent.
- avidiax 1y agoThere's a corollary: Even if you explicitly deny a guarantee of a certain behavior in your contract, if you usually deliver that behavior, most of your customers will depend on it. Some examples: If you make a queueing system, it's impossible to guarantee anything other than delivery "at most once" (some loss occurs), or "at least once" (some duplication occurs), but if you usually provide "exactly once" in practice, most of your customers will depend on this. If you provide a data bucket service, and guarantee availability, but not performance, and you usually provide 100MB/s throughput, your customers will have major problems if you only provide 10MB/s throughput in some cases. If you make a self-driving car, and it requires human monitoring, but it's really good, say one intervention per year of driving . . . your customers will die because they aren't paying attention.
- dongkyun 1y agoThis is explicitly recognized in contract law: course of performance / dealing is a factor courts will consider in evaluating the nature of a deal. (Most contracts will try and carve it out).
- chuckadams 1y agoCan't disagree with anything you said, though I think there are steps to address at least some of them: for queueing systems, testing with a chaos monkey isn't a bad idea... you'd want a test environment representative of production workloads, which is hard to do, but anything should be better than nothing. In the self-driving car scenario, you'd probably go with cold statistics: is it killing fewer people than ones that need more interventions? Just like queueing though, experiments in production could be problematic.
- worik 1y ago> In the self-driving car scenario, you'd probably go with cold statistics No. There is a big difference in an accident caused by human error and an accident caused by machine failure. We tolerate much more of the former than the latter. This feels like a cognitive failure, but I do not think it is
- 1y ago
- TehCorwiz 1y agoXKCD always has a relevant comic: https://xkcd.com/1172/ https://xkcd.com/1172/
- xatax 1y agoThat's linked in the OP.
- amelius 1y agoThis is why an API should always have an "assertions": true option. Why should normal function calls have assertion/invariant checks, and not API calls?
- porridgeraisin 1y agoThis idea looks good. Have you used it in practice? Can you share how?
- amelius 1y agoYes, you basically use the option whenever you have assertions turned on in your code. Then the service running the API will do extra checking when the assertions option is true, basically making it less forgiving and following the specification closely.
- prats226 1y agoThis is super interesting to think about in LLM world where lot of software is getting replaced with LLM calls. In terms of output of an LLM, there is no clear promise in the contract, only observable behaviour. Also the observable behaviour is subject to change with every update in LLM. So all the downstream systems have to have evals to counter this. One good example is claude code where now people have started complaining them switching models effecting their downstream coding workflows.
- worik 1y agoYes. This is the unfortunate thing about wrapping LLMs in API calls to provide services. Unless you control the model absolutely (even then?) you can prompt the model with a well manicured prompt on Tuesday and get an answer - a block of text - and on Thursday, using the exact same prompt, get a different answer. This is very hard to build good APIs around. If done expect rare corner case errors that cannot be fixed. Or reproduced.
- fsmv 1y agoA good example of defence against this is go maps randomize iteration order just so that people don't rely on it being consistent.
- kiitos 1y ago> if an interface has enough consumers, they will collectively depend on every aspect of the implementation ... Yep! > [and that] constrains changes to the implementation, which must now conform to both the explicitly documented interface, as well as the implicit interface captured by usage Nope! Software authors define the rules for the software that they author. I understand it's a spectrum and the rules are different in different circumstances but at the end of the day my API is what I say it is and if you rely on something that I don't guarantee that's on you and not me. Hyrum's Law describes a common pathology, it doesn't define an expected rule or requirement.
- cafard 1y agoActually, I think that in The Mythical Man-Month Brooks mentioned users depending on nominally undefined, practically consistent behavior, e.g. what was left in some part of a register.
- kibwen 1y ago> This effect serves to constrain changes to the implementation, which must now conform to both the explicitly documented interface, as well as the implicit interface captured by usage. Let's be clear that this is one interpretation of the phenomenon described here, which we might call "The Doomerist Interpretation of Hyrum's Law". For everyone else, the whole reason that we bother categorize interface details into "public" and "private" buckets is precisely so we have the moral high ground to to tell people to go kick rocks when they get they get uppity about their own failure to adhere to the publicly documented interface.
- atakan_gurkan 1y agoThis reminds me of the "illegal opcodes" in Commodore 64. I believe they were even present in their entirety in Commodore 128's (which uses 8502, not 6502 as its main CPU) C64 emulation mode. If someone knows how or why they remained part of 6502's instruction set over C64's lifetime, I really would like to know. I suspect, it was deliberate since "customers relied on them".