7 ms·

Hyrum's Law

125 points by eindiran 5y ago

tomrod 5y ago
Hyrum was a good friend of mine in grad school. We spent a lot of fun sessions defining stats and KPIs to monitor on SVN projects. It doesn't surprise me to see a law named for him. If he reads HN, hello old friend!
- Brotkrumen 5y ago
  Well, that seems to be his website, so he basically is trying to name a law after himself.
  - hn_go_brrrrr 5y ago
    No: > While I may have made the observation, credit goes to Titus Winters for actually naming it as "Hyrum's Law" and popularizing the concept more broadly.
wruza 5y ago
Reminded me of the first thing I test when working with stock exchanges. Send order #1, watch for events: trade, trade, trade, executed. … Send order #5: trade, trade, executed. Oh well, works as expected, right? Can't be different. It can. Send order #27: trade, trade, executed, trade. Damn ordering got me hard a couple of times.
- tialaramex 5y ago
  You could look at Hyrum's law as cautionary: don't be that person who relies on unspecified behaviour. This might save you from some unpleasant breakage. But its role for Hyrum and his team is about their burden as engineers. Any observable behaviour they change, no matter how seemingly unimportant, will break somebody who wasn't so cautious, maybe unwittingly. Example: If you're an experienced programmer you're probably aware of stable sorts. You know if you need a sorted list to stay as it is when sorted you must use a stable sort, and you're sensitive when reading documentation - did it say this sort is stable? No? Then maybe it isn't, and I need to guard against that. But a new programmer may just assume sorts are stable without ever thinking about it. Naively they might think of course all sorts are stable, why do the extra work to move things that are in the correct place already - without understanding that common sort techniques often aren't stable. And if they at first use sorts on objects with no sense of identity, like the integers or in many languages strings, there's no sign of a problem, all sorts work the same. [1,2,4,4,6,8] gets turned into [1,2,4,4,6,8] and that looks OK. Until one day, that programmer is working with data that has identity and then [1,2,4‡,4†,6,8] turns into [1,2,4†,4‡,6,8] when sorted due to an unstable sort that doesn't care which 4 is which, and the naive programmer's code blows up because they're astonished 4† and 4‡ changed places.
  - duxup 5y ago
    I recently emailed a vendor about a behavior that was documented…. but not in a specific enough way. The result was ideal for me but… suspicious / “too easy” for a vendor who doesn’t make things easy. Before I decided to rely on this I sent an email with some very simple questions. The answer was predictable: “IT DOES WHAT?!?!”
  - wruza 5y ago
    Yep, but also some docs are so underspecified that you find possibilities that the other side didn't even think of to consider, change or document. I think that complements this law (if not yet). And writing even shallow docs is already hard. When there is an knowledge gap between two parties, some assumptions are inevitable, cause otherwise each of two would have to implement overcomplicated logic over nothing. I would like if trades always preceded execution, because that could save me a couple of hours if not a simpler architecture. This is a great driver, even if that would fail once in a week, which may be fine. Edit: since we are talking examples, there is another one, js object key order. These keys go into Object.keys(), for-in and JSON.stringify() in an insertion order. But the order is not specified, even the reproducibility isn't. So in node, chrome and maybe IE it's always predictable (yet), but Firefox couldn't care less. Same for other languages. Some of them even go to lengths of always randomizing the iteration order to prevent false dependencies.
    - kevin_thibedeau 5y ago
      They randomize the hashing to protect against DoS attacks. You have to go to lengths to preserve the order.
      - wruza 5y ago
        It depends. Bare hash tables are semi-random and get reshuffled at growth points naturally. But I remember reading in some manual that that language reshuffles iteration even for immutable hash tables. Js objects are usually based on "shapes", which do not follow hash table semantics: https://mathiasbynens.be/notes/shapes-ics https://mathiasbynens.be/notes/shapes-ics https://v8.dev/blog/fast-properties https://v8.dev/blog/fast-properties
deleted 5y ago
[deleted]
mijoharas 5y ago
Obligatory xkcd: https://xkcd.com/1172/ https://xkcd.com/1172/
msla 5y ago
I agree with most of this, but not necessarily with this: > That is, if an interface has enough consumers, they will collectively depend on every aspect of the implementation, intentionally or not. This effect serves to constrain changes to the implementation, which must now conform to both the explicitly documented interface, as well as the implicit interface captured by usage. We often refer to this phenomenon as "bug-for-bug compatibility." This isn't necessarily the case. Plenty of software makers feel completely free to make incompatible upgrades and not offer any upgrade path. Mozilla, for example, completely destroyed a lot of Firefox functionality, even to the point of destroying bookmarks. Websites which rely on the blink tag are similarly out of luck; some, but not all, browsers support a CSS property which does the same thing, but the point is that the original API, the HTML element, is largely useless.
- sedatk 5y ago
  > Plenty of software makers feel completely free to make incompatible upgrades That isn't really contradictory, is it? It just means that you can do that only if you dare to break some of your consumers. Otherwise, you have to bear the costs of maintaining all those undocumented features. The law simply says that any observable behavior becomes an implicit promise.
- dblohm7 5y ago
  > Mozilla, for example, completely destroyed a lot of Firefox functionality, even to the point of destroying bookmarks. Relevant: https://dblohm7.ca/blog/2015/08/30/on-webextensions/ https://dblohm7.ca/blog/2015/08/30/on-webextensions/ https://dblohm7.ca/blog/2017/11/16/legacy-firefox-extensions-and-userspace/ https://dblohm7.ca/blog/2017/11/16/legacy-firefox-extensions...
  - tialaramex 5y ago
    Right. There actually were operating systems which have the exact problem you describe. I say were because although in fact like any software operating systems don't vanish, they're so unpopular today that they're irrelevant. Amiga is an example. The Amiga OS doesn't have that clean Unix-style border between userspace and kernel, so even on a "modern" PowerPC AmigaOS you can (and people do) write a user program that will say I must not be interrupted right now, I will disable interrupts, and do my thing and then I'll turn them back on and re-enable the operating system. When Linux was young it was a resolutely uni-processor operating system. Linus wasn't interested in fancy "server" hardware, so, why not. But because it has that Unix separation, when SMP was added they did the thing most Unixes did, they added a Big Lock, software which worked correctly before still worked because as far as it knows nothing changed, the radically new multi-processor world it is actually living it seems exactly like the previous world thanks to the Big Lock (getting rid of the Big Lock took many years of effort from there, but introducing the multi-processor feature was in this sense easy) Amiga couldn't (and even today can't) do that. Being able to turn off all interrupts and suspend the operating system is a feature, but, what does that feature mean if there are multiple processors? It becomes nonsense. Hyrum's Law is an observation about how users of your software behave, but it is not a law requiring you to give up and stop innovating because any changes would break people, it's a caution to design things that aren't so fragile, and a reminder that whenever you step over that line you'll cause pain. If they had the guts AmigaOS 4.0, many years ago, should have dropped the "disable interrupts" features. There would have been wailing and screaming, as with Firefox getting rid of legacy "reach inside the guts and do what you want" functionality but the wailing and screaming doesn't last forever and it sets you up to actually innovate and not remain stuck offering a 32-bit uni-processor OS decades later even though your flagship hardware has a 64-bit multi-core CPU.
sedatk 5y ago
> With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody. I find it too verbose. How about this alternative: "Every undocumented behavior of a system eventually becomes a contract."
- tooltower 5y ago
  Ah, so I can stop people from depending on internal details by documenting it! /s I'd use the word "observable" rather than "undocumented"
  - kijin 5y ago
    > Ah, so I can stop people from depending on internal details by documenting it! /s That actually makes sense. If some behavior is not to be relied upon, document it explicitly as such. With scary warning if necessary. At least this will provide you with more ammunition when your update inevitably breaks someone's workflow.
  - sedatk 5y ago
    I mean documented features are observable too; I wanted to narrow the scope.
- duxup 5y ago
  Honestly I needed to read your phrasing to understand the original phrasing of the law. Thank you.
- the8472 5y ago
  Someone depending on it does not mean it becomes the contract. Someone depending on implicit behavior means https://xkcd.com/1172/ https://xkcd.com/1172/ and you shrug and break them anyway because the contract never required you to uphold the property. It's called contract after all, there's a mutual relationship, not a one-sided one. Otherwise specifications would be meaningless and we would just tell people to read the source.
- deleted 5y ago
  [deleted]
ncmncm 5y ago
This is an argument for maintainers to routinely change any unspecified behavior that doesn't cost (much) more to do it the new way, on each release. In a hash, tweak the hash function. Downstream dependents breaking because they depend on undocumented behavior is their fault, not your fault. The sooner they break, the less it costs.
- zmj 5y ago
  For a cheap, specific recommendation: if consumers of an api shouldn't depend on the order of a result set, randomize it.
  - vanviegen 5y ago
    That would of course lead to consumers using your API to generate secret launch codes. Now your defacto spec mandates randomness.
- zarzavat 5y ago
  The other option is to consider non-determinism a bug and to fix it where possible. For instance, several language runtimes (JS, Python) now use dictionary collections that are enumerated in insertion order, removing a long standing source of non-deterministic behaviour.
  - sanxiyn 5y ago
    This happened only after the discovery that for languages like Python, guaranteeing insertion order iteration is virtually free. As I understand how to do that wasn't widely known in, say, 2010, and how to do that for C++/Rust/etc is still unknown.
    - tialaramex 5y ago
      It's not so much that it's free, as that what they were doing before to implement an unordered dict was so terrible that by comparison a reasonable design for an ordered dict is cheaper, so, why not? The options were: * Existing design: Unordered behaviour confuses new programmers, Slow, Wastes space * OrderedDict: New programmers aren't tripped up. Faster, smaller. * Hypothetical UnorderedDict: Confuses new programmers again. Maybe even faster, if anybody builds it, which no-one volunteered to do. Choosing OrderedDict was a no brainer for a language like Python. You can't beat say Swiss Tables (a modern C++ unordered map with excellent performance) with the approach taken in OrderedDict not because we just don't know some optimisation, but because Swiss Tables is doing something a lot faster than what Python does today. Swiss Tables is SIMD. In a single CPU instruction it can examine an entire cacheline of RAM to see if it matches your key (suggesting your value is probably in one of those slots) and in the process identify cases where any slots are empty and yet your key matched nothing, meaning it can stop looking because your key wasn't in this map at all. This requires a very low-level understanding of how the CPU and RAM hierarchy works, and also knowledge of how to use SIMD instructions to get right, but that's fine the Swiss Tables team had those things and needed that performance. To make it fly though, you can't preserve order, the information needed isn't stored anywhere and storing it would make it slower and bigger.
jimworm 5y ago
Once had a bug that set then re-set a cookie. Worked just fine in a browser. A customer wrote some automation that depended on the bug and ignored the first set-cookie. The bugfix caused some problems.
jiggawatts 5y ago
This immediately reminded me of the recent update to AWS S3 to provide strong read-after-write consistency, which Azure Storage Accounts do provide, but S3 historically did not: https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-read-after-write-consistency/ https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-rea... It's these subtle (timing!) differences that might be documented somewhere, but aren't an explicit part of the API surface in any meaningful way. Programmers who naively assumed that blob store APIs are "all the same" and are "just" HTTP GET and PUT calls were in for a lot of fun surprises... Programming theory has a long way to go before it can encode these kinds of subtleties in a way that it won't trip people up.
tome 5y ago
This is why strongly typed programming is good: it allows you to reduce the set of observable behaviours to only ones you actually want.
deleted 5y ago
[deleted]

125 points by eindiran 5y ago

Hyrum was a good friend of mine in grad school. We spent a lot of fun sessions defining stats and KPIs to monitor on SVN projects. It doesn't surprise me to see a law named for him. If he reads HN, hello old friend!

Brotkrumen 5y ago
Well, that seems to be his website, so he basically is trying to name a law after himself.
- hn_go_brrrrr 5y ago
  No: > While I may have made the observation, credit goes to Titus Winters for actually naming it as "Hyrum's Law" and popularizing the concept more broadly.

wruza 5y ago

Reminded me of the first thing I test when working with stock exchanges. Send order #1, watch for events: trade, trade, trade, executed. … Send order #5: trade, trade, executed. Oh well, works as expected, right? Can't be different. It can. Send order #27: trade, trade, executed, trade. Damn ordering got me hard a couple of times.

tialaramex 5y ago
You could look at Hyrum's law as cautionary: don't be that person who relies on unspecified behaviour. This might save you from some unpleasant breakage. But its role for Hyrum and his team is about their burden as engineers. Any observable behaviour they change, no matter how seemingly unimportant, will break somebody who wasn't so cautious, maybe unwittingly. Example: If you're an experienced programmer you're probably aware of stable sorts. You know if you need a sorted list to stay as it is when sorted you must use a stable sort, and you're sensitive when reading documentation - did it say this sort is stable? No? Then maybe it isn't, and I need to guard against that. But a new programmer may just assume sorts are stable without ever thinking about it. Naively they might think of course all sorts are stable, why do the extra work to move things that are in the correct place already - without understanding that common sort techniques often aren't stable. And if they at first use sorts on objects with no sense of identity, like the integers or in many languages strings, there's no sign of a problem, all sorts work the same. [1,2,4,4,6,8] gets turned into [1,2,4,4,6,8] and that looks OK. Until one day, that programmer is working with data that has identity and then [1,2,4‡,4†,6,8] turns into [1,2,4†,4‡,6,8] when sorted due to an unstable sort that doesn't care which 4 is which, and the naive programmer's code blows up because they're astonished 4† and 4‡ changed places.
- duxup 5y ago
  I recently emailed a vendor about a behavior that was documented…. but not in a specific enough way. The result was ideal for me but… suspicious / “too easy” for a vendor who doesn’t make things easy. Before I decided to rely on this I sent an email with some very simple questions. The answer was predictable: “IT DOES WHAT?!?!”
- wruza 5y ago
  Yep, but also some docs are so underspecified that you find possibilities that the other side didn't even think of to consider, change or document. I think that complements this law (if not yet). And writing even shallow docs is already hard. When there is an knowledge gap between two parties, some assumptions are inevitable, cause otherwise each of two would have to implement overcomplicated logic over nothing. I would like if trades always preceded execution, because that could save me a couple of hours if not a simpler architecture. This is a great driver, even if that would fail once in a week, which may be fine. Edit: since we are talking examples, there is another one, js object key order. These keys go into Object.keys(), for-in and JSON.stringify() in an insertion order. But the order is not specified, even the reproducibility isn't. So in node, chrome and maybe IE it's always predictable (yet), but Firefox couldn't care less. Same for other languages. Some of them even go to lengths of always randomizing the iteration order to prevent false dependencies.
  - kevin_thibedeau 5y ago
    They randomize the hashing to protect against DoS attacks. You have to go to lengths to preserve the order.
    - wruza 5y ago
      It depends. Bare hash tables are semi-random and get reshuffled at growth points naturally. But I remember reading in some manual that that language reshuffles iteration even for immutable hash tables. Js objects are usually based on "shapes", which do not follow hash table semantics: https://mathiasbynens.be/notes/shapes-ics https://mathiasbynens.be/notes/shapes-ics https://v8.dev/blog/fast-properties https://v8.dev/blog/fast-properties

deleted 5y ago

[deleted]

mijoharas 5y ago

Obligatory xkcd: https://xkcd.com/1172/ https://xkcd.com/1172/

msla 5y ago

I agree with most of this, but not necessarily with this: > That is, if an interface has enough consumers, they will collectively depend on every aspect of the implementation, intentionally or not. This effect serves to constrain changes to the implementation, which must now conform to both the explicitly documented interface, as well as the implicit interface captured by usage. We often refer to this phenomenon as "bug-for-bug compatibility." This isn't necessarily the case. Plenty of software makers feel completely free to make incompatible upgrades and not offer any upgrade path. Mozilla, for example, completely destroyed a lot of Firefox functionality, even to the point of destroying bookmarks. Websites which rely on the blink tag are similarly out of luck; some, but not all, browsers support a CSS property which does the same thing, but the point is that the original API, the HTML element, is largely useless.

sedatk 5y ago
> Plenty of software makers feel completely free to make incompatible upgrades That isn't really contradictory, is it? It just means that you can do that only if you dare to break some of your consumers. Otherwise, you have to bear the costs of maintaining all those undocumented features. The law simply says that any observable behavior becomes an implicit promise.
dblohm7 5y ago
> Mozilla, for example, completely destroyed a lot of Firefox functionality, even to the point of destroying bookmarks. Relevant: https://dblohm7.ca/blog/2015/08/30/on-webextensions/ https://dblohm7.ca/blog/2015/08/30/on-webextensions/ https://dblohm7.ca/blog/2017/11/16/legacy-firefox-extensions-and-userspace/ https://dblohm7.ca/blog/2017/11/16/legacy-firefox-extensions...
- tialaramex 5y ago
  Right. There actually were operating systems which have the exact problem you describe. I say were because although in fact like any software operating systems don't vanish, they're so unpopular today that they're irrelevant. Amiga is an example. The Amiga OS doesn't have that clean Unix-style border between userspace and kernel, so even on a "modern" PowerPC AmigaOS you can (and people do) write a user program that will say I must not be interrupted right now, I will disable interrupts, and do my thing and then I'll turn them back on and re-enable the operating system. When Linux was young it was a resolutely uni-processor operating system. Linus wasn't interested in fancy "server" hardware, so, why not. But because it has that Unix separation, when SMP was added they did the thing most Unixes did, they added a Big Lock, software which worked correctly before still worked because as far as it knows nothing changed, the radically new multi-processor world it is actually living it seems exactly like the previous world thanks to the Big Lock (getting rid of the Big Lock took many years of effort from there, but introducing the multi-processor feature was in this sense easy) Amiga couldn't (and even today can't) do that. Being able to turn off all interrupts and suspend the operating system is a feature, but, what does that feature mean if there are multiple processors? It becomes nonsense. Hyrum's Law is an observation about how users of your software behave, but it is not a law requiring you to give up and stop innovating because any changes would break people, it's a caution to design things that aren't so fragile, and a reminder that whenever you step over that line you'll cause pain. If they had the guts AmigaOS 4.0, many years ago, should have dropped the "disable interrupts" features. There would have been wailing and screaming, as with Firefox getting rid of legacy "reach inside the guts and do what you want" functionality but the wailing and screaming doesn't last forever and it sets you up to actually innovate and not remain stuck offering a 32-bit uni-processor OS decades later even though your flagship hardware has a 64-bit multi-core CPU.

sedatk 5y ago

> With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody. I find it too verbose. How about this alternative: "Every undocumented behavior of a system eventually becomes a contract."

tooltower 5y ago
Ah, so I can stop people from depending on internal details by documenting it! /s I'd use the word "observable" rather than "undocumented"
- kijin 5y ago
  > Ah, so I can stop people from depending on internal details by documenting it! /s That actually makes sense. If some behavior is not to be relied upon, document it explicitly as such. With scary warning if necessary. At least this will provide you with more ammunition when your update inevitably breaks someone's workflow.
- sedatk 5y ago
  I mean documented features are observable too; I wanted to narrow the scope.
duxup 5y ago
Honestly I needed to read your phrasing to understand the original phrasing of the law. Thank you.
the8472 5y ago
Someone depending on it does not mean it becomes the contract. Someone depending on implicit behavior means https://xkcd.com/1172/ https://xkcd.com/1172/ and you shrug and break them anyway because the contract never required you to uphold the property. It's called contract after all, there's a mutual relationship, not a one-sided one. Otherwise specifications would be meaningless and we would just tell people to read the source.
deleted 5y ago
[deleted]

ncmncm 5y ago

This is an argument for maintainers to routinely change any unspecified behavior that doesn't cost (much) more to do it the new way, on each release. In a hash, tweak the hash function. Downstream dependents breaking because they depend on undocumented behavior is their fault, not your fault. The sooner they break, the less it costs.

zmj 5y ago
For a cheap, specific recommendation: if consumers of an api shouldn't depend on the order of a result set, randomize it.
- vanviegen 5y ago
  That would of course lead to consumers using your API to generate secret launch codes. Now your defacto spec mandates randomness.
zarzavat 5y ago
The other option is to consider non-determinism a bug and to fix it where possible. For instance, several language runtimes (JS, Python) now use dictionary collections that are enumerated in insertion order, removing a long standing source of non-deterministic behaviour.
- sanxiyn 5y ago
  This happened only after the discovery that for languages like Python, guaranteeing insertion order iteration is virtually free. As I understand how to do that wasn't widely known in, say, 2010, and how to do that for C++/Rust/etc is still unknown.
  - tialaramex 5y ago
    It's not so much that it's free, as that what they were doing before to implement an unordered dict was so terrible that by comparison a reasonable design for an ordered dict is cheaper, so, why not? The options were: * Existing design: Unordered behaviour confuses new programmers, Slow, Wastes space * OrderedDict: New programmers aren't tripped up. Faster, smaller. * Hypothetical UnorderedDict: Confuses new programmers again. Maybe even faster, if anybody builds it, which no-one volunteered to do. Choosing OrderedDict was a no brainer for a language like Python. You can't beat say Swiss Tables (a modern C++ unordered map with excellent performance) with the approach taken in OrderedDict not because we just don't know some optimisation, but because Swiss Tables is doing something a lot faster than what Python does today. Swiss Tables is SIMD. In a single CPU instruction it can examine an entire cacheline of RAM to see if it matches your key (suggesting your value is probably in one of those slots) and in the process identify cases where any slots are empty and yet your key matched nothing, meaning it can stop looking because your key wasn't in this map at all. This requires a very low-level understanding of how the CPU and RAM hierarchy works, and also knowledge of how to use SIMD instructions to get right, but that's fine the Swiss Tables team had those things and needed that performance. To make it fly though, you can't preserve order, the information needed isn't stored anywhere and storing it would make it slower and bigger.

jimworm 5y ago

Once had a bug that set then re-set a cookie. Worked just fine in a browser. A customer wrote some automation that depended on the bug and ignored the first set-cookie. The bugfix caused some problems.

jiggawatts 5y ago

This immediately reminded me of the recent update to AWS S3 to provide strong read-after-write consistency, which Azure Storage Accounts do provide, but S3 historically did not: https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-read-after-write-consistency/ https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-rea... It's these subtle (timing!) differences that might be documented somewhere, but aren't an explicit part of the API surface in any meaningful way. Programmers who naively assumed that blob store APIs are "all the same" and are "just" HTTP GET and PUT calls were in for a lot of fun surprises... Programming theory has a long way to go before it can encode these kinds of subtleties in a way that it won't trip people up.

tome 5y ago

This is why strongly typed programming is good: it allows you to reduce the set of observable behaviours to only ones you actually want.

deleted 5y ago

[deleted]