11 ms·

API design note: Beware of adding an "Other" enum value

237 points by luu 2y ago

_3u10 2y ago
I usually use Unknown / Other as 0.
zdw 2y ago
I wonder how this aligns with the protobuf best practice of having the first value be UNSPECIFIED: https://protobuf.dev/best-practices/dos-donts/#unspecified-enum https://protobuf.dev/best-practices/dos-donts/#unspecified-e...
- jmole 2y ago
  The example code used added “other” as the last option, which was the source of the problems he described. This doesn’t happen when you make the first value in the enum unknown/unspecified
  - plorkyeran 2y ago
    No, the problem described in the article is entirely unrelated to where in the enum the Other option is located. There is a different problem where keeping the Other option at the end of the enum changes the value of Other, but that is not the problem that the article is about.
    - jmole 2y ago
      Well it simplifies the logic considerably - if you see an enum value you don’t recognize (mint), you treat it as uninitialized (0). So any future new flavor will be read back as ‘0’ in older versions.
- beart 2y ago
  "Unspecified" is semantically different from "other". The former is more like a default value whereas the latter is actually "specified, but not one of these listed options".
  - hamandcheese 2y ago
    Standard practice in protobuf is to never assign semantic meaning to the default value. I think some linters enforce that enum 0 is named "unknown" which is actually more semantically correct than "other" or "unspecified".
- seeknotfind 2y ago
  This is the same as a null pointer, and the requirement is very deeply tied to protobuf as it is used on large distributed systems that always need to handle version mismatch, and this advice doesn't necessarily apply to API design in general.
  - eddd-ddde 2y ago
    Even in the simplest web apps you can encounter version mismatch when a client requests a response from a server that just updated.
    - hansvm 2y ago
      Hence the advice to make that situation not happen. Update the client and server to support both versions and prefer the new one, then update both to not support the old version. With load balancers and other real-world problems you might have to break that down into 4 coordinated steps.
      - Joker_vD 2y ago
        That only really works if you control the clients, or can force them to update.
        LoganDark 2y ago
        > or can force them to update. I've used a few clients that completely lock me out for every tiniest minor version update. Very top-tier annoying imho.
        eru 2y ago
        But it does make the authors' jobs easier.
    - seeknotfind 2y ago
      This implies an API where the server has a single shared implementation. Imagine for instance that the server implements a shim for each version of the interface, then there isn't a need for the null in the API. Imagine another alternative, that the same API never adds a field, but you add a new method which takes the new type. Imagine yet again an API where you are able to version the clients in lockstep. So, it's a decision about how the API is used and evolves that recommends the API encoding or having a null default. However in a different environment or with different practices, you can avoid the null. Of course the reason to avoid the null is so that you can statically enforce this value is provided in new clients, though this also assumes your client language is typed. So in the end, protobuf teaches us, but it's not always the best in every situation.
- bocahtie 2y ago
  When the deserializing half of the protobuf definitions encounter an unknown value, it gets deserialized as the zero value. When that client updates, it will then be able to deserialize the new value appropriately (in this case, "Mint"). The advice on that page also specifies to not make the value semantically meaningful, which I take to mean to never set it to that value explicitly.
  - chen_dev 2y ago
    > it gets deserialized as the zero value It’s more complicated: https://protobuf.dev/programming-guides/enum/ https://protobuf.dev/programming-guides/enum/ >> What happens when a program parses binary data that contains field 1 with the value 2? >- Open enums will parse the value 2 and store it directly in the field. Accessor will report the field as being set and will return something that represents 2. >- Closed enums will parse the value 2 and store it in the message’s unknown field set. Accessors will report the field as being unset and will return the enum’s default value.
    - vitus 2y ago
      Ugh. I hate how we (Google) launched proto editions. It used to be that we broadly had two sets of semantics (modulo additional customizations): proto2 and proto3. Proto editions was supposed to unify the two versions, but instead now we have the option to mix and match all of the quirks of each of the versions. And, to make matters worse, you also have language-dependent implementations that don't conform to the spec (in fact, very few implementations are conformant). C++ and Java treat everything imported by a proto2 file as closed; C#, Golang, and JS treat everything as open. I don't see a path forward for removing these custom deprecated field features, or else we'd have already begun that effort during the initial adoption of editions.
  - dwattttt 2y ago
    > The advice on that page also specifies to not make the value semantically meaningful, which I take to mean to never set it to that value explicitly. I've taken to coding my C enums with the first value being "Invalid", indicating it is never intended to be created. If one is encountered, it's a bug.
- MarkMarine 2y ago
  I don’t mind the zero value for the proto enums, makes sense, but I require converting to my inner logic to not include this “unknown” and error during the conversion if it fails. I’ve seen engineers bring those unknowns or unspecified through to the business logic and that always made my face flush red with anger.
  - fmbb 2y ago
    Why the anger? If you are consuming data from some other system you have no power over what to require from users. You will have data points with unknown properties. Say you are tracking sign ups in some other system, and they collect the users’ browser in the process, and you want to see conversion rate per browser. If the browser could not be identified, you prefer it to say ”other” instead of ”unknown”? I think I prefer the protobuf best practices way: you have a 0 ”unknown”/”unset” value, and you enumerate the rest with a unique name (and number). The enum can be expanded in the future so your code must be prepared for unknown enumerated values tagged with the new (future for your code) number. They are all unique, you just don’t yet know the name of some of the enum values. You can choose to not consume them until your code is updated with a more recent schema. Or you can reconcile later, annotating with the name of you need it. Now personally, I would not pick an enum for any set och things that is not closed when you are designing. But I’m starting to think that such sets hardly exist in the real world. Humans redefine everything over time.
  - crabbone 2y ago
    I wrote my own Protobuf implementation (well, with some changes). Ditching the default values was one of the changes I made. I don't see any reason to have that. But I don't think that Protobuf is a reasonable or even decent protocol in general. It has a lot of nonsense and bad planning. Having default values is probably not in the ten worst things about Protobuf.
- deleted 2y ago
  [deleted]
NoboruWataya 2y ago
> Just document that the enumeration is open-ended, and programs should treat any unrecognized values as if they were “Other”. Possibly just showing my lack of knowledge here but are open-ended enumerations a common thing? I always thought the whole point of an enum is that it is closed-ended?
- sd9 2y ago
  I’ve worked on systems which where the set of enum values was fixed at any particular point in time, but could change over time as business requirements changed. For instance, we had an enum that represented a sport that we supported. Initially we supported some sports (say FOOTBALL and ICE_HOCKEY), and over time we added support for other sports, so the enum had to be expanded. Unfortunately this always required the entire estate to be redeployed. Thankfully this didn’t happen often. At great expense, we eventually converted this and other enums to “open-ended” enums (essentially Strings with a bit more structure around them, so that you could operate on them as if they were “real” enums). This made upgrades significantly easier. Now, whether those things should have been enums in the first place is open for debate. But that decision had been made long before I joined the team. Another example is gender. Initially an enum might represent MALE, FEMALE, UNKNOWN. But over time you might decide you have need for other values: PREFER_NOT_TO_SAY, OTHER, etc.
- XorNot 2y ago
  The first time you have to add a new schema value, you'll realise you needed "unknown" or similar - because during an upgrade your old systems need a way to deal with new values (or during a rollback you need to handle new entries in the database).
  - sitkack 2y ago
    Your comment is the only in the entire discussion that mentions "schema". Having an "other" in a schema is a way to ensure you can run n and n+1 versions at the same time. It is Data Model design, of which API design a subset. You can only ever avoid having an other if 1) your schema is fixed and 2) if it is total over the universe of values.
o11c 2y ago
The approach in the link is fine for consumers, but for producers you really do need some way of saying "create a value that's not one of the known values". Still, there's nothing that says this needs to be pretty.
coin 2y ago
Just call it "unknown" or "unspecified" or better yet use an optional to hold the enum.
- 101011 2y ago
  This ended up being the preferred pattern we moved into. If, like us, you were passing the object between two applications, the owning API would serialize the enum value as a String value, then we had a client helper method that would parse the string value into an Optional enum value. If the original service started transferring a new String object between services, it wouldn't break any downstream clients, because the clients would just end up with Optional empty
  - janci 2y ago
    How that works when you need to distinguish between "no value provided" and "a value that is not in the list" - in some applications they have different semantics.
    - 101011 2y ago
      You could treat it as an nullable Option<SomeType>. In practice, as it relates to enums, I don't usually see 'no value provided' as a frequently used case - it's more likely that 'no value provided' maps to a more informative 'enum' value
remram 2y ago
Rust has the "non_exhaustive" attribute that lets you declare that an enum might get more fields in the future. In practice that means that when you match on an enum value, you have to add a default case. It's like a "other" field in the enum except you can't reference it directly, you use a default case. IIRC a secret 'other' field (or '__non_exhaustive' or something) is actually how we did thing before non_exhaustive was introduced.
- airstrike 2y ago
  TIL https://doc.rust-lang.org/reference/attributes/type_system.html https://doc.rust-lang.org/reference/attributes/type_system.h...
- kibwen 2y ago
  Note that the stance of the OP here is broadly in agreement with what Rust does. His main objection is this: > The word “other” means “not mentioned elsewhere”, so the presence of an Other logically implies that the enumeration is exhaustive. In Rust, because all enums are exhaustive by default and exhaustive matching is enforced by the compiler, there is no risk of this sort of confusion. And then the fact that his proposed solution is: > Just document that the enumeration is open-ended The non_exhaustive attribute is effectively compiler-enforced documentation; users now cannot forget to treat the enum as open-ended. Of course, adding non_exhaustive to Rust was not without its own detractors; it usage for any given enum fundamentally means shifting power away from library consumers (who lose the ability to guarantee exhaustive matching) and towards library authors (who gain the ability to evolve their API without causing guaranteed compilation errors in all of their users (which some users desire!)). As such, the guidance is that it should be used sparingly, mostly for things like error types. But that's an argument against open-ended enums in general, not against the mechanisms we use to achieve those (which, as you say, was already possible in Rust via hacks).
  - tyre 2y ago
    Maybe there should be a compiler option or function to assert that a match is exhaustive. If the match does not handle a defined case, it blows up.
esafak 2y ago
Just add a free-form text field to hold the other value, and revise your enum as necessary, while migrating the data.
- AceJohnny2 2y ago
  I can't even tell if you're trolling.
- bilekas 2y ago
  As the other commenter mentioned, I hope this is an excersize in trolling. Please don't do this when using enums. > Free-form text firle to hold the other value > revise your enum as necessary, while migrating the dat Both instances would defeat the purpose of an enum.. As for the original post, my 2cents are valued enums. { Other = 0, Vanilla = 1, Chocolate = 2, Strawberry = 3 } In this case it allows some flexibility to add later while still being able to make use of 0 (Other). Or maybe I missed the OP's point ?
  - esafak 2y ago
    With a form you will have an empirical basis to determine what the missing enum values should be, and the user won't enter any text unless necessary.
    - bilekas 2y ago
      > and the user won't enter any text unless necessary You are underestimating users' ability to ignore non strict constraints. Also when I can, why would I not increase my compile time checking ? There are very good reasons to use Enums, you seem to just want to ignore those reasons.
      - esafak 2y ago
        I'm not ignoring enumerations. I'm supplementing the existing options with a text field rather than shoehorning everything into an uninformative "other" value. I acknowledge your point about the user ignoring constraints.
1oooqooq 2y ago
jr: add other option sr: omit other option illuminated: add other option in front end only and alert when the backend crashes.
hchja 2y ago
[dead]
layer8 2y ago
Slight counterpoint: Unless there is some guarantee that the respective enum type will never ever be extended with a new value, each and every case distinction on an enum value needs to consider the case of receiving an unexpected value (like Mint in the example). When case distinctions do adhere to that principle, then the problem described doesn’t arise. On the other hand, if the above principle is adhered to as it should, then there is also little benefit in having an Other value. One minor conceivable benefit is that intermediate code can map unsupported values to Other in order to simplify logic in lower-level code. But I agree that it’s usually better to not have it. A somewhat related topic that comes to mind is error codes. There is a common pattern, used for example by the HTTP status codes, where error codes are organized into categories by using different prefixes. For example in a five-digit error code scheme, the first three digits might indicate the category (e.g. 123 for “authentication errors”), and the remaining two digits represent a more specific error condition in that category. In that setup, the all-zeros code in each category represents a generic error for that category (i.e. 12300 would be “generic authentication error”). When implementing code that detects a new error situation not covered by the existing specific error codes, the implementer has now the choice of either introducing a new error code (e.g. 12366 — this is analogous to adding a new enum value), which has to be documented and maybe its message text be localized, or else using the generic error code of the appropriate category. In any case, when error-processing code receives an unknown — maybe newly assigned — error code, they can still map it according to the category. For example, if the above 12366 is unknown, it can be handled like 12300 (e.g. for the purpose of mapping it to a corresponding error message). This is quite similar to the case of having an Other enum value, but with a better justification.
vadim_phystech 2y ago
...since the set of all possible behaviour, that is not specified, it much greater, and densier, than one would initially feel and assume, one might cause lot's of possible bad outcomes and success-breaking-points if use "Other" type in their API. Because "Other" if the 1st thing to look for vulnerabilities, for attack vectors. Because the spirit of UB the Terrible lurks there! The spirit of UB feeds upon thee juices of "Other" omnimorphic (fel) type! скверный бесформенный "ЛЮБОЙ" тип! разврат и дисгармоничность! разложение и редуцирующие гетероморфизмы! decomposition, descriptive semantic matrix rank reduction, richness degradation, devolution...empoorness...scarcity pressure increase... </shutting_the_fuck_up_my_wetware_machine_whispering_kek>
deleted 2y ago
[deleted]
akamoonknight 2y ago
One of the tactics I end up using in Verilog, for better or worse, is to define enums with a'0 value (repeat 0s for the size of the variable), and '1 value (repeat 1s for the size of the value) '0 stays as "null"-like (e.g INVALID), and '1 (which would be 0xFF in an 8 bit byte for instance) becomes "something, but I'm not sure what" (e.g. UNKNOWN). Definitely has the same issues as referenced when needing to grow the variable, and the times where it's useful aren't super common, but I do feel like the general concept of an unknown-but-not-invalid value can help with tracking down errors in processing chains Definitely do run into the need to "beware" though with enums for sure.
bob1029 2y ago
Making things into enums that shouldn't be enums is a fun trap to fall into. Much of the time what you really want is a complex type so that you can communicate these additional facts. In this case I'd do something like: class Widget { WidgetFlavor Flavor; //Undefined, Vanilla, Chocolate, Strawberry string? OtherFlavor; } This is easy to work from a consumer standpoint because if you have a deviant flavor to specify, you don't bother setting the Flavor member to anything at all. You just set OtherFlavor. Fewer moving pieces == less chance for bad times. The first (default) member in an enum should generally be something approximating "Undefined". This also makes working with serializers and databases easier.
- ryanschaefer 2y ago
  > Fewer moving pieces == less chance for bad times. Is this not a case for explicitly specifying all flavors? Other flavor has essentially introduced infinite moving pieces.
- IshKebab 2y ago
  This is not a good design. You've introduced representable invalid states (Flavor=Vanilla, Other flavor="DarkChocolate"). At the least you want this... enum Flavor { Chocolate, Banana, Strawberry, Other(String), } But that's not right either. What you really want is #[non_exhaustive] enum Flavor { Chocolate, Banana, Strawberry, } impl ToString for Flavor ...
- msy 2y ago
  Until you get a Widget.OtherFlavor = 'Vanilla'
DonHopkins 2y ago
https://en.wikipedia.org/wiki/Tony_Hoare#Research_and_career https://en.wikipedia.org/wiki/Tony_Hoare#Research_and_career >Speaking at a software conference in 2009, Tony Hoare apologized for inventing the null reference, his "Billion Dollar Mistake": >"I call it my billion-dollar mistake. It was the invention of the null reference in 1965. At that time, I was designing the first comprehensive type system for references in an object oriented language (ALGOL W). My goal was to ensure that all use of references should be absolutely safe, with checking performed automatically by the compiler. But I couldn't resist the temptation to put in a null reference, simply because it was so easy to implement. This has led to innumerable errors, vulnerabilities, and system crashes, which have probably caused a billion dollars of pain and damage in the last forty years." -Tony Hoare Anders Hejlsberg brilliantly points out how JavaScript doubled the cost of that mistake: >"My favorite is always the Billion-Dollar Mistake of having null in the language. And since JavaScript has both null and undefined, it's the Two-Billion-Dollar Mistake." -Anders Hejlsberg >"It is by far the most problematic part of language design. And it's a single value that -- ha ha ha ha -- that if only that wasn't there, imagine all the problems we wouldn't have, right? If type systems were designed that way. And some type systems are, and some type systems are getting there, but boy, trying to retrofit that on top of a type system that has null in the first place is quite an undertaking." -Anders Hejlsberg The JavaScript Equality Table shows how Brendan Eich simply doesn't understand equality for either data types or human beings and their right to freely choose who they love and marry: https://dorey.github.io/JavaScript-Equality-Table/ https://dorey.github.io/JavaScript-Equality-Table/ Do any languages implement the full Rumsfeld Awareness–Understanding Matrix Agnoiology, quadrupling the cost? Why stop at null, when you can have both null and undefined? Throw in unknown, and you've got a hat trick, a holy trinity of nihilistic ignorance, nothingness, and void! The Rumsfeld Awareness–Understanding Matrix Agnoiology breaks knowledge down into known knows, plus the three different types of unknowns: https://en.wikipedia.org/wiki/There_are_unknown_unknowns https://en.wikipedia.org/wiki/There_are_unknown_unknowns >"Reports that say that something hasn't happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don't know we don't know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones." -Donald Rumsfeld 1) Known knowns: These are the things we know that we know. They represent the clear, confirmed knowledge that can be easily communicated and utilized in decision-making. 2) Known unknowns: These are the things we know we do not know. This category acknowledges the presence of uncertainties or gaps in our knowledge that are recognized and can be specifically identified. 3) Unknown knowns: Things we are not aware of but do understand or know implicitly 4) Unknown unknowns: These are the things we do not know we do not know. This category represents unforeseen challenges and surprises, indicating a deeper level of ignorance where we are unaware of our lack of knowledge. https://en.wikipedia.org/wiki/Agnoiology https://en.wikipedia.org/wiki/Agnoiology >Agnoiology (from the Greek ἀγνοέω, meaning ignorance) is the theoretical study of the quality and conditions of ignorance, and in particular of what can truly be considered "unknowable" (as distinct from "unknown"). The term was coined by James Frederick Ferrier, in his Institutes of Metaphysic (1854), as a foil to the theory of knowledge, or epistemology. I don't know if you know, but Microsoft COM hinges on the IUnknown interface. Microsoft COM's IUnknown interface takes the Rumsfeldian principle to heart: it doesn't assume what an object is but provides a structured way to query for knowledge (or interfaces). In a way, it models known unknowns, since a caller knows that an interface might exist but must explicitly ask if it does. Then there's Schulz's Known Nothing Nesiology, representing the existential conclusion of all this: when knowledge itself is questioned, where does that leave us? Right back at JavaScript's Equality Table, which remains an unfathomable unknown unknown to Brendan Eich and his well known but knowingly ignorant War on Equality. https://www.youtube.com/watch?v=HblPucwN-m0 https://www.youtube.com/watch?v=HblPucwN-m0 Nescience vs. Ignorance (on semantics and moral accountability): https://cognitive-liberty.online/nescience-vs-ignorance/ https://cognitive-liberty.online/nescience-vs-ignorance/ >From a psycholinguistic vantage point, the term “ignorance” and the term “nescience” have very different semantic connotations. The term ignorance is more generally more widely colloquially utilized than the term nescience and it is often wrongly used in contexts where the word nescience would be appropriate. “Ignorance” is associated with “the act of ignoring”. Per contrast, “nescience” means “to not know” (viz., Latin prefix ne = not, and the verb scire = “to know”; cf. the etymology of the word “science”/prescience). >As Mark Passio points out, the important underlying question which can be derived from this semantic distinction pertains to whether our individual and global problems are caused by “ignorance” or “nescience”? That is, “ignoring” or “not knowing”? It seems clear that it is the later. We know about the truth but we actively ignore it for the most part. Currently people have all the necessary information available (literally at their fingertips). Ignoring the facts is a decision, an irrational decision, and people can be held accountable for this decision. Nescience, on the other hand, acquits from accountability (i.e., someone cannot be held accountable when he/she for not knowing something but for ignoring something). Quasi-Freudian suppression plays a pivotal role in this scenario. Suppression is very costly in energetic terms. The energy and effort which is used for suppression lacks elsewhere (cf. prefrontal executive control is based on limited cognitive resources). The suppression of truth through the act of active ignoring thus has negative implications on multiple levels – on the individual and the societal level, the cognitive and the political, the psychological and the physiological. Brendan: While we can measure the economic consequences of your culpably ignorant mistakes of both bad programming language design and marriage inequality in billions of dollars, the emotional, social, and moral costs of the latter -- like diminished human dignity and the perpetuation of discrimination -- are, by their very nature, priceless. Ultimately, these deeper impacts underscore that the fight for marriage equality, defending against the offensive uninvited invasion of your War on Equality into other people's marriages, is about much more than economics; it’s about ensuring fairness, respect, and equality for all members of society.
sgondala_ycapp 2y ago
Random tidbit: We use LLM to identify document types and use an enum to show a list of options. Initially, we didn’t include an "Other" category - which led the LLM to force-fit documents into existing types even when they didn’t belong. Obv this wasn't LLM's fault. We realized the mistake and added "Other". This significantly improved output accuracy!
dataflow 2y ago
I think there are multiple concerns here, and they need to be analyzed separately -- they don't converge to the same solution: - Naming: "Other" should probably be called "Unrecognized" in these situations. Then users understand that members may not be mutually exclusive. - ABI: If you need ABI compatibility, the constraint you have is "don't change the meanings of values or members", which is somewhat stronger. The practical implication is that if you do need to have an Other value, its value should be something out of range of possible future values. - Protocol updates: If you can atomically update all the places where the enum is used, then there's no inherent need to avoid Other values. Instead, you can use compile-time techniques (exhaustive switch statements, compiler warnings, temporarily removing the Other member, grep, clang-query, etc.) to find and update the usage sites at compile time. This requires being a little disciplined in how you use the enum during development, but it's doable. - Distributed code: If you don't have control over all the code using your enum might, then you must avoid an Other value, unless you can somehow ensure out-of-band that users have updated their code.
KPGv2 2y ago
> Rust has the "non_exhaustive" attribute that lets you declare that an enum might get more fields in the future.a Is there a reason, aside from documentation, that this is ever desirable? I rarely program in Rust, but why would this ever be useful in practice, outside of documentation? (Seems like code-as-documentation gone awry when your code is doing nothing but making a statement about future code possibilities)
- LegionMammal978 2y ago
  Normally, when you match on the value of an enum, Rust forces you to either add a case for every possible variant, or add a default arm "_ => ..." that acts as a 'none of the above' case. This is called exhaustiveness checking [0]. When you add #[non_exhaustive] to an enum, the compiler says to external users, "You're no longer allowed to just match every existing variant. You must always have a default 'none of the above' case when you're matching on this enum." This lets you add more variants in the future without breaking the API for existing users, since they all have a 'none of the above' case for the new variants to fall into. [0] https://doc.rust-lang.org/book/ch06-02-match.html#matches-are-exhaustive https://doc.rust-lang.org/book/ch06-02-match.html#matches-ar...
- jeroenhd 2y ago
  If your library processes data from another language, you'll probably need to deal with the possibility that the library returns open ended enums. I believe I've also seen this declaration for generated bindings for a JSON API that promises backwards compatibility for calls and basic functionality at least. Future versions may include more options, but the code will still compile fine against the older API. I don't think it's a great tool to use everywhere, but there are edge cases where Rust's demand for exhaustive matches conflicts with the non-Rust world, and that's where stuff like this becomes hard to avoid.
hello12343214 2y ago
Good idea. I appreciate that he thought through future compatibility with old versions.
mkleczek 2y ago
I've had a short discussion with Brian Goetz about a similar case (sealed types in Java): https://mail.openjdk.org/pipermail/amber-dev/2020-April/005844.html https://mail.openjdk.org/pipermail/amber-dev/2020-April/0058... I wonder when we are going to re-discover OOP style dynamic dispatch (or even better: multiple dispatch) to deal with software evolution.
jasonkester 2y ago
This got me wondering what I actually do in practice. I think it's this: const KnownFlavors { Vanilla: "Vanilla", Chocolate: "Chocolate", Strawberry: "Strawberry" } Then, use a string to hold the actual value. doug.favoriteFlavor = KnownFlavors.Chocolate; cindy.favoriteFlavor = "Mint" case: KnownFlavors.Chocolate: Expand your list of known flavors whenever you like, your system will still always hold valid data. You get all the benefits of typo-proofing your code, switching on an enum, etc., without having to pile on any wackiness to fool your compiler or keep the data normalized. It acknowledges the reality that a non-exhaustive enum isn’t really an enum. It’s just a list of things that people might type into that field.
- Boldened15 2y ago
  Sorry I don't get the example, are both code blocks meant to be client-side code? > It acknowledges the reality that a non-exhaustive enum isn’t really an enum. It’s just a list of things that people might type into that field. I would say the opposite, the kinds of enums that map a case to a few hardcoded branches (SUCCESS, NETWORK_ERROR, API_ERROR) are often an approximation of algebraic data types which Rust implements as enums [0] but not most languages or data formats. Since often using those will require something like a `nullthrows($response->getNetworkError())` once you've matched the enum case. The kind of enum that's just a string whitelist, like flavors or colors, which you can freely pass around and store, likely converting it into a human-readable string or RGB values in one or two utils, is the classic kind of enum to me. [0] https://doc.rust-lang.org/std/keyword.enum.html https://doc.rust-lang.org/std/keyword.enum.html
- PartiallyTyped 2y ago
  The way we do this with AWS SDK in rust is by leveraging #non_exhaustive, and matching the (_@other) pattern, this is forward compatible and allows us to do something like ( _@other) if other.name() == “foo” for known cases without upgrading down the road or if the user uses an older version than our API.
kstenerud 2y ago
I use the "other" technique when it's necessary for the user to be able to mix in their own: enum WidgetFlavor { Vanilla, Chocolate, Strawberry, Other=10000, }; Now users can add their own (and are also responsible for making sure it works in all APIs): enum CustomWidgetFlavor { RockyRoad=Other, GroovyGrape, Cola, }; And now you can amend the enum without breaking the client: enum WidgetFlavor { Vanilla, Chocolate, Strawberry, Mint, Other=10000, };
- qingcharles 2y ago
  It's code like this that ends in a terminal choco-banana shake hang: http://www.technofileonline.com/texts/chocobanana.gif http://www.technofileonline.com/texts/chocobanana.gif
  - fingerlocks 2y ago
    What is the context here? Is this just a silly nonsense tech support page/meme or an actual product from the late 90s?
    - sandblast 2y ago
      Is the context not clear for you from the information that the article applies to "DreamWorks Interactive, Someone's in the Kitchen, version 1.0" and other clues that it's a game?
    - qingcharles 2y ago
      It's a real Knowledge Base article from Microsoft. I can't find the original, but it is on archive.org somewhere if you dig around.
- dominicrose 2y ago
  "Other" doesn't mean the same thing before and after Cola has been added. "Unknown" would be more accurate. My personal opinion would be to make the enum nullable and not add a fake value.
jffhn 2y ago
>"programs should treat any unrecognized values as if they were “Other”" Having such an "Other" value does not prevent from considering that the enum is open-ended, and it simplifies a lot all the code that has to deal with potentially invalid or unknown values (no need for a validity flag or null). That's probably why in DIS (Distributed Interactive Simulation) standard, which defines many enums, all start with OTHER, which has the value zero. In STANAGs (NATO standards), the value zero is used for NO_STATEMENT, which can also be used when the actual value is in the enum but you can't or don't need to indicate it. I remember an "architecture astronaut" who claimed that NO_STATEMENT was not a domain value, and removed it from all the enums in its application. That did not last long. That also reminds me of Philippe Khan (Bordland) having in some presentation the ellipse extend the circle, to add a radius. A scientist said he would do the other way around, and Khan replied: "This is exactly the difference between research and industry".
- ivan_gammel 2y ago
  >That also reminds me of Philippe Khan (Bordland) having in some presentation the ellipse extend the circle, to add a radius. A scientist said he would do the other way around, and Khan replied: "This is exactly the difference between research and industry". My favorite question on interviews on the OOP topic. It can be correct either way or both can be wrong, so the good answer would be "It depends". When developers rush to give a specific answer, they do not demonstrate due attention to the domain and it may mean that they will assume thousand other falsehoods from those articles on Github.
- jffhn 2y ago
  err that's Kahn
moomin 2y ago
Also Microsoft: your enum should have an explicit Unknown entry with value 0.
khana 2y ago
[dead]
sylware 2y ago
As another example: vulkan3D made the mistake to use enum in its API. Now, they must be sure it is a signed 32bits on 32 or 64 bits systems, namely check the compiler behavior. You can check the code, they always add a 0x7fffffff as the last enum value to "force" the compiler and tell developers (which have enough experience) "hey, this is a signed 32bits"... whoopsie! We should eat the bullet: remove the enum in vulkan3D, and use the appropriate primitive type for each platform ABI (not API...), so the "fix" should be transparent as it would no break the ABI. But all the "code generators" using khronos xml specifications and static source code are to be modified in one shot to stay consistent. This ain't small feat. [NOTE: enum is one of those things which should be removed from the "legacy profile" of C (like tons of keywords, integer promotion, implicit cast, etc).]
oytis 2y ago
Worth noting that in C and C++ enum-typed variable holding a value not in the enum is a UB. Had some funny bugs because of that.
qbane 2y ago
How about putting Other at the top? You can convince yourself that the value zero (or one if you like) is reserved for unknown values.
- shakna 2y ago
  This is what I tend to do. Because 0 is "default", it means "unspecified" in a lot of my API designs.
- Cthulhu_ 2y ago
  That's the Go approach, where every value is zeroed so it makes sense for enum values to have a 'none' or 'other' or 'unknown' value as the first value. (note that Go doesn't have enums as a language feature, but you can use its const declaration to create enum-like constants)
spjt 2y ago
I guess I just don't see much value in using enums at all. My most memorable experience with them is when our production system went down because a third-party service added a value to an enum.
shortrounddev2 2y ago
On a similar note: what do you think is best practice for reserving memory in a struct for future usage? For example, if you have a binary file format with a header like this: struct Header { char waterMark[3]; uint16_t width; uint16_t height; uint8_t reserved[16]; } So that you can future proof v1 binaries to still be compatible with v2 by adding empty padding on "reserved" which lets you add fields in the future. I do this sometimes and always wonder if there are other philosophies on it
mococa 2y ago
SDL does this trick https://wiki.libsdl.org/SDL2/SDL_EventType https://wiki.libsdl.org/SDL2/SDL_EventType

237 points by luu 2y ago

I usually use Unknown / Other as 0.

zdw 2y ago

I wonder how this aligns with the protobuf best practice of having the first value be UNSPECIFIED: https://protobuf.dev/best-practices/dos-donts/#unspecified-enum https://protobuf.dev/best-practices/dos-donts/#unspecified-e...

jmole 2y ago
The example code used added “other” as the last option, which was the source of the problems he described. This doesn’t happen when you make the first value in the enum unknown/unspecified
- plorkyeran 2y ago
  No, the problem described in the article is entirely unrelated to where in the enum the Other option is located. There is a different problem where keeping the Other option at the end of the enum changes the value of Other, but that is not the problem that the article is about.
  - jmole 2y ago
    Well it simplifies the logic considerably - if you see an enum value you don’t recognize (mint), you treat it as uninitialized (0). So any future new flavor will be read back as ‘0’ in older versions.
beart 2y ago
"Unspecified" is semantically different from "other". The former is more like a default value whereas the latter is actually "specified, but not one of these listed options".
- hamandcheese 2y ago
  Standard practice in protobuf is to never assign semantic meaning to the default value. I think some linters enforce that enum 0 is named "unknown" which is actually more semantically correct than "other" or "unspecified".
seeknotfind 2y ago
This is the same as a null pointer, and the requirement is very deeply tied to protobuf as it is used on large distributed systems that always need to handle version mismatch, and this advice doesn't necessarily apply to API design in general.
- eddd-ddde 2y ago
  Even in the simplest web apps you can encounter version mismatch when a client requests a response from a server that just updated.
  - hansvm 2y ago
    Hence the advice to make that situation not happen. Update the client and server to support both versions and prefer the new one, then update both to not support the old version. With load balancers and other real-world problems you might have to break that down into 4 coordinated steps.
    - Joker_vD 2y ago
      That only really works if you control the clients, or can force them to update.
      - LoganDark 2y ago
        > or can force them to update. I've used a few clients that completely lock me out for every tiniest minor version update. Very top-tier annoying imho.
        eru 2y ago
        But it does make the authors' jobs easier.
  - seeknotfind 2y ago
    This implies an API where the server has a single shared implementation. Imagine for instance that the server implements a shim for each version of the interface, then there isn't a need for the null in the API. Imagine another alternative, that the same API never adds a field, but you add a new method which takes the new type. Imagine yet again an API where you are able to version the clients in lockstep. So, it's a decision about how the API is used and evolves that recommends the API encoding or having a null default. However in a different environment or with different practices, you can avoid the null. Of course the reason to avoid the null is so that you can statically enforce this value is provided in new clients, though this also assumes your client language is typed. So in the end, protobuf teaches us, but it's not always the best in every situation.
bocahtie 2y ago
When the deserializing half of the protobuf definitions encounter an unknown value, it gets deserialized as the zero value. When that client updates, it will then be able to deserialize the new value appropriately (in this case, "Mint"). The advice on that page also specifies to not make the value semantically meaningful, which I take to mean to never set it to that value explicitly.
- chen_dev 2y ago
  > it gets deserialized as the zero value It’s more complicated: https://protobuf.dev/programming-guides/enum/ https://protobuf.dev/programming-guides/enum/ >> What happens when a program parses binary data that contains field 1 with the value 2? >- Open enums will parse the value 2 and store it directly in the field. Accessor will report the field as being set and will return something that represents 2. >- Closed enums will parse the value 2 and store it in the message’s unknown field set. Accessors will report the field as being unset and will return the enum’s default value.
  - vitus 2y ago
    Ugh. I hate how we (Google) launched proto editions. It used to be that we broadly had two sets of semantics (modulo additional customizations): proto2 and proto3. Proto editions was supposed to unify the two versions, but instead now we have the option to mix and match all of the quirks of each of the versions. And, to make matters worse, you also have language-dependent implementations that don't conform to the spec (in fact, very few implementations are conformant). C++ and Java treat everything imported by a proto2 file as closed; C#, Golang, and JS treat everything as open. I don't see a path forward for removing these custom deprecated field features, or else we'd have already begun that effort during the initial adoption of editions.
- dwattttt 2y ago
  > The advice on that page also specifies to not make the value semantically meaningful, which I take to mean to never set it to that value explicitly. I've taken to coding my C enums with the first value being "Invalid", indicating it is never intended to be created. If one is encountered, it's a bug.
MarkMarine 2y ago
I don’t mind the zero value for the proto enums, makes sense, but I require converting to my inner logic to not include this “unknown” and error during the conversion if it fails. I’ve seen engineers bring those unknowns or unspecified through to the business logic and that always made my face flush red with anger.
- fmbb 2y ago
  Why the anger? If you are consuming data from some other system you have no power over what to require from users. You will have data points with unknown properties. Say you are tracking sign ups in some other system, and they collect the users’ browser in the process, and you want to see conversion rate per browser. If the browser could not be identified, you prefer it to say ”other” instead of ”unknown”? I think I prefer the protobuf best practices way: you have a 0 ”unknown”/”unset” value, and you enumerate the rest with a unique name (and number). The enum can be expanded in the future so your code must be prepared for unknown enumerated values tagged with the new (future for your code) number. They are all unique, you just don’t yet know the name of some of the enum values. You can choose to not consume them until your code is updated with a more recent schema. Or you can reconcile later, annotating with the name of you need it. Now personally, I would not pick an enum for any set och things that is not closed when you are designing. But I’m starting to think that such sets hardly exist in the real world. Humans redefine everything over time.
- crabbone 2y ago
  I wrote my own Protobuf implementation (well, with some changes). Ditching the default values was one of the changes I made. I don't see any reason to have that. But I don't think that Protobuf is a reasonable or even decent protocol in general. It has a lot of nonsense and bad planning. Having default values is probably not in the ten worst things about Protobuf.
deleted 2y ago
[deleted]

NoboruWataya 2y ago

> Just document that the enumeration is open-ended, and programs should treat any unrecognized values as if they were “Other”. Possibly just showing my lack of knowledge here but are open-ended enumerations a common thing? I always thought the whole point of an enum is that it is closed-ended?

sd9 2y ago
I’ve worked on systems which where the set of enum values was fixed at any particular point in time, but could change over time as business requirements changed. For instance, we had an enum that represented a sport that we supported. Initially we supported some sports (say FOOTBALL and ICE_HOCKEY), and over time we added support for other sports, so the enum had to be expanded. Unfortunately this always required the entire estate to be redeployed. Thankfully this didn’t happen often. At great expense, we eventually converted this and other enums to “open-ended” enums (essentially Strings with a bit more structure around them, so that you could operate on them as if they were “real” enums). This made upgrades significantly easier. Now, whether those things should have been enums in the first place is open for debate. But that decision had been made long before I joined the team. Another example is gender. Initially an enum might represent MALE, FEMALE, UNKNOWN. But over time you might decide you have need for other values: PREFER_NOT_TO_SAY, OTHER, etc.
XorNot 2y ago
The first time you have to add a new schema value, you'll realise you needed "unknown" or similar - because during an upgrade your old systems need a way to deal with new values (or during a rollback you need to handle new entries in the database).
- sitkack 2y ago
  Your comment is the only in the entire discussion that mentions "schema". Having an "other" in a schema is a way to ensure you can run n and n+1 versions at the same time. It is Data Model design, of which API design a subset. You can only ever avoid having an other if 1) your schema is fixed and 2) if it is total over the universe of values.

o11c 2y ago

The approach in the link is fine for consumers, but for producers you really do need some way of saying "create a value that's not one of the known values". Still, there's nothing that says this needs to be pretty.

coin 2y ago

Just call it "unknown" or "unspecified" or better yet use an optional to hold the enum.

101011 2y ago
This ended up being the preferred pattern we moved into. If, like us, you were passing the object between two applications, the owning API would serialize the enum value as a String value, then we had a client helper method that would parse the string value into an Optional enum value. If the original service started transferring a new String object between services, it wouldn't break any downstream clients, because the clients would just end up with Optional empty
- janci 2y ago
  How that works when you need to distinguish between "no value provided" and "a value that is not in the list" - in some applications they have different semantics.
  - 101011 2y ago
    You could treat it as an nullable Option<SomeType>. In practice, as it relates to enums, I don't usually see 'no value provided' as a frequently used case - it's more likely that 'no value provided' maps to a more informative 'enum' value

remram 2y ago

Rust has the "non_exhaustive" attribute that lets you declare that an enum might get more fields in the future. In practice that means that when you match on an enum value, you have to add a default case. It's like a "other" field in the enum except you can't reference it directly, you use a default case. IIRC a secret 'other' field (or '__non_exhaustive' or something) is actually how we did thing before non_exhaustive was introduced.

airstrike 2y ago
TIL https://doc.rust-lang.org/reference/attributes/type_system.html https://doc.rust-lang.org/reference/attributes/type_system.h...
kibwen 2y ago
Note that the stance of the OP here is broadly in agreement with what Rust does. His main objection is this: > The word “other” means “not mentioned elsewhere”, so the presence of an Other logically implies that the enumeration is exhaustive. In Rust, because all enums are exhaustive by default and exhaustive matching is enforced by the compiler, there is no risk of this sort of confusion. And then the fact that his proposed solution is: > Just document that the enumeration is open-ended The non_exhaustive attribute is effectively compiler-enforced documentation; users now cannot forget to treat the enum as open-ended. Of course, adding non_exhaustive to Rust was not without its own detractors; it usage for any given enum fundamentally means shifting power away from library consumers (who lose the ability to guarantee exhaustive matching) and towards library authors (who gain the ability to evolve their API without causing guaranteed compilation errors in all of their users (which some users desire!)). As such, the guidance is that it should be used sparingly, mostly for things like error types. But that's an argument against open-ended enums in general, not against the mechanisms we use to achieve those (which, as you say, was already possible in Rust via hacks).
- tyre 2y ago
  Maybe there should be a compiler option or function to assert that a match is exhaustive. If the match does not handle a defined case, it blows up.

esafak 2y ago

Just add a free-form text field to hold the other value, and revise your enum as necessary, while migrating the data.

AceJohnny2 2y ago
I can't even tell if you're trolling.
bilekas 2y ago
As the other commenter mentioned, I hope this is an excersize in trolling. Please don't do this when using enums. > Free-form text firle to hold the other value > revise your enum as necessary, while migrating the dat Both instances would defeat the purpose of an enum.. As for the original post, my 2cents are valued enums. { Other = 0, Vanilla = 1, Chocolate = 2, Strawberry = 3 } In this case it allows some flexibility to add later while still being able to make use of 0 (Other). Or maybe I missed the OP's point ?
- esafak 2y ago
  With a form you will have an empirical basis to determine what the missing enum values should be, and the user won't enter any text unless necessary.
  - bilekas 2y ago
    > and the user won't enter any text unless necessary You are underestimating users' ability to ignore non strict constraints. Also when I can, why would I not increase my compile time checking ? There are very good reasons to use Enums, you seem to just want to ignore those reasons.
    - esafak 2y ago
      I'm not ignoring enumerations. I'm supplementing the existing options with a text field rather than shoehorning everything into an uninformative "other" value. I acknowledge your point about the user ignoring constraints.

1oooqooq 2y ago

jr: add other option sr: omit other option illuminated: add other option in front end only and alert when the backend crashes.

hchja 2y ago

[dead]

layer8 2y ago

Slight counterpoint: Unless there is some guarantee that the respective enum type will never ever be extended with a new value, each and every case distinction on an enum value needs to consider the case of receiving an unexpected value (like Mint in the example). When case distinctions do adhere to that principle, then the problem described doesn’t arise. On the other hand, if the above principle is adhered to as it should, then there is also little benefit in having an Other value. One minor conceivable benefit is that intermediate code can map unsupported values to Other in order to simplify logic in lower-level code. But I agree that it’s usually better to not have it. A somewhat related topic that comes to mind is error codes. There is a common pattern, used for example by the HTTP status codes, where error codes are organized into categories by using different prefixes. For example in a five-digit error code scheme, the first three digits might indicate the category (e.g. 123 for “authentication errors”), and the remaining two digits represent a more specific error condition in that category. In that setup, the all-zeros code in each category represents a generic error for that category (i.e. 12300 would be “generic authentication error”). When implementing code that detects a new error situation not covered by the existing specific error codes, the implementer has now the choice of either introducing a new error code (e.g. 12366 — this is analogous to adding a new enum value), which has to be documented and maybe its message text be localized, or else using the generic error code of the appropriate category. In any case, when error-processing code receives an unknown — maybe newly assigned — error code, they can still map it according to the category. For example, if the above 12366 is unknown, it can be handled like 12300 (e.g. for the purpose of mapping it to a corresponding error message). This is quite similar to the case of having an Other enum value, but with a better justification.

vadim_phystech 2y ago

...since the set of all possible behaviour, that is not specified, it much greater, and densier, than one would initially feel and assume, one might cause lot's of possible bad outcomes and success-breaking-points if use "Other" type in their API. Because "Other" if the 1st thing to look for vulnerabilities, for attack vectors. Because the spirit of UB the Terrible lurks there! The spirit of UB feeds upon thee juices of "Other" omnimorphic (fel) type! скверный бесформенный "ЛЮБОЙ" тип! разврат и дисгармоничность! разложение и редуцирующие гетероморфизмы! decomposition, descriptive semantic matrix rank reduction, richness degradation, devolution...empoorness...scarcity pressure increase... </shutting_the_fuck_up_my_wetware_machine_whispering_kek>

deleted 2y ago

[deleted]

akamoonknight 2y ago

One of the tactics I end up using in Verilog, for better or worse, is to define enums with a'0 value (repeat 0s for the size of the variable), and '1 value (repeat 1s for the size of the value) '0 stays as "null"-like (e.g INVALID), and '1 (which would be 0xFF in an 8 bit byte for instance) becomes "something, but I'm not sure what" (e.g. UNKNOWN). Definitely has the same issues as referenced when needing to grow the variable, and the times where it's useful aren't super common, but I do feel like the general concept of an unknown-but-not-invalid value can help with tracking down errors in processing chains Definitely do run into the need to "beware" though with enums for sure.

bob1029 2y ago

Making things into enums that shouldn't be enums is a fun trap to fall into. Much of the time what you really want is a complex type so that you can communicate these additional facts. In this case I'd do something like: class Widget { WidgetFlavor Flavor; //Undefined, Vanilla, Chocolate, Strawberry string? OtherFlavor; } This is easy to work from a consumer standpoint because if you have a deviant flavor to specify, you don't bother setting the Flavor member to anything at all. You just set OtherFlavor. Fewer moving pieces == less chance for bad times. The first (default) member in an enum should generally be something approximating "Undefined". This also makes working with serializers and databases easier.

ryanschaefer 2y ago
> Fewer moving pieces == less chance for bad times. Is this not a case for explicitly specifying all flavors? Other flavor has essentially introduced infinite moving pieces.
IshKebab 2y ago
This is not a good design. You've introduced representable invalid states (Flavor=Vanilla, Other flavor="DarkChocolate"). At the least you want this... enum Flavor { Chocolate, Banana, Strawberry, Other(String), } But that's not right either. What you really want is #[non_exhaustive] enum Flavor { Chocolate, Banana, Strawberry, } impl ToString for Flavor ...
msy 2y ago
Until you get a Widget.OtherFlavor = 'Vanilla'

DonHopkins 2y ago

https://en.wikipedia.org/wiki/Tony_Hoare#Research_and_career https://en.wikipedia.org/wiki/Tony_Hoare#Research_and_career >Speaking at a software conference in 2009, Tony Hoare apologized for inventing the null reference, his "Billion Dollar Mistake": >"I call it my billion-dollar mistake. It was the invention of the null reference in 1965. At that time, I was designing the first comprehensive type system for references in an object oriented language (ALGOL W). My goal was to ensure that all use of references should be absolutely safe, with checking performed automatically by the compiler. But I couldn't resist the temptation to put in a null reference, simply because it was so easy to implement. This has led to innumerable errors, vulnerabilities, and system crashes, which have probably caused a billion dollars of pain and damage in the last forty years." -Tony Hoare Anders Hejlsberg brilliantly points out how JavaScript doubled the cost of that mistake: >"My favorite is always the Billion-Dollar Mistake of having null in the language. And since JavaScript has both null and undefined, it's the Two-Billion-Dollar Mistake." -Anders Hejlsberg >"It is by far the most problematic part of language design. And it's a single value that -- ha ha ha ha -- that if only that wasn't there, imagine all the problems we wouldn't have, right? If type systems were designed that way. And some type systems are, and some type systems are getting there, but boy, trying to retrofit that on top of a type system that has null in the first place is quite an undertaking." -Anders Hejlsberg The JavaScript Equality Table shows how Brendan Eich simply doesn't understand equality for either data types or human beings and their right to freely choose who they love and marry: https://dorey.github.io/JavaScript-Equality-Table/ https://dorey.github.io/JavaScript-Equality-Table/ Do any languages implement the full Rumsfeld Awareness–Understanding Matrix Agnoiology, quadrupling the cost? Why stop at null, when you can have both null and undefined? Throw in unknown, and you've got a hat trick, a holy trinity of nihilistic ignorance, nothingness, and void! The Rumsfeld Awareness–Understanding Matrix Agnoiology breaks knowledge down into known knows, plus the three different types of unknowns: https://en.wikipedia.org/wiki/There_are_unknown_unknowns https://en.wikipedia.org/wiki/There_are_unknown_unknowns >"Reports that say that something hasn't happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don't know we don't know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones." -Donald Rumsfeld 1) Known knowns: These are the things we know that we know. They represent the clear, confirmed knowledge that can be easily communicated and utilized in decision-making. 2) Known unknowns: These are the things we know we do not know. This category acknowledges the presence of uncertainties or gaps in our knowledge that are recognized and can be specifically identified. 3) Unknown knowns: Things we are not aware of but do understand or know implicitly 4) Unknown unknowns: These are the things we do not know we do not know. This category represents unforeseen challenges and surprises, indicating a deeper level of ignorance where we are unaware of our lack of knowledge. https://en.wikipedia.org/wiki/Agnoiology https://en.wikipedia.org/wiki/Agnoiology >Agnoiology (from the Greek ἀγνοέω, meaning ignorance) is the theoretical study of the quality and conditions of ignorance, and in particular of what can truly be considered "unknowable" (as distinct from "unknown"). The term was coined by James Frederick Ferrier, in his Institutes of Metaphysic (1854), as a foil to the theory of knowledge, or epistemology. I don't know if you know, but Microsoft COM hinges on the IUnknown interface. Microsoft COM's IUnknown interface takes the Rumsfeldian principle to heart: it doesn't assume what an object is but provides a structured way to query for knowledge (or interfaces). In a way, it models known unknowns, since a caller knows that an interface might exist but must explicitly ask if it does. Then there's Schulz's Known Nothing Nesiology, representing the existential conclusion of all this: when knowledge itself is questioned, where does that leave us? Right back at JavaScript's Equality Table, which remains an unfathomable unknown unknown to Brendan Eich and his well known but knowingly ignorant War on Equality. https://www.youtube.com/watch?v=HblPucwN-m0 https://www.youtube.com/watch?v=HblPucwN-m0 Nescience vs. Ignorance (on semantics and moral accountability): https://cognitive-liberty.online/nescience-vs-ignorance/ https://cognitive-liberty.online/nescience-vs-ignorance/ >From a psycholinguistic vantage point, the term “ignorance” and the term “nescience” have very different semantic connotations. The term ignorance is more generally more widely colloquially utilized than the term nescience and it is often wrongly used in contexts where the word nescience would be appropriate. “Ignorance” is associated with “the act of ignoring”. Per contrast, “nescience” means “to not know” (viz., Latin prefix ne = not, and the verb scire = “to know”; cf. the etymology of the word “science”/prescience). >As Mark Passio points out, the important underlying question which can be derived from this semantic distinction pertains to whether our individual and global problems are caused by “ignorance” or “nescience”? That is, “ignoring” or “not knowing”? It seems clear that it is the later. We know about the truth but we actively ignore it for the most part. Currently people have all the necessary information available (literally at their fingertips). Ignoring the facts is a decision, an irrational decision, and people can be held accountable for this decision. Nescience, on the other hand, acquits from accountability (i.e., someone cannot be held accountable when he/she for not knowing something but for ignoring something). Quasi-Freudian suppression plays a pivotal role in this scenario. Suppression is very costly in energetic terms. The energy and effort which is used for suppression lacks elsewhere (cf. prefrontal executive control is based on limited cognitive resources). The suppression of truth through the act of active ignoring thus has negative implications on multiple levels – on the individual and the societal level, the cognitive and the political, the psychological and the physiological. Brendan: While we can measure the economic consequences of your culpably ignorant mistakes of both bad programming language design and marriage inequality in billions of dollars, the emotional, social, and moral costs of the latter -- like diminished human dignity and the perpetuation of discrimination -- are, by their very nature, priceless. Ultimately, these deeper impacts underscore that the fight for marriage equality, defending against the offensive uninvited invasion of your War on Equality into other people's marriages, is about much more than economics; it’s about ensuring fairness, respect, and equality for all members of society.

sgondala_ycapp 2y ago

Random tidbit: We use LLM to identify document types and use an enum to show a list of options. Initially, we didn’t include an "Other" category - which led the LLM to force-fit documents into existing types even when they didn’t belong. Obv this wasn't LLM's fault. We realized the mistake and added "Other". This significantly improved output accuracy!

dataflow 2y ago

I think there are multiple concerns here, and they need to be analyzed separately -- they don't converge to the same solution: - Naming: "Other" should probably be called "Unrecognized" in these situations. Then users understand that members may not be mutually exclusive. - ABI: If you need ABI compatibility, the constraint you have is "don't change the meanings of values or members", which is somewhat stronger. The practical implication is that if you do need to have an Other value, its value should be something out of range of possible future values. - Protocol updates: If you can atomically update all the places where the enum is used, then there's no inherent need to avoid Other values. Instead, you can use compile-time techniques (exhaustive switch statements, compiler warnings, temporarily removing the Other member, grep, clang-query, etc.) to find and update the usage sites at compile time. This requires being a little disciplined in how you use the enum during development, but it's doable. - Distributed code: If you don't have control over all the code using your enum might, then you must avoid an Other value, unless you can somehow ensure out-of-band that users have updated their code.

KPGv2 2y ago

> Rust has the "non_exhaustive" attribute that lets you declare that an enum might get more fields in the future.a Is there a reason, aside from documentation, that this is ever desirable? I rarely program in Rust, but why would this ever be useful in practice, outside of documentation? (Seems like code-as-documentation gone awry when your code is doing nothing but making a statement about future code possibilities)

LegionMammal978 2y ago
Normally, when you match on the value of an enum, Rust forces you to either add a case for every possible variant, or add a default arm "_ => ..." that acts as a 'none of the above' case. This is called exhaustiveness checking [0]. When you add #[non_exhaustive] to an enum, the compiler says to external users, "You're no longer allowed to just match every existing variant. You must always have a default 'none of the above' case when you're matching on this enum." This lets you add more variants in the future without breaking the API for existing users, since they all have a 'none of the above' case for the new variants to fall into. [0] https://doc.rust-lang.org/book/ch06-02-match.html#matches-are-exhaustive https://doc.rust-lang.org/book/ch06-02-match.html#matches-ar...
jeroenhd 2y ago
If your library processes data from another language, you'll probably need to deal with the possibility that the library returns open ended enums. I believe I've also seen this declaration for generated bindings for a JSON API that promises backwards compatibility for calls and basic functionality at least. Future versions may include more options, but the code will still compile fine against the older API. I don't think it's a great tool to use everywhere, but there are edge cases where Rust's demand for exhaustive matches conflicts with the non-Rust world, and that's where stuff like this becomes hard to avoid.

hello12343214 2y ago

Good idea. I appreciate that he thought through future compatibility with old versions.

mkleczek 2y ago

I've had a short discussion with Brian Goetz about a similar case (sealed types in Java): https://mail.openjdk.org/pipermail/amber-dev/2020-April/005844.html https://mail.openjdk.org/pipermail/amber-dev/2020-April/0058... I wonder when we are going to re-discover OOP style dynamic dispatch (or even better: multiple dispatch) to deal with software evolution.

jasonkester 2y ago

This got me wondering what I actually do in practice. I think it's this: const KnownFlavors { Vanilla: "Vanilla", Chocolate: "Chocolate", Strawberry: "Strawberry" } Then, use a string to hold the actual value. doug.favoriteFlavor = KnownFlavors.Chocolate; cindy.favoriteFlavor = "Mint" case: KnownFlavors.Chocolate: Expand your list of known flavors whenever you like, your system will still always hold valid data. You get all the benefits of typo-proofing your code, switching on an enum, etc., without having to pile on any wackiness to fool your compiler or keep the data normalized. It acknowledges the reality that a non-exhaustive enum isn’t really an enum. It’s just a list of things that people might type into that field.

Boldened15 2y ago
Sorry I don't get the example, are both code blocks meant to be client-side code? > It acknowledges the reality that a non-exhaustive enum isn’t really an enum. It’s just a list of things that people might type into that field. I would say the opposite, the kinds of enums that map a case to a few hardcoded branches (SUCCESS, NETWORK_ERROR, API_ERROR) are often an approximation of algebraic data types which Rust implements as enums [0] but not most languages or data formats. Since often using those will require something like a `nullthrows($response->getNetworkError())` once you've matched the enum case. The kind of enum that's just a string whitelist, like flavors or colors, which you can freely pass around and store, likely converting it into a human-readable string or RGB values in one or two utils, is the classic kind of enum to me. [0] https://doc.rust-lang.org/std/keyword.enum.html https://doc.rust-lang.org/std/keyword.enum.html
PartiallyTyped 2y ago
The way we do this with AWS SDK in rust is by leveraging #non_exhaustive, and matching the (_@other) pattern, this is forward compatible and allows us to do something like ( _@other) if other.name() == “foo” for known cases without upgrading down the road or if the user uses an older version than our API.

kstenerud 2y ago

I use the "other" technique when it's necessary for the user to be able to mix in their own: enum WidgetFlavor { Vanilla, Chocolate, Strawberry, Other=10000, }; Now users can add their own (and are also responsible for making sure it works in all APIs): enum CustomWidgetFlavor { RockyRoad=Other, GroovyGrape, Cola, }; And now you can amend the enum without breaking the client: enum WidgetFlavor { Vanilla, Chocolate, Strawberry, Mint, Other=10000, };

qingcharles 2y ago
It's code like this that ends in a terminal choco-banana shake hang: http://www.technofileonline.com/texts/chocobanana.gif http://www.technofileonline.com/texts/chocobanana.gif
- fingerlocks 2y ago
  What is the context here? Is this just a silly nonsense tech support page/meme or an actual product from the late 90s?
  - sandblast 2y ago
    Is the context not clear for you from the information that the article applies to "DreamWorks Interactive, Someone's in the Kitchen, version 1.0" and other clues that it's a game?
  - qingcharles 2y ago
    It's a real Knowledge Base article from Microsoft. I can't find the original, but it is on archive.org somewhere if you dig around.
dominicrose 2y ago
"Other" doesn't mean the same thing before and after Cola has been added. "Unknown" would be more accurate. My personal opinion would be to make the enum nullable and not add a fake value.

jffhn 2y ago

>"programs should treat any unrecognized values as if they were “Other”" Having such an "Other" value does not prevent from considering that the enum is open-ended, and it simplifies a lot all the code that has to deal with potentially invalid or unknown values (no need for a validity flag or null). That's probably why in DIS (Distributed Interactive Simulation) standard, which defines many enums, all start with OTHER, which has the value zero. In STANAGs (NATO standards), the value zero is used for NO_STATEMENT, which can also be used when the actual value is in the enum but you can't or don't need to indicate it. I remember an "architecture astronaut" who claimed that NO_STATEMENT was not a domain value, and removed it from all the enums in its application. That did not last long. That also reminds me of Philippe Khan (Bordland) having in some presentation the ellipse extend the circle, to add a radius. A scientist said he would do the other way around, and Khan replied: "This is exactly the difference between research and industry".

ivan_gammel 2y ago
>That also reminds me of Philippe Khan (Bordland) having in some presentation the ellipse extend the circle, to add a radius. A scientist said he would do the other way around, and Khan replied: "This is exactly the difference between research and industry". My favorite question on interviews on the OOP topic. It can be correct either way or both can be wrong, so the good answer would be "It depends". When developers rush to give a specific answer, they do not demonstrate due attention to the domain and it may mean that they will assume thousand other falsehoods from those articles on Github.
jffhn 2y ago
err that's Kahn

moomin 2y ago

Also Microsoft: your enum should have an explicit Unknown entry with value 0.

khana 2y ago

[dead]

sylware 2y ago

As another example: vulkan3D made the mistake to use enum in its API. Now, they must be sure it is a signed 32bits on 32 or 64 bits systems, namely check the compiler behavior. You can check the code, they always add a 0x7fffffff as the last enum value to "force" the compiler and tell developers (which have enough experience) "hey, this is a signed 32bits"... whoopsie! We should eat the bullet: remove the enum in vulkan3D, and use the appropriate primitive type for each platform ABI (not API...), so the "fix" should be transparent as it would no break the ABI. But all the "code generators" using khronos xml specifications and static source code are to be modified in one shot to stay consistent. This ain't small feat. [NOTE: enum is one of those things which should be removed from the "legacy profile" of C (like tons of keywords, integer promotion, implicit cast, etc).]

oytis 2y ago

Worth noting that in C and C++ enum-typed variable holding a value not in the enum is a UB. Had some funny bugs because of that.

qbane 2y ago

How about putting Other at the top? You can convince yourself that the value zero (or one if you like) is reserved for unknown values.

shakna 2y ago
This is what I tend to do. Because 0 is "default", it means "unspecified" in a lot of my API designs.
Cthulhu_ 2y ago
That's the Go approach, where every value is zeroed so it makes sense for enum values to have a 'none' or 'other' or 'unknown' value as the first value. (note that Go doesn't have enums as a language feature, but you can use its const declaration to create enum-like constants)

spjt 2y ago

I guess I just don't see much value in using enums at all. My most memorable experience with them is when our production system went down because a third-party service added a value to an enum.

shortrounddev2 2y ago

On a similar note: what do you think is best practice for reserving memory in a struct for future usage? For example, if you have a binary file format with a header like this: struct Header { char waterMark[3]; uint16_t width; uint16_t height; uint8_t reserved[16]; } So that you can future proof v1 binaries to still be compatible with v2 by adding empty padding on "reserved" which lets you add fields in the future. I do this sometimes and always wonder if there are other philosophies on it

mococa 2y ago

SDL does this trick https://wiki.libsdl.org/SDL2/SDL_EventType https://wiki.libsdl.org/SDL2/SDL_EventType