7 ms·

Having used enums with large teams across many distributed systems over the years, enums are my least favorite type in that setting, because it's very easy to w

by cle 6y ago

Having used enums with large teams across many distributed systems over the years, enums are my least favorite type in that setting, because it's very easy to write an enum that does not tolerate unknown/new values (they usually don't by default, which seems to be the case for Rust as well). This often makes the system brittle to new changes because subsystems have very strict expectations of data schema, even when they don't really need to care about it (e.g. when they're just passing data around). I've seen a lot of attempts on these teams to ensure these mistakes aren't made (special parsing libraries, using default fallback values, etc.), but someone always manages to sneak in an enum somewhere where they shouldn't and then something blows up 6 months later when a new value is introduced. Enthusiastic engineers will spend a lot of time trying to figure out how to add an over-strict enum, because hey, who doesn't want type safety?

dwaltrip 6y ago
Rust prevents the program from compiling if a new enum variant was not handled, unless one uses a catch all in the match expression. Did the codebase have an over-usage of catch alls? Or were you using a different language that doesn’t catch enum errors during compilation?
- ori_b 6y ago
  What happens when you have a mix of new and old binaries being incrementally rolled out?
  - dwaltrip 6y ago
    Good point, but isn’t that a more general code compatibility issue that applies to many things beyond just enums?
- Tyr42 6y ago
  I think the key words here are "distributed system". Just because HEAD has the new enum used everywhere doesn't mean all the deployed code everywhere can handle the new enum value. Imagine you have two servers communicating with enum Packet { Ping, Pong, } Where the semantics are that if you receive a ping, you say pong, and if you see pong, you say ping. Suppose we deploy this out to several servers, with code built on 2020-7-12. Now on 2020-07-13 you commit to head a new enum value `Stats`, which returns a new kind of reply. If you send outgoing requests in the same 2020-07-13 build, you're going to have a bad time, as your fleet is going to start at 100% 2020-07-12 and switch over to 100% 2020-07-13, but during that switch you'll have some 50% 50% split. If a new server sends a request to an old one, you'll have a crash. Instead you have to add receivers in 2020-07-13, but not use it yet. Roll it out, make sure it's past how far you might roll back, then you can deploy a 2020-07-20 build which makes calls using the new enum. If you skip that step, you're going to have a bad time. Maybe you want to instead have a default case handler instead of crashing, but that means each of your enums has to know about "UNKNOWN" and you're deserialization error has to be future proof.
  - woah 6y ago
    Any time you break an API that accepts serialized data it's going to crash if you try to use it with something that is on a different version of the API. How is this any different with enums than it would be with changing a string to a float, or anything else?
  - zenhack 6y ago
    Related, capnproto has a corresponding feature (which it calls unions), but being a dsl for defining _protocols_, you obviously have to deal with the possibility of introducing new variants as the protocol evolves. I wrote the Haskell implementation. It maps unions to variants (what Rust calls enums), BUT it always generates an extra 'unkown' variant, which gets used whenever the variant is one not recognized by the generated code. In this case the value proposition of variants/enums, including exhaustiveness checking, is still really useful -- it can not only deal with the possibility of more variants added in the future, it forces you to handle that possibility, which imo is the best of both worlds. But yes, in general, when modeling data that comes from the outside world, you need to about overfitting. Vaguely related: https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-type-systems-are-not-inherently-more-open/ https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-typ...
Tyr42 6y ago
I had some problem like this using protobuf2 enums The way protobuf handles enums, it recommends having an "UNKNOWN" as value 0, as when deserializing from the wire any unknown enum value gets turned into value 0. So it's recommended to have something like enum FailReason { UNKNOWN_ERROR = 0, ERROR_1 = 1, ERROR_2 = 2, etc } But someone long ago made the enum I was dealing with look like this enum { SUCCESS = 0, ERROR_1 = 1, ERROR_2 = 2, etc. } But this means I can't easily add a new error case, as it'll get interpreted as a success unless I carefully roll out the change to the receiving servers first. But if I wasn't dealing with a distributed system I would like to have exhaustive checking.
- onei 6y ago
  I don't think that's protobuf's fault, it's simply a side effect of the common pattern in C of using memset when you initialise structs. I don't think there's a really good way around it other than remembering to think like a C programmer every now and then. I guess one option is to replace the endpoints that use the enum, but that seems heavy handed if it's wide spread.
woah 6y ago
I'm not sure I understand. If you have some code which doesn't care about the types which it is passing around, why isn't it generic? Also, I'm not sure if you understand Rust enums. They should probably be called union types, because an enum in Rust signifies that the value can be one of a number of different types. This is extremely useful for code which has to handle a few different things differently. It's much more useful than the standard concept of an enum in languages such as Java and Rust, where it's just a dressed up integer flag.
deleted 6y ago
[deleted]
yawaramin 6y ago
Never trust anything coming in over the network. Always decode with the possibility of failure. I.e., a decoder function from a byte array would return a `Result<EnumType, Error>` where `Error` should usefully describe the failure. Sorry to say but JavaScript is the only ecosystem I've seen where people blithely `JSON.parse` anything they get over the network and directly access the JSON structure expecting everything they want to be there without checking. Enum types are supposed to be strict. Decoding raw data into enums can always fail but the failure should be handled at a higher level.
Dowwie 6y ago
What I'm referring to are fields that must be one of the expected values. It seems like you're talking about a scenario that doesn't meet that criteria. One of five values or null is simply an Option<MyEnum>.
nilkn 6y ago
I am not sure this has anything to do with enums. When sending serialized data over the network you always have to have logic for dealing with the situation where what you get isn't quite what you expected. Not doing that is what causes things to break, not using enums.
- cle 6y ago
  In my experience, these kind of bugs happen overwhelmingly with enums. The nature of enums makes this mistake pretty easy to make. Enums are otherwise fantastic tools, I’m just pointing out that in certain common contexts they have some ergonomic problems.
  - deleted 6y ago
    [deleted]
mamcx 6y ago
> This often makes the system brittle to new changes because subsystems have very strict expectations of data schema, even when they don't really need to care about i I deal a lot with this, because I interface with software outside my control. In the case of Rust I find super easy to solve, just use this litte trick: https://serde.rs/attr-flatten.html https://serde.rs/attr-flatten.html #[derive(Serialize, Deserialize)] struct User { id: String, username: String, #[serde(flatten)] extra: HashMap<String, Value>, } Serde is very good. Near all you want is there.
- Dowwie 6y ago
  this is a great recipe

by cle 6y ago

dwaltrip 6y ago

Rust prevents the program from compiling if a new enum variant was not handled, unless one uses a catch all in the match expression. Did the codebase have an over-usage of catch alls? Or were you using a different language that doesn’t catch enum errors during compilation?

ori_b 6y ago
What happens when you have a mix of new and old binaries being incrementally rolled out?
- dwaltrip 6y ago
  Good point, but isn’t that a more general code compatibility issue that applies to many things beyond just enums?
Tyr42 6y ago
I think the key words here are "distributed system". Just because HEAD has the new enum used everywhere doesn't mean all the deployed code everywhere can handle the new enum value. Imagine you have two servers communicating with enum Packet { Ping, Pong, } Where the semantics are that if you receive a ping, you say pong, and if you see pong, you say ping. Suppose we deploy this out to several servers, with code built on 2020-7-12. Now on 2020-07-13 you commit to head a new enum value `Stats`, which returns a new kind of reply. If you send outgoing requests in the same 2020-07-13 build, you're going to have a bad time, as your fleet is going to start at 100% 2020-07-12 and switch over to 100% 2020-07-13, but during that switch you'll have some 50% 50% split. If a new server sends a request to an old one, you'll have a crash. Instead you have to add receivers in 2020-07-13, but not use it yet. Roll it out, make sure it's past how far you might roll back, then you can deploy a 2020-07-20 build which makes calls using the new enum. If you skip that step, you're going to have a bad time. Maybe you want to instead have a default case handler instead of crashing, but that means each of your enums has to know about "UNKNOWN" and you're deserialization error has to be future proof.
- woah 6y ago
  Any time you break an API that accepts serialized data it's going to crash if you try to use it with something that is on a different version of the API. How is this any different with enums than it would be with changing a string to a float, or anything else?
- zenhack 6y ago
  Related, capnproto has a corresponding feature (which it calls unions), but being a dsl for defining _protocols_, you obviously have to deal with the possibility of introducing new variants as the protocol evolves. I wrote the Haskell implementation. It maps unions to variants (what Rust calls enums), BUT it always generates an extra 'unkown' variant, which gets used whenever the variant is one not recognized by the generated code. In this case the value proposition of variants/enums, including exhaustiveness checking, is still really useful -- it can not only deal with the possibility of more variants added in the future, it forces you to handle that possibility, which imo is the best of both worlds. But yes, in general, when modeling data that comes from the outside world, you need to about overfitting. Vaguely related: https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-type-systems-are-not-inherently-more-open/ https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-typ...

Tyr42 6y ago

I had some problem like this using protobuf2 enums The way protobuf handles enums, it recommends having an "UNKNOWN" as value 0, as when deserializing from the wire any unknown enum value gets turned into value 0. So it's recommended to have something like enum FailReason { UNKNOWN_ERROR = 0, ERROR_1 = 1, ERROR_2 = 2, etc } But someone long ago made the enum I was dealing with look like this enum { SUCCESS = 0, ERROR_1 = 1, ERROR_2 = 2, etc. } But this means I can't easily add a new error case, as it'll get interpreted as a success unless I carefully roll out the change to the receiving servers first. But if I wasn't dealing with a distributed system I would like to have exhaustive checking.

onei 6y ago
I don't think that's protobuf's fault, it's simply a side effect of the common pattern in C of using memset when you initialise structs. I don't think there's a really good way around it other than remembering to think like a C programmer every now and then. I guess one option is to replace the endpoints that use the enum, but that seems heavy handed if it's wide spread.

woah 6y ago

I'm not sure I understand. If you have some code which doesn't care about the types which it is passing around, why isn't it generic? Also, I'm not sure if you understand Rust enums. They should probably be called union types, because an enum in Rust signifies that the value can be one of a number of different types. This is extremely useful for code which has to handle a few different things differently. It's much more useful than the standard concept of an enum in languages such as Java and Rust, where it's just a dressed up integer flag.

deleted 6y ago

[deleted]

yawaramin 6y ago

Never trust anything coming in over the network. Always decode with the possibility of failure. I.e., a decoder function from a byte array would return a `Result<EnumType, Error>` where `Error` should usefully describe the failure. Sorry to say but JavaScript is the only ecosystem I've seen where people blithely `JSON.parse` anything they get over the network and directly access the JSON structure expecting everything they want to be there without checking. Enum types are supposed to be strict. Decoding raw data into enums can always fail but the failure should be handled at a higher level.

Dowwie 6y ago

What I'm referring to are fields that must be one of the expected values. It seems like you're talking about a scenario that doesn't meet that criteria. One of five values or null is simply an Option<MyEnum>.

nilkn 6y ago

I am not sure this has anything to do with enums. When sending serialized data over the network you always have to have logic for dealing with the situation where what you get isn't quite what you expected. Not doing that is what causes things to break, not using enums.

cle 6y ago
In my experience, these kind of bugs happen overwhelmingly with enums. The nature of enums makes this mistake pretty easy to make. Enums are otherwise fantastic tools, I’m just pointing out that in certain common contexts they have some ergonomic problems.
- deleted 6y ago
  [deleted]

mamcx 6y ago

> This often makes the system brittle to new changes because subsystems have very strict expectations of data schema, even when they don't really need to care about i I deal a lot with this, because I interface with software outside my control. In the case of Rust I find super easy to solve, just use this litte trick: https://serde.rs/attr-flatten.html https://serde.rs/attr-flatten.html #[derive(Serialize, Deserialize)] struct User { id: String, username: String, #[serde(flatten)] extra: HashMap<String, Value>, } Serde is very good. Near all you want is there.

Dowwie 6y ago
this is a great recipe