3 ms·
We had a cluster of servers, dynamically scaling up and down in response to load, and one day started seeing errors where an enum string field had an impossible
by bluesnowmonkey 1y ago
We had a cluster of servers, dynamically scaling up and down in response to load, and one day started seeing errors where an enum string field had an impossible value. Imagine the field is supposed to be "FOO" or "BAR" but one day in the logs you start seeing "FOO", "BAR", "GOO", and "CAR". Impossible. "GOO" and "CAR" did not exist in the code, nothing manipulated these strings, yet there they were.
Long story short, a particular machine that joined the cluster that morning had some kind of CPU or memory flaw that flipped a bit sometimes. Our Elixir server was fine because we were matching on valid values. Imagine a typed language compiler that makes assumptions about things that "can't" happen because the code says it can't... yet it does.
- 3eb7988a1663 1y agoI am not sure I want the system to continue operating in that scenario. You have corrupted hardware that could be trashing other records. What if it is flipping bits on financial transactions?
- spooky_deep 1y agoDepends how you use the statically typed system. For example match parseFoo json with | Ok foo -> process foo | Error message -> print message This will skip bad messages and be statically typed in the valid case.
- bccdee 1y agoThat only works for enum discriminants. What if it were modifying the data payload, producing valid but corrupt messages? The only way to tolerate that kind of error is checksums on your data, and I don't think any language has that kind of redundancy built in.