7 ms·

Zero-Cost Exceptions and Error Interfaces

36 points by rekahrv 4y ago

Rygian 4y ago
How is "dictionary[key] = None" equivalent to dictionary[key] not existing?
- masklinn 4y ago
  In general it’s not, although for a number of languages (e.g. java, ruby, go) it kinda is. Here the dictionary is supposed to hold Item values, not Optional[Item], so None would be the signal of a missing item. Think Lookup k -> Map k a -> Maybe a Not sure how that came about tho, as the article specifically argues against returning a None for missing entries?
- dontlaugh 4y ago
  Depends on the language. In Python, dictionary[key] will raise an exception if the key doesn't exist. If there is a value it's returned, even if that value is None.
- quietbritishjim 4y ago
  The article never claims that anywhere. That makes it hard to interpret your comment because there are a few places where the article slightly conflates a non-existent key with value == None and maybe you're obliquely referring to one of those, but I'm not sure which one. All of them are OK so long as the dictionary doesn't have having any values equal to None, which it should really spell out but is obviously implied. Actually, most of them even require that none of the values are falsey (e.g. 0, empty list, None, or indeed False). It would actually be possible to work around that but still fundamentally use the same ideas by using a sentinel value: # At file scope, not exposed _not_present_sentinel = object() # Version B def read(self, id_: str) -> Item: result = self.items.get(id_, _not_present_sentinel) if result is _not_present_sentinel: raise ItemNotFound(id_) return result
  - Rygian 4y ago
    The article starts with a poll showing Versions A), B) and C). Implicit in that poll would be that all three "versions" produce the same behavior. Options A) and B) conflate None with non-existent key: if the key is set and has value None, they will return ItemNotFound. Option C) on the other hand treats them separately and will happily return None (instead of raising exception) when the key exists and its value is None.
BoorishBears 4y ago
I like how the closest thing to a "correct" answer gets dismissed almost immediately with fairly weak reasoning. You're adding type hints so they're not going to be blind sided anyways, just return a union of Item and None. If None is a valid value then use a result type. Exceptions are for exceptional cases isn't just a performance thing, it's a reasoning thing: You're jumping out of the established control flow and there should be a very good reason for that. If you're struggling with handling None even when you know it's a valid return value, living in a codebase that overuses exceptions is going to be 10x worse.
- moring 4y ago
  You are assuming that a missing item is a normal (as in "not exceptional") case, but that is entirely application-dependent. But then, the article doesn't give any context either -- it just throws an exception in every variant of the code, so the most likely assumption here is that missing items are exceptional.
  - qprofyeh 4y ago
    To be honest, the example given (a "read" method that wraps direct access and raises a custom exception) does not inspire confidence that the item is expected to be there more often than not. There's certainly too little context to assume anything. Expected or unexpected, either way there are two paths; I would return a Result/Option type.
  - BoorishBears 4y ago
    Except the article quite literally say None is an option, except <insert not great reasons>. That wasn't an assumption on my part. If the missing item wasn't exceptional they would have just said that. - Also if even if it were exceptional, you'd still served extremely well by a Result type. Instead of just guessing where the landmine is sitting the type system can help you. Python explicitly opted out of type hints for exceptions to avoid a Java-style "checked exceptions" world.
- UncleEntity 4y ago
  > Exceptions are for exceptional cases isn't just a performance thing, it's a reasoning thing: You're jumping out of the established control flow and there should be a very good reason for that. Python uses exceptions for all sorts of things like to signal the end of an iteration which isn’t exceptional at all. That ship sailed a long time ago.
peterkelly 4y ago
Exceptions are an anti-pattern. If a function can encounter an error, it should expose this fact through its interface (specifically its return type). For example, Rust has a Result type [1], so this function would return Result<Item, ItemNotFound>. This forces the caller to think about what to do in case of an error and specify what should happen. [1] https://doc.rust-lang.org/std/result/index.html https://doc.rust-lang.org/std/result/index.html
- bhawks 4y ago
  Rust also has a much more rigorous type system than other languages - so yes in rust exceptions should be considered an antipattern. The implementation proposed in the article seems very reasonable python. Writing code for language X in the style of language Y is usually an antipattern. When Rome do as the Romans do.
- moring 4y ago
  Even Rust has panic for things like out-of-bounds access. In fact, Rust's panic / Result is actually very similar technically to Java's checked / unchecked exceptions. I'd argue that this is more a difference in usage and culture, with likely influence from the fact that Java makes it easy to catch unchecked exceptions, so many prefer unchecked exceptions to avoid being forced to think about what to do in case of an error (which, of course, means that probably the wrong thing happens then). Also, Rust culture (IMHO) tends to be more strict about technical errors, probably because it is applied to more low-level problem areas. If high-level business logic is cluttered with technical error handling in every other line, it becomes near impossible to read and maintain, so other patterns are applied (e.g. atomic transactions that are undone when something fails).
  - masklinn 4y ago
    > I'd argue that this is more a difference in usage and culture Usage and culture and affordances, as in: - the stdlib doesn’t signal errors by panic-ing - rust learning documents drive users towards Result and Option - the standard library provides APIs to manipulate results in ways Java is utterly missing e.g. map_err, ok, ok_or, ?, … - because results are reified values, the language allows abstracting over results (creating macros or functions around them), as well as providing utilities around them (e.g. eyre, thiserror, anyhow, …) - because results are reified values, they can be ignored until relevant, an `Iterator<Item = Result<…>>` is just an `Iterator<Item = _>` until you actually need to process the result. - panics have very little signaling power, technically your panic can carry a payload but I’ve never seen anyone go beyond a message Also rust panics are explicitly noted to not necessarily be recoverable from, so they can not be used for error signaling: whoever compiles the program can set panic=abort, and now you have no error handling at all, the program just straight dies. And even if you actually, genuinely, want to use checked exceptions in Java, it’s hell, because it does not mesh with the rest of the language. Just look at the contorsions necessary to get checked exceptions through Stream APIs. Checked exceptions are not necessarily bad in and of themselves, but Java’s implementation is so awful it has completely poisoned the field. And frankly it just… doesn’t seem like a very useful thing to put work into: you can just use exceptions-flavoured results à la Swift or Zig if you like the “look” of exceptions.
lawxls 4y ago
Just a small suggestion to consider: If the title was "Zero-Cost Exceptions and Error Interfaces in Python" I would have received it in my personal Hacker News feed in Telegram (https://github.com/lawxls/HackerNews-personalized https://github.com/lawxls/HackerNews-personalized) as I do have "Python" keyword added. And there's also email notification services people use which might have Python as a keyword. Thanks for providing further resources btw, added Sebastian Witowski presentation to "watch later" list :)
- rekahrv 4y ago
  Thanks a lot, that's a very helpful tip. :-) We'll adjust the title for future posts. (And I'll set up such a keyword-based notification for myself. :-) Sebastian's talk is awesome indeed. I learned a lot from it about how Python works without going too much into the implementation details.
randyrand 4y ago
As far as perf goes, a callee raising an exception is never the answer. The perf arguments are a sham imo. A callee cannot know how many times it is going to be called, only a caller, or a caller's caller, etc, can know that.
mypy_abuser 4y ago
def read(self, id_: str) -> Item | None: ...

36 points by rekahrv 4y ago

Rygian 4y ago

How is "dictionary[key] = None" equivalent to dictionary[key] not existing?

masklinn 4y ago
In general it’s not, although for a number of languages (e.g. java, ruby, go) it kinda is. Here the dictionary is supposed to hold Item values, not Optional[Item], so None would be the signal of a missing item. Think Lookup k -> Map k a -> Maybe a Not sure how that came about tho, as the article specifically argues against returning a None for missing entries?
dontlaugh 4y ago
Depends on the language. In Python, dictionary[key] will raise an exception if the key doesn't exist. If there is a value it's returned, even if that value is None.
quietbritishjim 4y ago
The article never claims that anywhere. That makes it hard to interpret your comment because there are a few places where the article slightly conflates a non-existent key with value == None and maybe you're obliquely referring to one of those, but I'm not sure which one. All of them are OK so long as the dictionary doesn't have having any values equal to None, which it should really spell out but is obviously implied. Actually, most of them even require that none of the values are falsey (e.g. 0, empty list, None, or indeed False). It would actually be possible to work around that but still fundamentally use the same ideas by using a sentinel value: # At file scope, not exposed _not_present_sentinel = object() # Version B def read(self, id_: str) -> Item: result = self.items.get(id_, _not_present_sentinel) if result is _not_present_sentinel: raise ItemNotFound(id_) return result
- Rygian 4y ago
  The article starts with a poll showing Versions A), B) and C). Implicit in that poll would be that all three "versions" produce the same behavior. Options A) and B) conflate None with non-existent key: if the key is set and has value None, they will return ItemNotFound. Option C) on the other hand treats them separately and will happily return None (instead of raising exception) when the key exists and its value is None.

BoorishBears 4y ago

I like how the closest thing to a "correct" answer gets dismissed almost immediately with fairly weak reasoning. You're adding type hints so they're not going to be blind sided anyways, just return a union of Item and None. If None is a valid value then use a result type. Exceptions are for exceptional cases isn't just a performance thing, it's a reasoning thing: You're jumping out of the established control flow and there should be a very good reason for that. If you're struggling with handling None even when you know it's a valid return value, living in a codebase that overuses exceptions is going to be 10x worse.

moring 4y ago
You are assuming that a missing item is a normal (as in "not exceptional") case, but that is entirely application-dependent. But then, the article doesn't give any context either -- it just throws an exception in every variant of the code, so the most likely assumption here is that missing items are exceptional.
- qprofyeh 4y ago
  To be honest, the example given (a "read" method that wraps direct access and raises a custom exception) does not inspire confidence that the item is expected to be there more often than not. There's certainly too little context to assume anything. Expected or unexpected, either way there are two paths; I would return a Result/Option type.
- BoorishBears 4y ago
  Except the article quite literally say None is an option, except <insert not great reasons>. That wasn't an assumption on my part. If the missing item wasn't exceptional they would have just said that. - Also if even if it were exceptional, you'd still served extremely well by a Result type. Instead of just guessing where the landmine is sitting the type system can help you. Python explicitly opted out of type hints for exceptions to avoid a Java-style "checked exceptions" world.
UncleEntity 4y ago
> Exceptions are for exceptional cases isn't just a performance thing, it's a reasoning thing: You're jumping out of the established control flow and there should be a very good reason for that. Python uses exceptions for all sorts of things like to signal the end of an iteration which isn’t exceptional at all. That ship sailed a long time ago.

peterkelly 4y ago

Exceptions are an anti-pattern. If a function can encounter an error, it should expose this fact through its interface (specifically its return type). For example, Rust has a Result type [1], so this function would return Result<Item, ItemNotFound>. This forces the caller to think about what to do in case of an error and specify what should happen. [1] https://doc.rust-lang.org/std/result/index.html https://doc.rust-lang.org/std/result/index.html

bhawks 4y ago
Rust also has a much more rigorous type system than other languages - so yes in rust exceptions should be considered an antipattern. The implementation proposed in the article seems very reasonable python. Writing code for language X in the style of language Y is usually an antipattern. When Rome do as the Romans do.
moring 4y ago
Even Rust has panic for things like out-of-bounds access. In fact, Rust's panic / Result is actually very similar technically to Java's checked / unchecked exceptions. I'd argue that this is more a difference in usage and culture, with likely influence from the fact that Java makes it easy to catch unchecked exceptions, so many prefer unchecked exceptions to avoid being forced to think about what to do in case of an error (which, of course, means that probably the wrong thing happens then). Also, Rust culture (IMHO) tends to be more strict about technical errors, probably because it is applied to more low-level problem areas. If high-level business logic is cluttered with technical error handling in every other line, it becomes near impossible to read and maintain, so other patterns are applied (e.g. atomic transactions that are undone when something fails).
- masklinn 4y ago
  > I'd argue that this is more a difference in usage and culture Usage and culture and affordances, as in: - the stdlib doesn’t signal errors by panic-ing - rust learning documents drive users towards Result and Option - the standard library provides APIs to manipulate results in ways Java is utterly missing e.g. map_err, ok, ok_or, ?, … - because results are reified values, the language allows abstracting over results (creating macros or functions around them), as well as providing utilities around them (e.g. eyre, thiserror, anyhow, …) - because results are reified values, they can be ignored until relevant, an `Iterator<Item = Result<…>>` is just an `Iterator<Item = _>` until you actually need to process the result. - panics have very little signaling power, technically your panic can carry a payload but I’ve never seen anyone go beyond a message Also rust panics are explicitly noted to not necessarily be recoverable from, so they can not be used for error signaling: whoever compiles the program can set panic=abort, and now you have no error handling at all, the program just straight dies. And even if you actually, genuinely, want to use checked exceptions in Java, it’s hell, because it does not mesh with the rest of the language. Just look at the contorsions necessary to get checked exceptions through Stream APIs. Checked exceptions are not necessarily bad in and of themselves, but Java’s implementation is so awful it has completely poisoned the field. And frankly it just… doesn’t seem like a very useful thing to put work into: you can just use exceptions-flavoured results à la Swift or Zig if you like the “look” of exceptions.

lawxls 4y ago

Just a small suggestion to consider: If the title was "Zero-Cost Exceptions and Error Interfaces in Python" I would have received it in my personal Hacker News feed in Telegram (https://github.com/lawxls/HackerNews-personalized https://github.com/lawxls/HackerNews-personalized) as I do have "Python" keyword added. And there's also email notification services people use which might have Python as a keyword. Thanks for providing further resources btw, added Sebastian Witowski presentation to "watch later" list :)

rekahrv 4y ago
Thanks a lot, that's a very helpful tip. :-) We'll adjust the title for future posts. (And I'll set up such a keyword-based notification for myself. :-) Sebastian's talk is awesome indeed. I learned a lot from it about how Python works without going too much into the implementation details.

randyrand 4y ago

As far as perf goes, a callee raising an exception is never the answer. The perf arguments are a sham imo. A callee cannot know how many times it is going to be called, only a caller, or a caller's caller, etc, can know that.

mypy_abuser 4y ago

def read(self, id_: str) -> Item | None: ...