4 ms·

Idiomatic Errors in Clojure

97 points by harperlee 2y ago

roenxi 2y ago
> if something is expected then return either nil or some {:ok false :message "..."} value (and {:ok true :value ...} for success) Maps used in this way are uncomfortable. You end up with a function (foo x y z) and in practice you don't know how may values it is about to return. Technically one, but that one might be a map with who-knows-what in it. There is a general API problem here of how to handle operations which really require multiple communication channels to report back with. I'm not sure if there is a good way to handle it, but complex objects as return value isn't very satisfying. Although in practice I find it works great in exceptions because the map is secretly just a string that is about to be logged somewhere and discarded.
- MarkMarine 2y ago
  Maps used in this way are clojure idiom. https://softwareengineering.stackexchange.com/questions/272302/use-map-instead-of-class-to-represent-data-rich-hickey https://softwareengineering.stackexchange.com/questions/2723...
- worthless-trash 2y ago
  > but that one might be a map with who-knows-what in it. I think thatthe beauty of maps, you can take the parts you need and discard the rest (I am suddenly reminded of my old shifu).
  - IceDane 2y ago
    In a dynamically typed language, you can't even be sure you have the parts you need.
    - Capricorn2481 2y ago
      You check the keys.
    - diggan 2y ago
      If you'd say "you can't even be sure you have the parts you have", it kind of makes sense, but the way we (Clojure devs) commonly use the REPL-in-editor kind of makes that moot. But as it stands, I'm not sure I understand what you mean?
    - worthless-trash 2y ago
      I.. know which parts I need, thats a very abstract thought pattern, what do you mean ?
    - socksy 2y ago
      Just because it’s dynamically typed, doesn’t mean there aren’t types. Additionally, usage of runtime type enforcement such as malli schemas and core.spec are commonplace.
- MathMonkeyMan 2y ago
  Not idiomatic, but it looks like [core.match][1] could be used to "if error" unpack the map: (let [result (do-thing x y z)] (match [result] [{:ok false :message msg}] {:ok false :message (format "do-thing failed: %s" msg)} [{:ok true, :value value}] (continue-computation-with value))) meh [1]: https://github.com/clojure/core.match/wiki/Overview https://github.com/clojure/core.match/wiki/Overview
  - roenxi 2y ago
    What is the advantage there that justifies bringing in another dependency? You've already got a let for binding parts of result and could use cond from clojure.core instead of match. It'd be effectively identical.
    - MathMonkeyMan 2y ago
      The point of `match` is you can combine the destructuring with the conditional. It's also why Rich Hickey dislikes it.
  - dustingetz 2y ago
    i don’t think core.match has worked out in practice for many prod projects for subtle reasons, there’s a je ne sais quoi about it that seems not quite right
    - kokada 2y ago
      Do you have a list of those reasons? I find it curious that I really enjoy `match` in both Scala and Python. While it can be argued that Scala is a completely different beast than Clojure, Python is much closer (in the sense that both are dynamic typed languages).
      - socksy 2y ago
        Unfortunately (IMO), `core.match` in Clojure is a macro provided by a library you have to install, rather than a builtin function as in Scala or Python. It’s a really cool demonstration of the power of lisps, since macros basically let you edit the code at compile time, rather than at runtime, and match is maybe the most extreme example of this as it’s really compiling down the match statement into completely different code, rather than treating it like a cond statement (more info on the match algorithm here: https://github.com/clojure/core.match/wiki/Understanding-the-algorithm https://github.com/clojure/core.match/wiki/Understanding-the...). However, when using macros there’s always some trade-offs — for example, you usually can’t treat them as a first class function, passing them around (although I’m not sure if that’s true for core.match tbf). Additionally, they can be confusing to debug because the code that you’re writing doesn’t match the code that’s actually being run… stack traces can be particularly weird. Finally, by not being a builtin, it doesn’t really feel blessed as a language feature, and if it’s a choice between case, condp, or cond, I’m going to reach for one of those before core.match, simply because I don’t have to add a new library, I can be sure that I’ll understand the stack traces, and most importantly I can be sure that others will understand the code better. The fact that destructuring is built into Clojure means that you can get sort of half of the use cases for match in the first place, so it ends up being quite rare when you’ll actually reach for it. I’m not sure exactly what Dustin was referring to exactly, but those are my gripes with it. I think it’s a shame, because `match` is a more powerful construct every time I’ve encountered it in other languages, and it’s one of the few things I really miss as a built in part of Clojure.
- eduction 2y ago
  Not really, you just have to check your return values, and it is trivial to write a macro to make this more convenient (if-success, success->, etc). Checking return values for errs is a common idiom even in languages without this affordance.
whalesalad 2y ago
Great post this has cleared up a lot of things for me.
- daveliepmann 2y ago
  I appreciate the kind words and am glad it helped :)
thih9 2y ago
A bit off topic, it took me a while to figure out that the article is about “handling errors in clojure in an idiomatic way” and not “error prone clojure code that gets written so often it can be considered idiomatic”. Especially since some of these can be controversial, e.g. error maps.
- NooneAtAll3 2y ago
  having no experience in Closure, I was thinking exactly the latter
- daveliepmann 2y ago
  I considered "Idiomatic error handling in Clojure" and decided to err on the side of concision. Tbh the latter interpretation was not one that occurred to me. Curious what you would put in such an article.
lbj 2y ago
A must-read for Clojurians.I especially appreciated that he took the time to comment on the correct use of assert, which is too often overlooked and makes debugging harder than it needs to be.
- daveliepmann 2y ago
  Much appreciated. You might appreciate this follow-up on assertions: https://gist.github.com/daveliepmann/8289f0ee5b00a5f05b50379e07fceb76 https://gist.github.com/daveliepmann/8289f0ee5b00a5f05b50379...
dustingetz 2y ago
- regarding the bulk of these patterns, which are all just different encodings of error values: - the primary value prop of error values is they are concurrent, i.e. you can map over a collection with an effectful fn and end up with a collection of maybe errors (where error is encoded as nil, map, Either, whatever) - exceptions cannot do this - furthermore, clojure’s default collection operators (mapcat etc) are lazy, which means exceptions can teleport out of their natural call stack, which can be very confusing - error values defend this - the problem is that now you have a function coloring problem: most functions throw, but some functions return some error encoding - this additional structure is difficult to balance, you’re now playing type tetris without a type system. Clojure works best when you can write short, simple code and fall into the pit of success. Type tetris is pretty much not allowed, it doesn’t scale, you’ll regret it - you’ll also find yourself with a red function deep in your logic that is called by a blue function, at which point you’ll find your self doing the log-and-discard anti pattern - therefore, i agree with the first bullet: it’s a hosted language, host exceptions are idiomatic, don’t over complicate it - i do think error values can work great locally, for example (group-by (comp some ex-message) (map #(try-ok (f! %))), here i am using ex-message as a predicate. the point is you need to gather and rethrow asap to rejoin the language-native error semantics so your functions are no longer colored - i am not an authority on this, just my experience having explored this a bit, wrote a big system once using a monadic error value encoding in clojure (using the funcool either type) and was very unhappy. The minute you see >>= in a clojure codebase, it’s over. (mlet is ok locally) - one thing building Electric Clojure taught me, is that the language/runtime can encode exception semantics “however” and still expose them to the user as try/catch. Which means we can deliver the value prop of error values under the syntax of try/catch. - That means, interestingly, Electric v2’s exceptions are concurrent - which means an electric for loop can throw many exceptions at the same time, and if some of them resolve those branches can resume while the others stay parked. - For Electric v3 we have not decided if we will implement try/catch yet, because Electric userland code is essentially “pure” (given that IO is managed by the runtime and resource effects are managed by an effect system). Userland doesn’t throw, platform interop (database txns) is what throws, and we’ve found only very minor use cases for needing to catch that from Electric programs, again due to their purity. Having network failure not be your problem is really great for code complexity and abstraction!
phoe-krk 2y ago
Also note an implementation of Common Lisp condition system in Clojure that allows you to have CL-style condition handling: https://github.com/IGJoshua/farolero/ https://github.com/IGJoshua/farolero/
TacticalCoder 2y ago
They all feel kinda monadic'ish to me (and the term monad is used several times in TFA) and I kinda dig them when coupled with "enhanced" threading macros that shall short-circuit on an error but... How'd that all work with Clojure spec? I use spec and spec on functions (defn-spec) all the time in Clojure. It's my way to keep things sane in a dynamic language (and I know quite some swear by spec too). I'd now need to spec, say, all my maps so that they're basically a "Maybe" or "Either" with the right side being my actual specc'ed map and the left side being specc'ed as an error dealing thinggy? Would that be cromulent? Did anyone try mixing such idiomatic error handling in Clojure mixed with specs and does it work fine?
- daveliepmann 2y ago
  People's use of spec varies widely. My take is that error maps are for internal work. Contracts are for what passes over some sort of system boundary, so it seems feasible to just...not spec the monadic aspect? Personally I haven't worked on a project where the two overlapped, so perhaps someone with direct experience can chime in on how they navigated it.
eduction 2y ago
For failure maps, I’ve found it useful to have a :tried key, which is the parameter that in some sense “caused” the err, or a map of params if there are multiple. I also usually have an error :type key. I’ve also found it useful to distinguish between expected errs and those that should end execution more quickly. Clojure allows hierarchical keys with “derive” so I inherit these from a top level error key and set them as the :type. (Why not use exceptions - because I’ve already got exit flow and error reporting built around the maps.)
- daveliepmann 2y ago
  :tried is new to me, thanks for the tip! In the context where I've used this the most the entire map was the payload/parameter so it didn't make sense there, but I see where that could be useful. I like having a :type key - for me usually :error/kind
jimbokun 2y ago
My take away: Idiomatic Clojure error handling can be idiomatic Java error handling, idiomatic Erlang/Elixir error handling, or idiomatic Haskell error handling. If everything is idiomatic, is anything idiomatic? (This article also makes me strangely appreciative of Go's idiomatic error handling. Use multiple return values, so the error value is clearly visible in function signature and must at least be explicitly ignored by the caller. Avoids the action at a distance of exceptions, and the invisibility of errors in the dynamic approaches recommended in the article.)
- Capricorn2481 2y ago
  > so the error value is clearly visible in function signature and must at least be explicitly ignored by the caller Not exactly. There's nothing requiring you to handle errors. But Go doesn't let you have unused values in general. So in a transaction commit function, if you forget that it returns an error, Go will just let you ignore it. This is a big issue with treating errors as values. https://github.com/golang/go/issues/20803 https://github.com/golang/go/issues/20803
- daveliepmann 2y ago
  Throwing ex-info is conspicuously missing from your list of examples. >If everything is idiomatic, is anything idiomatic? Welcome to lisp! We like it here.
- ncruces 2y ago
  Java's exceptions are also visible in the signature, and harder to ignore. I actually like Go's solution better, because it's verbose, because the fact that I pay the cost of "if err != nil" every effin time, I'm more likely to consider what happens. Because if the flow is not linear with those, I'm more likely to break out a function and give it a name.
  - daveliepmann 2y ago
    The "error maps" approach from the article doesn't have Go's compiler enforcement, but shares many attributes including the "being explicit about how to handle an error" aspect you mention.
layer8 2y ago
> It’s common to see (catch Throwable) sprinkled liberally across a Clojure codebase. Just like (catch Exception), this also breaks the semantics of InterruptedException, which (to maintain its semantics) either has to be rethrown, or the catching code has to set the the current thread’s interrupt flag (Thread::interrupt).

97 points by harperlee 2y ago

> if something is expected then return either nil or some {:ok false :message "..."} value (and {:ok true :value ...} for success) Maps used in this way are uncomfortable. You end up with a function (foo x y z) and in practice you don't know how may values it is about to return. Technically one, but that one might be a map with who-knows-what in it. There is a general API problem here of how to handle operations which really require multiple communication channels to report back with. I'm not sure if there is a good way to handle it, but complex objects as return value isn't very satisfying. Although in practice I find it works great in exceptions because the map is secretly just a string that is about to be logged somewhere and discarded.

MarkMarine 2y ago
Maps used in this way are clojure idiom. https://softwareengineering.stackexchange.com/questions/272302/use-map-instead-of-class-to-represent-data-rich-hickey https://softwareengineering.stackexchange.com/questions/2723...
worthless-trash 2y ago
> but that one might be a map with who-knows-what in it. I think thatthe beauty of maps, you can take the parts you need and discard the rest (I am suddenly reminded of my old shifu).
- IceDane 2y ago
  In a dynamically typed language, you can't even be sure you have the parts you need.
  - Capricorn2481 2y ago
    You check the keys.
  - diggan 2y ago
    If you'd say "you can't even be sure you have the parts you have", it kind of makes sense, but the way we (Clojure devs) commonly use the REPL-in-editor kind of makes that moot. But as it stands, I'm not sure I understand what you mean?
  - worthless-trash 2y ago
    I.. know which parts I need, thats a very abstract thought pattern, what do you mean ?
  - socksy 2y ago
    Just because it’s dynamically typed, doesn’t mean there aren’t types. Additionally, usage of runtime type enforcement such as malli schemas and core.spec are commonplace.
MathMonkeyMan 2y ago
Not idiomatic, but it looks like [core.match][1] could be used to "if error" unpack the map: (let [result (do-thing x y z)] (match [result] [{:ok false :message msg}] {:ok false :message (format "do-thing failed: %s" msg)} [{:ok true, :value value}] (continue-computation-with value))) meh [1]: https://github.com/clojure/core.match/wiki/Overview https://github.com/clojure/core.match/wiki/Overview
- roenxi 2y ago
  What is the advantage there that justifies bringing in another dependency? You've already got a let for binding parts of result and could use cond from clojure.core instead of match. It'd be effectively identical.
  - MathMonkeyMan 2y ago
    The point of `match` is you can combine the destructuring with the conditional. It's also why Rich Hickey dislikes it.
- dustingetz 2y ago
  i don’t think core.match has worked out in practice for many prod projects for subtle reasons, there’s a je ne sais quoi about it that seems not quite right
  - kokada 2y ago
    Do you have a list of those reasons? I find it curious that I really enjoy `match` in both Scala and Python. While it can be argued that Scala is a completely different beast than Clojure, Python is much closer (in the sense that both are dynamic typed languages).
    - socksy 2y ago
      Unfortunately (IMO), `core.match` in Clojure is a macro provided by a library you have to install, rather than a builtin function as in Scala or Python. It’s a really cool demonstration of the power of lisps, since macros basically let you edit the code at compile time, rather than at runtime, and match is maybe the most extreme example of this as it’s really compiling down the match statement into completely different code, rather than treating it like a cond statement (more info on the match algorithm here: https://github.com/clojure/core.match/wiki/Understanding-the-algorithm https://github.com/clojure/core.match/wiki/Understanding-the...). However, when using macros there’s always some trade-offs — for example, you usually can’t treat them as a first class function, passing them around (although I’m not sure if that’s true for core.match tbf). Additionally, they can be confusing to debug because the code that you’re writing doesn’t match the code that’s actually being run… stack traces can be particularly weird. Finally, by not being a builtin, it doesn’t really feel blessed as a language feature, and if it’s a choice between case, condp, or cond, I’m going to reach for one of those before core.match, simply because I don’t have to add a new library, I can be sure that I’ll understand the stack traces, and most importantly I can be sure that others will understand the code better. The fact that destructuring is built into Clojure means that you can get sort of half of the use cases for match in the first place, so it ends up being quite rare when you’ll actually reach for it. I’m not sure exactly what Dustin was referring to exactly, but those are my gripes with it. I think it’s a shame, because `match` is a more powerful construct every time I’ve encountered it in other languages, and it’s one of the few things I really miss as a built in part of Clojure.
eduction 2y ago
Not really, you just have to check your return values, and it is trivial to write a macro to make this more convenient (if-success, success->, etc). Checking return values for errs is a common idiom even in languages without this affordance.

whalesalad 2y ago

Great post this has cleared up a lot of things for me.

daveliepmann 2y ago
I appreciate the kind words and am glad it helped :)

thih9 2y ago

A bit off topic, it took me a while to figure out that the article is about “handling errors in clojure in an idiomatic way” and not “error prone clojure code that gets written so often it can be considered idiomatic”. Especially since some of these can be controversial, e.g. error maps.

NooneAtAll3 2y ago
having no experience in Closure, I was thinking exactly the latter
daveliepmann 2y ago
I considered "Idiomatic error handling in Clojure" and decided to err on the side of concision. Tbh the latter interpretation was not one that occurred to me. Curious what you would put in such an article.

lbj 2y ago

A must-read for Clojurians.I especially appreciated that he took the time to comment on the correct use of assert, which is too often overlooked and makes debugging harder than it needs to be.

daveliepmann 2y ago
Much appreciated. You might appreciate this follow-up on assertions: https://gist.github.com/daveliepmann/8289f0ee5b00a5f05b50379e07fceb76 https://gist.github.com/daveliepmann/8289f0ee5b00a5f05b50379...

dustingetz 2y ago

- regarding the bulk of these patterns, which are all just different encodings of error values: - the primary value prop of error values is they are concurrent, i.e. you can map over a collection with an effectful fn and end up with a collection of maybe errors (where error is encoded as nil, map, Either, whatever) - exceptions cannot do this - furthermore, clojure’s default collection operators (mapcat etc) are lazy, which means exceptions can teleport out of their natural call stack, which can be very confusing - error values defend this - the problem is that now you have a function coloring problem: most functions throw, but some functions return some error encoding - this additional structure is difficult to balance, you’re now playing type tetris without a type system. Clojure works best when you can write short, simple code and fall into the pit of success. Type tetris is pretty much not allowed, it doesn’t scale, you’ll regret it - you’ll also find yourself with a red function deep in your logic that is called by a blue function, at which point you’ll find your self doing the log-and-discard anti pattern - therefore, i agree with the first bullet: it’s a hosted language, host exceptions are idiomatic, don’t over complicate it - i do think error values can work great locally, for example (group-by (comp some ex-message) (map #(try-ok (f! %))), here i am using ex-message as a predicate. the point is you need to gather and rethrow asap to rejoin the language-native error semantics so your functions are no longer colored - i am not an authority on this, just my experience having explored this a bit, wrote a big system once using a monadic error value encoding in clojure (using the funcool either type) and was very unhappy. The minute you see >>= in a clojure codebase, it’s over. (mlet is ok locally) - one thing building Electric Clojure taught me, is that the language/runtime can encode exception semantics “however” and still expose them to the user as try/catch. Which means we can deliver the value prop of error values under the syntax of try/catch. - That means, interestingly, Electric v2’s exceptions are concurrent - which means an electric for loop can throw many exceptions at the same time, and if some of them resolve those branches can resume while the others stay parked. - For Electric v3 we have not decided if we will implement try/catch yet, because Electric userland code is essentially “pure” (given that IO is managed by the runtime and resource effects are managed by an effect system). Userland doesn’t throw, platform interop (database txns) is what throws, and we’ve found only very minor use cases for needing to catch that from Electric programs, again due to their purity. Having network failure not be your problem is really great for code complexity and abstraction!

phoe-krk 2y ago

Also note an implementation of Common Lisp condition system in Clojure that allows you to have CL-style condition handling: https://github.com/IGJoshua/farolero/ https://github.com/IGJoshua/farolero/

TacticalCoder 2y ago

They all feel kinda monadic'ish to me (and the term monad is used several times in TFA) and I kinda dig them when coupled with "enhanced" threading macros that shall short-circuit on an error but... How'd that all work with Clojure spec? I use spec and spec on functions (defn-spec) all the time in Clojure. It's my way to keep things sane in a dynamic language (and I know quite some swear by spec too). I'd now need to spec, say, all my maps so that they're basically a "Maybe" or "Either" with the right side being my actual specc'ed map and the left side being specc'ed as an error dealing thinggy? Would that be cromulent? Did anyone try mixing such idiomatic error handling in Clojure mixed with specs and does it work fine?

daveliepmann 2y ago
People's use of spec varies widely. My take is that error maps are for internal work. Contracts are for what passes over some sort of system boundary, so it seems feasible to just...not spec the monadic aspect? Personally I haven't worked on a project where the two overlapped, so perhaps someone with direct experience can chime in on how they navigated it.

eduction 2y ago

For failure maps, I’ve found it useful to have a :tried key, which is the parameter that in some sense “caused” the err, or a map of params if there are multiple. I also usually have an error :type key. I’ve also found it useful to distinguish between expected errs and those that should end execution more quickly. Clojure allows hierarchical keys with “derive” so I inherit these from a top level error key and set them as the :type. (Why not use exceptions - because I’ve already got exit flow and error reporting built around the maps.)

daveliepmann 2y ago
:tried is new to me, thanks for the tip! In the context where I've used this the most the entire map was the payload/parameter so it didn't make sense there, but I see where that could be useful. I like having a :type key - for me usually :error/kind

jimbokun 2y ago

My take away: Idiomatic Clojure error handling can be idiomatic Java error handling, idiomatic Erlang/Elixir error handling, or idiomatic Haskell error handling. If everything is idiomatic, is anything idiomatic? (This article also makes me strangely appreciative of Go's idiomatic error handling. Use multiple return values, so the error value is clearly visible in function signature and must at least be explicitly ignored by the caller. Avoids the action at a distance of exceptions, and the invisibility of errors in the dynamic approaches recommended in the article.)

Capricorn2481 2y ago
> so the error value is clearly visible in function signature and must at least be explicitly ignored by the caller Not exactly. There's nothing requiring you to handle errors. But Go doesn't let you have unused values in general. So in a transaction commit function, if you forget that it returns an error, Go will just let you ignore it. This is a big issue with treating errors as values. https://github.com/golang/go/issues/20803 https://github.com/golang/go/issues/20803
daveliepmann 2y ago
Throwing ex-info is conspicuously missing from your list of examples. >If everything is idiomatic, is anything idiomatic? Welcome to lisp! We like it here.
ncruces 2y ago
Java's exceptions are also visible in the signature, and harder to ignore. I actually like Go's solution better, because it's verbose, because the fact that I pay the cost of "if err != nil" every effin time, I'm more likely to consider what happens. Because if the flow is not linear with those, I'm more likely to break out a function and give it a name.
- daveliepmann 2y ago
  The "error maps" approach from the article doesn't have Go's compiler enforcement, but shares many attributes including the "being explicit about how to handle an error" aspect you mention.

layer8 2y ago

> It’s common to see (catch Throwable) sprinkled liberally across a Clojure codebase. Just like (catch Exception), this also breaks the semantics of InterruptedException, which (to maintain its semantics) either has to be rethrown, or the catching code has to set the the current thread’s interrupt flag (Thread::interrupt).