31 ms·

Gopher Wrangling: Effective error handling in Go

• showdeddd 3y ago
  For #3 you can also use errgroup from sync/errgroup. It's a nice recent addition to the stdlib. For #4 wrapping your errors creates pretty and logical error messages for free. It should be done in most cases.
  • benhoyt 3y ago
    It looks like sync/errgroup is a proposal, but not in the standard library (not yet, at any rate): https://github.com/golang/go/issues/57534 https://github.com/golang/go/issues/57534
    • showdeddd 3y ago
      Oh that's right. I imported it via golang.org/x/sync/errgroup
      • lenkite 3y ago
        Even the author of errgroup does not want errgroup to enter the stdlib for the reasons he mentions: There are two significant problems with the API: An errgroup.WithContext today cancels the Context when its Wait method returns, which makes it easier to avoid leaking resources but somewhat prone to bugs involving accidental reuse of the Context after the call to Wait. The need to call Wait in order to free resources makes errgroup.WithContext unfortunately still somewhat prone to leaks. If you start a bunch of goroutines and then encounter an error while setting up another one, it's easy to accidentally leak all of the goroutines started so far — along with their associated Context — by writing
• softirq 3y ago
  Always wrapping errors can be a good way to get a stack trace of the error path in the logs.
  • linux2647 3y ago
    Is that a real stack trace, or just a trace of error message wrapping? I haven’t figured out how to extract a real error message using Go stdlib
    • coffeebeqn 3y ago
      No it’s a manual “stack” you build yourself with wrap. It’ll take you to the nearest error handler to the error which is usually not that far from the real problem
      • linux2647 3y ago
        Ah yeah that’s what I figured. In some of the repos at work, I’ve noticed that some intermediate error messages are identical, which makes it hard to know which error has actually been used.
• movedx 3y ago
  One thing I did for a project of mine was define my own error type, so that I can include some specific information for the next layer up. In short, I added information about the severity of the error so that the calling function that's capturing it can decide if it can recover from it or not based on the severity, and I added a flag to determine if the result value (think "T, error") is empty, partial, or complete. I added .Empty() and .Partial() because if you're returning "string, error" from a function, for example, then "" doesn't cut it for me and instead of checking for "" in the calling function, I can instead check for err.Empty(). This doesn't seem like it's useful, but take that idea and apply it to two additional scenarios: a non-pointer to a struct{} with 10 fields (are they all empty?), and partial return values i.e. the function you called threw a warning and only partially populated the return value. Now the calling function can shift the "is empty" checks to the function that actually constructs the return value (or not.) Now I can call a function, get my custom error type back, and I can determine if there was an issue and whether or not the value is empty or partial regardless of the type (and its complexity.) This paid me back in dividends the moment I wanted to be able to return a warning and a partial result - so not workflow breaking, but also not everything the caller asked for... it's up to the caller to determine if it has what it needs to continue.
• eddythompson80 3y ago
  There is a lot that I like about Go. Error handling is not one of them. On one hand, I appreciate the simplicity of it all. Nothing special about an error, it’s just part of how you do everything else. But on the other hand, there is something clearly special about an error. It’s something 100% of go users have to deal with in almost every single function call. There is something clearly special about it. These grassroots patterns and efforts to handle multiple errors per function, or async errors etc all should really have better solutions in the language. I understand that maybe the language authors in the early days didn’t want to lock anyone into a strict paradigm for how to deal with errors. Like I’m not thrilled about Java’s approach either, but that can never change. But Go is a very popular and established language now. It’s time to fix the error handling mess. There are so many good examples out there to get inspiration from. F#, Swift and Rust have a perfect error handling mechanism.
  • lolinder 3y ago
    > F#, Swift and Rust have a perfect error handling mechanism. Rust's is good but not perfect. I often find myself missing stack traces (there are solutions but they're not easy to use), and you're still constrained to a single type of error per function, which means you see a proliferation of specialized error types that are mutually incompatible and have to be converted back and forth.
    • eddythompson80 3y ago
      Interesting. That’s fair. To be completely honest, out of the 3 I listed, I’m only familiar with putting an F# service into production. We’re a very heavy C# shop (legacy Java shop, but anything since 2015 has been in C#) and I thought I could slip in an F# project in there. And I was able to until I needed to hand it over to another team that wrote wrappers to all the F# code in C# and did all new development in C#. Swift and Rust error handling reminded me of how nice error handing felt in F#. But I only did very simple toy projects in them.
    • treyd 3y ago
      > you're still constrained to a single type of error per function This is true, but the ? operator expands into a form that does `.into()` conversions of the error variants. If there's a implementation of `From`/`Into` between the error type you're unwrapping and the error type on the function, it automatically converts. This is aided by the "thiserror" crate which provides a derive macro that can generate these automatically.
• dvt 3y ago
  For a language where coroutines are such a first-class citizen, I wish there was a more idiomatic way of returning and handling async errors in Go. I know it's all over the docs, but using a channel has always felt so "wrong." The errgroup lib tries to fix this, but it's still not as flexible as using a channel (for example, if you want to store or log all routine errors).
  • drdaeman 3y ago
    As far as I understand Go, passing back results via channels is the idiomatic approach for this. But the provided example is wrong - it is synchronous, as it awaits the computations to finish; and it is broken, because if either `refresh` call panics the caller will hang indefinitely. So it needs some extra defers and maybe a sync.WaitGroup Also, example 5 is also somewhat not good, because it uses `if err == context.DeadlineExceeded` where it should've said `errors.Is(err, context.DeadlineExceeded)` as it's a good practice to always assume that exceptions may get wrapped (#4 just mentioned that).
    • dvt 3y ago
      > As far as I understand Go, passing back results via channels is the idiomatic approach for this. It's definitely the canonical way, but communicating errors via channels feels very.. weird, for the lack of a better word (hence why I don't find it idiomatic).
• incompatible 3y ago
  "Always handle errors" sounds good until you remember that every read or write can potentially fail. My go programs are littered with unchecked fmt.Printf or Println statements.
  • BobbyJo 3y ago
    That seems like the kind of thing you'd want to wrap and panic on rather than ignore. If fmt.Print doesn't work, you should probably just kill the process.
    • comex 3y ago
      That has the downside that if the user pipes your program’s standard output to, say, `head`, then once `head` is done reading the first few lines and exits, the program will blow up and probably print a message to standard error that clutters the user’s terminal (where they were expecting to see the first few lines of output). Though, continuing to spend time generating more output that goes nowhere may not be particularly useful either, depending on what the program does. Still I think ignoring errors writing to stdout is better as a general default. If nothing else, it’s the most common behavior and is thus more likely to fit the user’s expectations.
  • preseinger 3y ago
    fmt.Printf failures are un-actionable, so there's no reason to handle them
    • tgv 3y ago
      That's what the parent comment means, I think.
      • incompatible 3y ago
        I didn't mean anything in particular, I just made an observation. I feel bad for having all those unchecked error returns. I wouldn't want to wrap them in a panic, especially if it was some kind of server. Actually, I'm reminded of certain errors I've seen along the lines of "exception raised while handing exception".
• tester457 3y ago
  Go error "handling" blocks don't seem like error handling, when it's 3+ visual polluting LOC that just return the error up the call stack, occasionally with context like tip #4 of the blogpost. I've tried to like go's verbose error handling (follow the “happy path”) but the error handling signal to noise ratio is skewed in a way that makes developing in go feel slow and boring.
  • preseinger 3y ago
    the idea that error handling "pollutes" code is a misunderstanding which go addresses the "sad path" of error handling is equally as important as the "happy path"
    • andrewjf 3y ago
      Then it would seem that it's required for the compiler to make sure you're consuming the return values (happy and sad) correctly by having the compiler enforce access to them, which go completely punts.
    • za3faran 3y ago
      How does it address it? By making it painstakingly verbose (not to mention error prone) to deal with errors? Not referring to you personally, but I've heard that sentiment several times now, and I have not seen anything to back it up (as with several other golang claims).
      • lanstin 3y ago
        with robust and potentially high volume code, the most important feature is good behavior in failure domains. disk full, do you abort or continue once the cron job frees some space; cant alloc memory, do you abort or return a static 503 page? bad contents in some file, do you exit or log the error and carry on? does a bad pyc file generate a good error message or crash python. this robustness is the famed second 90% of the project. normal go looks a cerain way when it is handling these errors.
• dpifke 3y ago
  I've mostly evolved to making err a named return parameter, and inverting the err != nil check. For example: func foo() (err error) { var x any if x, err = bar(); err == nil { err = baz(x) } if err == nil { err = bat() } if err != nil { err = fmt.Errorf("%w doing foo <additional info here>", err) } return } This feels somewhat cleaner to me, in particular by combining error handling (in this case just a simple wrap) in a single place at the end of the function.
  • marcus_holmes 3y ago
    Curious why you're using fmt.Fprintf and not fmt.Errorf? Or is that a typo? And I think you're going to have problems with this pattern if you join a team using Go in an organisation. The `if err != nil` pattern is the norm, and everyone's used to it (and the regular cadence of Go code; "do the thing, check the error, do the thing, check the error" is very readable).
    • dpifke 3y ago
      That was a typo, thanks for pointing it out. :)
  • preseinger 3y ago
    please don't do this it obfuscates the control flow, specifically the value that is actually returned early returns on errors are good, not bad edit you want func foo() error { x, err := bar() if err != nil { return fmt.Errorf("bar: %w", err) } if err := baz(x); err != nil { return fmt.Errorf("baz: %w", err) } if err := bat(); err != nil { return fmt.Errorf("bat: %w", err) } return nil }
    • meling 3y ago
      Yes, I absolutely agree with this. I think there is great value in returning early on error; think of them as guards checking that you have the values you need for the next logic step. In the original version you may have to read the whole function to understand why it failed.
• marcus_holmes 3y ago
  Surprised that the "always wrap your errors" rule isn't in there. It's been the rule in the last few Go teams I've been in.
  • AlexCoventry 3y ago
    We moved away from wrapping them recently for performance reasons.
    • za3faran 3y ago
      How much of a performance hit were they causing?
      • lanstin 3y ago
        and how many errors are they getting?
    • sethammons 3y ago
      You got to add more context here. An entire blog post would be worthy. We do some very performant code at high scale and error wrapping has never put pressure on our systems. Highly interested in what you saw.
    • preseinger 3y ago
      what programs are you writing where error wrapping represents a performance cost that's worth avoiding???
  • stephen123 3y ago
    It seems like a waste of time to me. Wrapping errors adds context. But you can usually get enough context from stack traces.
    • masklinn 3y ago
      But you don’t have stacktraces.
• bedobi 3y ago
  For the love of all that is good in the world, this is a solved problem, I don't understand why languages like Go, Kotlin, Python etc etc etc continue to insist on not having sane Option, Either, Try etc types.
  • erik_seaberg 3y ago
    Kotlin is deliberately trying to stay closer to Java and more approachable than, say, Scala. It does have sum types and a generic Result, but the builtin special cases for nullability and exceptions are a little more ergonomic (and simplify interop with JVM APIs).
    • bedobi 3y ago
      I don't understand this. Kotlin has very deliberately made many choices that don't align with Java, that's the whole point. But more and more people are realizing that Kotlins error handling is a failed experiment and are adopting Arrow instead, as they should. There's nothing demanding about Option, Either, Try etc types, they're literally just objects no different to any other, and they enable you to write functions that actually only return what they say they do, unlike with exceptions. It is exceptions that are difficult and demanding lol. And the Result type isn't very good and not even meant to be used by users of the language.
  • enriquto 3y ago
    > this is a solved problem, I don't understand why languages (...) insist on not having sane Option, Either, Try etc types. This is not a "problem" as much as a conscious philosophical stance: Errors don't actually exist, only conditions that you dislike. All the error handling you need is if/else. Everything else is unnecessary emotional baggage on some conditions that should not pollute your language. And even less so, gasp, your types (!).
  • unscaled 3y ago
    It was only in recent years that Rust has proven that monadic error-handling can be accepted in a mainstream language. At least, I hope it convinced enough people. The more generic approach to error handling, using do monads (in Haskell and Scala) require some sort of do-notation (Scala's "for comprehensions") to be convenient. And I think this is a step that most mainstream languages are still too afraid to take. I would personally be glad for mainstream and some sort of monadic comprehension to become a mainstream language feature the same way closures became, but this is far from the reality. So we are left with special-case solutions for specific problems like error-handling, iteration and nullability. Kotlin made it very easy do deal with nulls without a much ceremony (this is slightly more troublesome in Rust or Scala, for instance), while Rust chose to make error handling easier. Of course, they both repurposed the same operator ("?") for this purpose. What Kotlin does with nullability and what Rust does with error-handling are both becoming quite palatable for mainstream language developers, but it's quite late to change language which have used exceptions (like Kotlin, Java and Python) or error values (like Go) to use monads right now. Entire APIs are built on the existing (and insufficient) error handling scheme. For instance, we're using Arrow's Either on most new projects at work, but still have to deal with a lot of existing Java APIs, which are exception-based.
• za3faran 3y ago
  The provided examples highlight exactly why error handling in golang is verbose, error prone, and lacks context. Do people really not care about stack traces?
  • tail_exchange 3y ago
    Less than I thought I would. I work with a very large Go codebase, and I don't remember the last time I had problems because I needed a stack trace. Just grepping for the error message is enough to show me exactly where it happened. Still, this doesn't mean that Go does not have stack traces. It does have stack traces for panics, and you can create stack traces by wrapping errors.
    • iudqnolq 3y ago
      I frequently am sad about the kind of Rust error that doesn't have stack traces. Do you not often see something like "file not found" and need to know what file wasn't found? Or do the lowest level go error types carry more context?
      • philosopher1234 3y ago
        IIRC os.Open includes the pathname of the file it’s operating on. But still, if you don’t wrap your errors, you get useless errors which are unlocatable. Wrapping is essential in go. Thankfully the STL has good facilities for adding relevant data (like pathnames) to your wrapping
      • deleted 3y ago
        [deleted]
      • tail_exchange 3y ago
        The programmer needs to be aware that they will need to provide enough context in case of a failure. One thing I see a lot in Go examples is this pattern: body, err := readFile(fileName) if err != nil { return "", err } If the error returned by readFile is just "not found", it would indeed be very vague. This is still poor error handling, in my opinion, since a lot of the context is lost. Yes, they are "handling" the error, but only enough to stop the linter from complaining. I prefer something like this: body, err := readFile(fileName) if err != nil { return "", errors.Wrapf(err, "readFile(%s)", fileName) } This would result inan error like this: readFile(file.txt): not found This way I get all the context I need to know where the error happened and the arguments that caused the error. If the error happened not because of a function call, but, say, an invalid value, instead of this: if n < 10 { return fmt.Error("invalid argument") } Do this: if n < 10 { return fmt.Errorf("invalid argument n=%d is less than 10", n) } In languages like Java, it feels very tempting to let errors bubble up and then let the stack trace take care of explaining what went wrong, but it is often insufficient and may result in hours of debugging. I feel like Go makes it very easy to add extra context to errors, and if you foster the practice of adding context every time you return an errlr, it will be much richer than a stack trace.
• kaba0 3y ago
  Could someone explain why is Go so hyped? In my personal opinion it is just not a good language, and I think many judge it based on some false basis that it is somehow “close to the hardware” because it produces a binary. Like, the amount of time it is put next to Rust when the two have almost nothing in common.. It is very verbose, yet Java is the one that is called that, often by Gophers, which is much more concise. It has terrible expressivity, a managed language which is a perfectly fine design choice, yet seemingly every other language with a GC is somehow living in sin. And still, it doesn’t fail to show up each day on HN.
  • SmooL 3y ago
    1. It's opinionated, so there's often only one way of doings things. Largely, the "one way" is a good way, so people appreciate the forced consistency 2. It's simple. It is very easy to read and write. It is hard to shoot yourself in the foot. 3. It's powerful. They have a few core abstractions that compose well (generic io, http stuff). 4. It's fast. It runs fast because it's compiled, and it compiles fast because it's simple. Me personally: I appreciate the simplicity of it. It's a great language for working with in a team. I wish it was more functional, and had better ways to handle errors, but the simplicity of it all was a breath of fresh air using it in a working environment.
    • mirekrusin 3y ago
      You can shoot yourself in the foot with null pointers.
      • konart 3y ago
        You can find a gun to shoot yourself in the foot in any language.
        • saghm 3y ago
          > It is hard to shoot yourself in the foot. >> You can find a gun to shoot yourself in the foot in any language. Sure, but finding that one isn't particularly hard, which is what GP was responding to
• Jacobinski 3y ago
  In the last example, it is preferable to use `if errors.Is(err, context.DeadlineExceeded) {...}` instead of the given `if err == context.DeadlineExceeded {...}` since the `errors.Is()` function will recursively unwrap error chains to find the specified error. https://go.dev/blog/go1.13-errors https://go.dev/blog/go1.13-errors
• janosd 3y ago
  Go's error handling is a horrible mess: 1. It's easy to ignore returned errors without any compiler warnings. You have to rely on third party tools such as golangci-lint to report missing error handling. 2. Errors don't carry stack traces with them, you have to rely on third party libraries or custom errors to get that functionality and you will only get it for your own code, not in other libraries you are using. 3. It's unclear who should add context to error messages is it the caller or callee? Usually it gets skipped, leading to useless error messages. 4. Errors are untyped. If you want to decide based on error types, you have to use errors.Is or errors.As, which, surprise, is roughly as expensive computationally as panic-recover. (Source: I did a performance tests on this with Go 1.18) Go might as well add a simpler way to create exceptions. (I wrote a prototype library to that effect a while ago: https://github.com/APItalist/lang https://github.com/APItalist/lang ) 5. Error messages are too terse and hard to read when using the recommended semantic of "message (cause(cause(cause)))". I'd rather see stack traces, that's much more useful. 6. Most loggers are globally scoped and cannot be injected into code, leading to an all-or-nothing approach. It is not uncommon that you have 3-4 logging libraries as dependencies, which you need to configure separately (if you even can). Also, good luck securing this mess.
  • notTooFarGone 3y ago
    Calling a linter thirdparty in Go is really disingenuous. Like you install go in your favourite IDE and it's batteries included. It's part of the standard set.
    • aniforprez 3y ago
      golangci-lint does not come batteries included. It is a third party library. Saying it's "part of the standard set" is really disingenuous
      • ryapric 3y ago
        I wonder if they might be talking about `go vet`?
    • fulafel 3y ago
• skarlso 3y ago
  This feels like it has been written by someone who recently started using the language, considering that the code in many places simply doesn't compile and has syntax errors or logical errors in it. Many people coming into Go as a new language immediately start bickering about how they want their previous language features in Go rather than accept what Go has to offer and at least try to understand it. This is the equivalent of moving to another country and then refusing to integrate but being very vocal about how said country sucks. I genuinely appreciate Go's error handling because it's clean and on the nose. It's not hidden behind weird syntax/values that you have to unpack. It's right in your face all the time. When you read the code, it reads cleanly and understandably, even for a beginner. They don't have to adapt to some weird combination of failures / unpacking/choosing something different when there is an error; you immediately see that there could be an error. And regarding stack traces, wrapping errors will provide you with failure locations to the line code. You can have all sorts of nice output for errors you can later parse and identify. I get that some people go into Go because of a shift in the company and have no choice; I feel you. For me, it was a life changer. I learned to love coding again after 15 years of writing Java Beans, Spring annotations, CreateMyFriggingObjectFactorySingletonBuilderFactoryBuilders.
  • MrBuddyCasino 3y ago
    > CreateMyFriggingObjectFactorySingletonBuilderFactoryBuilders 2005 called, they want their Enterprise Java™ jokes back.
    • skarlso 3y ago
      I _WISH_ this would be 2005. Did you ever work at a bank? They are still using 1.6-1.9 maybe. I still know modern Java codebases where long descriptive class names are a must. So sadly, while I understand your sarcasm, it is not the case.
• saturn_vk 3y ago
  1. It's so easy to skip errors that I have yet to encounter such a problem across two companies now. It's weird
  • preseinger 3y ago
    yeah it is literally a non-issue in practice, and yet
• hknmtt 3y ago
  Majority of people who complain about errors in Go don't primarily work with compiled languages that produce programs that run indefinitely or for a very long periods of time. It's easy to throw exception in PHP which is interpreted on the fly and run once so any failure can be simply thrown out and ignored. With constantly running programs one has to always handle all the cases where things don't go as wanted to prevent program form crashing. If people truly hate Go's errors, just panic, it's literally no different than exceptions in other languages. You can catch them and stop or continue whatever code you want. Just STFU about errors in Go already!
• liampulles 3y ago
  I differ with the author here, I prefer to log errors as soon as they come into "my" code (e.g. from external library or network call, etc.). This is a good rule for any language, because you always ensure an error is logged once. In Go, you can add additional info from the caller to the Context to log higher level info, e.g. a trace span Id.
  • preseinger 3y ago
    an error should be handled in precisely one way - logged (and control flow continues) - returned (and control flow returns) - managed (and control flow (probably) continues) if you log an error, then you should not return it if you return an error, then you should not log it etc.
• nathants 3y ago
  gopls, staticcheck, errcheck and ineffassign are non-optional for golang dev. add them to flycheck or similar, and go is a fantastic experience. should they be part of the compiler? maybe. i’m not losing sleep over it.
• evercast 3y ago
  > Usually this isn’t necessary and its better to just return the error unwrapped. This is a terrible advice. Wrapping is extremely helpful in providing additional context for the error travelling up the call stack. Without wrapping, one typically ends up with software logging generic errors like "file not found" , which you can't act on because... you don't know where it's coming from. If you skip error wrapping, better be ready to enjoy quality time when production crashes.
• drakonka 3y ago
  I know there's lots of complaint about error handling in Go, but I always liked it. I find it straightforward, intuitive, and it forces you to be absolutely explicit if you _really_ want to ignore something.
  • simiones 3y ago
    Not always. For example, you'll see things like `file.Close()` or `defer file.Close()` very often, ignoring any error entirely.
• flippinburgers 3y ago
  Man I definitely handle errors the "wrong way" in almost all go code I have written. I take a "log immediately with filename and line number" approach. For me, it works. For teams maybe not. For large codebases with a bunch of 'I am a "programmer" (because it makes me loads of cash) individuals', it is definitely not a good idea. It requires discipline. Personally I hate stack traces.
• talideon 3y ago
  There was just so much nonsense back in the day around Go's error handling and about how it was so much more straightforward than adding exceptions to the language. In reality, the only reason why errors in Go work the way they do is that it kept the runtime simpler by offloading checking to the developer. The alternative would've been for Go to support sum types, which would've helped make error handling a lot saner, but that was dismissed because they overlapped a little with structurally-typed interfaces (Go's one really good idea). Oh, and the stupid hack that is 'iota'. And then Go eventually ended up badly re-inventing most of what exceptions do with errors.Is(), errors.As(), and fmt.Errorf("%w", err). It's such a hot mess.
  • preseinger 3y ago
    nope! go's error handling is actually good! it turns out that treating errors the same as normal values makes programs more reliable lots of people get salty about it, for sure
    • talideon 3y ago
      The _right_ way to treat them as normal values is by using sum types. So no, Go's error handling isn't at all good. 1.13 might've made them less execrable, but it didn't make it good.
      • preseinger 3y ago
        you know i looked into it and it turns out that there is no actual consensus on what "the _right_ way" to treat errors is! huh! how about that
• quicklime 3y ago
  > Make it the top layer’s responsibility and don’t log in any services or lower level code. > Make sure your logging framework is including stack traces so you can trace the error to its cause. > For example in a web app you would log the error in the http handler when returning the Internal Server status code. This is different from how I do it, am I doing anything wrong? I prefer to make it the bottom layer’s responsibility - so, the first source of the error at the boundary of my application and the library that produces the error, rather than the top level of the http handler. Go errors infamously don’t include stack traces, so how are you supposed to know where your error originated from if you log it from the top level of the http handler?
• euroderf 3y ago
  I like that errors stay in the flow of control. No "Exception"s leaping up thru multiple levels of call stack. Instead the plan is: 1) see the error, 2) "%w" the error, 3) kick it upstairs, 4) Mission Accomplished. And at some level, some piece of code will grab the bull by the horns and wrestle it to the ground. All in all, errors-as-values is a calm way to deal with unhappy code paths. A clear renunciation of longjump. (Golang system-originated panics are excepted from this gloss, but they are defined quite narrowly, and ofc catchable.)
• suralind 3y ago
  Do not wrap your errors like in the article. A better way to do it is to create your own error type where you can pass additional values alongside message. It means that you can actually handle the error and not just log it.