21 ms·

Go stack traces and the errors package

145 points by bootload 10y ago

TheDong 10y ago
The fact that you need a third party non-standard package just to reliably get error stacks is ridiculous. What's worse, is this solution doesn't solve random third-party packages you depend on all that much since they'll still use the utterly useless std-library error package or their own variation which won't necessarily be compatible.
- catnaroek 10y ago
  Purely out of curiosity, what alternative solution would you propose?
  - TheDong 10y ago
    The standard library's error package supporting chaining errors would be a start. The standard library's error package recording call stacks as this one does. The standard library's error package supporting more rich information than just a string, but having both a message, a parent error (recursive data structure), a code-location (package / file / line number), and structured metadata available. Rich tooling as rust has in the form of Either + macros to make it easy to do the right thing(tm) ... And I know most people won't like this, but exceptions would also solve this. Note all the above suggestions are viable without exceptions, but exceptions are a fine solution in their own right. This really is just an area the standard library is utterly deficient in (much like logging), and the lack of macros, generics, and sum types makes elegant error handling much more difficult.
    - jimjimjim 10y ago
      I believe there are many languages that you could use that would have all of these features. That way you would have a language that would be more appropriate to your programming style without needing to add extra features to Go that possibly don't fit with Go's language style. (Note: post edited to remove snark.)
      - catnaroek 10y ago
        I don't see how `panic` and `recover` are any simpler than exceptions. Could you explain? P.D.: The downvote didn't come from me.
        jimjimjim 10y ago
        Well, they are kind of more difficult than exceptions, but I think that's the point. Handling errors by use of an exception mechanism is discouraged in idiomatic go. A big part of the style and simplicity comes from not needing to worry about what possible exceptions could come bubbling up from a library stack and under what conditions. Even though people complain about the repetition of "if err !=nil" at least you know from the function declaration if an error is could occur.
        catnaroek 10y ago
        FWIW, I don't have anything against checking errors - I agree that it's preferable to exceptions, but the way it happens to be done in Go is rather distasteful. A sum type, as suggested by TheDong, would make things clearer, by statically forcing the alternatives to be: (0) A correct result (and no error object) (1) An error object (and no correct result) Ruling out: (0) Both a correct result and an error object (1) Neither a correct result nor an error object By “ruling out”, I mean “it's a compile error”, rather than “it's a pattern, and the community won't appreciate it if you deviate from it”. Panics as normal control flow are just as unidiomatic in Rust as they are in Go, but at least: (0) There's a decent (in fact, superior) alternative for error handling. (1) You don't risk leaking resources. RAII has that covered.
        fauigerzigerk 10y ago
        I like the conceptual clarity of sum types, but in some cases they may not be flexible enough (same as exceptions). Consider the case of io.Reader: https://golang.org/pkg/io/#Reader https://golang.org/pkg/io/#Reader Read always returns the number of bytes read, and if an error occurs after some bytes have been read (or at EOF) it returns that error as well. It does feel a little dirty, but it's also efficient and pragmatic.
    - catnaroek 10y ago
      Although sum types make it easier to handle errors (or, more generally, multiple classes of results), Go probably won't adopt them anytime soon, if ever. Even then, not supporting more informative and structured error objects in the standard library is indeed very difficult to defend.
- 4ad 10y ago
  You don't need Dave's package to get stack traces. All you need is the runtime package. How do you think Dave's package works? There is no error package, perhaps you mean the errors package. But it doesn't matter how you create your error in Go as long as you assert for behavior, because errors are interfaces. That is the point really. If you import a package that uses Dave's error package, in general, you don't need to import Dave's package. That is the point of interfaces, to decouple things.
  - nothrabannosir 10y ago
    There is no error package, perhaps you mean the errors package. Of course (s)he does. Why even point that out...?
    - xkarga00 10y ago
      Why not? It's cool to point things out.
  - twic 10y ago
    I suspect you've misunderstood TheDong's complaints. Firstly, "to reliably get error stacks" doesn't mean "to get stack traces"; it means "to get stack traces attached to errors". You do indeed need Dave's package, or some equivalent, for that; it's not something the standard way of creating errors gives you. Secondly, "this solution doesn't solve random third-party packages you depend on all that much since they'll still use the utterly useless std-library error package" doesn't refer to the situation where "you import a package that uses Dave's error package", but the converse - if you import a package that uses the standard way of creating errors, you won't get stack traces on those errors. That's true, and it's why Dave's package provides the Wrap function as a partial mitigation.
    - tptacek 10y ago
      Go errors are not exceptions. It would not make sense to attach a stack trace to every one of them. If you want a stack trace, generate one when you return the error. It's hello-world simple.
- fauigerzigerk 10y ago
  What bothers me a lot more is that Go has somehow managed to get a trivial but crucial thing totally wrong: Some errors can only be identified by value and others can only be identified by type. There is no general and reliable way to handle specific types of errors. And there is no way to add instance specific information to errors in a later version of a package, because that would require using an error type instead of (or in addition to) the existing well known error value, which breaks callers.
  - themartorana 10y ago
    That's not true. All errors adhere to the error interface [0] and can be parameterized as such. It's not significant different to subclassing a stdlib Error class in another language and using reflection or down-casting to get its sub-type. So almost all packages either return a base error, or a custom error type that adheres to the error interface. [0] https://golang.org/pkg/builtin/#error https://golang.org/pkg/builtin/#error
    - nemothekid 10y ago
      I think you missed his point. >Some errors can only be identified by value and others can only be identified by type Let's say I'm reading a csv file, and I need to check whether I'm done reading the file - I'd do something like so: if _, err := csvReader.Read(); err == io.ErrEOF { quit() } However (I can't think of an example off the top of my head), there are many library that make you check the error type to find out what went wrong. if c_err, ok := err.(LibError); ok { c_err.Code } I commonly see this version in libraries that need to return an error with a non-static message (such as the `hostname goolge.com could not be reached`).
  - rakoo 10y ago
    > And there is no way to add instance specific information to errors in a later version of a package, because that would require using an error type instead of (or in addition to) the existing well known error value, which breaks callers. Yes you can. Errors are interfaces, not values. As long as you define an Errror() method your struct behaves as an error, but it can have anything a struct has, such as specific information. You should read Dave's previous presentation (http://dave.cheney.net/paste/gocon-spring-2016.pdf http://dave.cheney.net/paste/gocon-spring-2016.pdf) to understand a bit more about what's already possible today and why Dave's package makes it easier.
- tptacek 10y ago
  You've misunderstood what this library does. You do not need a third-party package to reliably get error stacks. You can get stack traces from any point in a program --- for errors, or for performance monitoring, or anything else --- any time you want in Go, which is probably the reason they're not attached to errors by default. Go errors are not exceptions.
  - fidget 10y ago
    A stack trace is not an error stack. The GP was lamenting the fact that a 3rd party package is required to create error stacks, not stack traces.
    - nialo 10y ago
      Out of curiosity: what's the difference between an error stack and a stack trace?
      - tptacek 10y ago
        I'm unclear on whether they mean "errors with stack traces attached" (trivially available) or "errors that are stacks of errors" (requires small third party library". If the latter: fair enough, but it's not as if that's a common feature of other mainstream languages.
buro9 10y ago
I've been using my own errors package for a while to address a different issue with errors when building RESTful APIs or web applications... when the error occurs, the code that touches it first best knows the HTTP status code to ultimately return. My package in it's entirety is: package errors const ErrUnknown = 0 type RESTError struct { HTTPStatus int `json:"-"` Code int `json:"code,omitempty"` Message string `json:"error"` Err error `json:"-"` } func (e *RESTError) Error() string { return e.Message } func New(status int, code int, message string) *RESTError { return &RESTError{HTTPStatus: status, Code: code, Message: message} } Doc comments removed just to show the code and nothing else. When those errors finally get back to my web handlers, a standard renderError wraps all errors: func renderError(w http.ResponseWriter, err error) int { if e, ok := err.(*errors.RESTError); ok { return render.JSON(w, e.HTTPStatus, e) } e = errors.New(http.StatusInternalServerError, errors.ErrUnknown, err.Error()) return render.JSON(w, e.HTTPStatus, e) } I handle all errors as soon as possible and then immediately assign the HTTP status code that should be returned, the error code to enable an external system to look up the error (without string parsing), a sanitised error message (to allow devs to get an idea without looking up the code) and I do put the original error in the Err so that my log files can contain the raw underlying error. It effectively acts as a map between internal errors, and external communication of errors, and allows the place where the error occurred to say "this HTTP status". And because the Err is never stringified I am comforted that internal sensitive data (this is a billing system and unsanitised data should never be leaked even via error messages) does not get leaked. That's my goal, to handle the error in the way that allows it to best communicate to: 1. HTTP clients via the status 2. Developers via an error code and sanitised message (which they can use to communicate to users) 3. SREs via detail in the secure logs I found the native errors nice, but not the best for dealing with my 3 audiences.
- oolongCat 10y ago
  Hah, this is very similar to the one I use within my apps. type AppError struct { Code int Message string InternalMessage string Time time.Time HttpCode int } All my errors are handled in a http handler function. Before that these errors are produced in any functions called by my handler functions. I concatenate in to the InternalMessage var any error message I want to log, basically anything I want to show an admin. The message tag will have anything I want to show a user. Normally separated by a semicolon. Because of this I can give good errors to my users, and I have enough information when something goes wrong. The only negative might be that, the internal error logs are a bit verbose.
astrobe_ 10y ago
There are two types of errors: the ones you log, and the ones you report to the user. The difference is that the latter has to be translated to be useful (even when your user understands english).
- idank 10y ago
  What about the ones you retry?
  - astrobe_ 10y ago
    This part was covered by Cheney already, isn't it?
justme24 10y ago
http://webcache.googleusercontent.com/search?q=cache%3Arobotwealth.com%2Fmachine-learning-financial-prediction-david-aronson%2F&oq=cache%3Arobotwealth.com%2Fmachine-learning-financial-prediction-david-aronson%2F&aqs=chrome..69i57j69i58.1735j0j4&sourceid=chrome&ie=UTF-8 http://webcache.googleusercontent.com/search?q=cache%3Arobot...
justme24 10y ago
Link below from google web cache since website is unavailable
nickcw 10y ago
Having been writing Go for 4 years now, I can say that Dave Cheney's package is exactly what I've been looking for. The thing that really sells it to me is the being able to wrap the error when necessary with errors.Wrap and find the underlying cause with errors.Cause. I've yet to experiment with the new formatting `%+v` but I can see that coming in useful too. There have been earlier attempts at something similar (eg https://github.com/juju/errors https://github.com/juju/errors) but none with the same clarity of thought. So thanks for a great package Dave and for taking the time to whittle it down into the simplest, most elegant thing.
- themartorana 10y ago
  We've been doing this via an in-house errors package for a while. It does other things, like log request ID with the error for tracing and whatnot, and uses a bit of code adopted from panicparse [0] to print pretty, easy-on-the-eyes, quickly-parsable stack traces. Honestly I don't know why stack traces aren't formatted for quick parsing. Many thanks to maruel [1] for the inspiration. [0] https://github.com/maruel/panicparse https://github.com/maruel/panicparse [1] https://github.com/maruel https://github.com/maruel
aboodman 10y ago
I don't follow the logic of avoiding error types. If I'm calling `foo.Bar()` directly, I already have a direct dependency on package `foo`, so what difference does it make to have another one to get `foo.BarError`? If I'm calling `foo.Bar()` indirectly, then it's true that checking `foo.ErrorType` introduces a novel dependency upon `foo`, but it's equally true that: a) my direct dependency is frequently going to check and wrap that error in some other type anyway, and b) I can still use the interface trick that Dave describes to avoid a dependency on `foo`. I don't see why forcing callers to declare helper interfaces every single time they want to inspect an error is preferable to them only having to do that as a rare workaround.

145 points by bootload 10y ago

The fact that you need a third party non-standard package just to reliably get error stacks is ridiculous. What's worse, is this solution doesn't solve random third-party packages you depend on all that much since they'll still use the utterly useless std-library error package or their own variation which won't necessarily be compatible.

catnaroek 10y ago
Purely out of curiosity, what alternative solution would you propose?
- TheDong 10y ago
  The standard library's error package supporting chaining errors would be a start. The standard library's error package recording call stacks as this one does. The standard library's error package supporting more rich information than just a string, but having both a message, a parent error (recursive data structure), a code-location (package / file / line number), and structured metadata available. Rich tooling as rust has in the form of Either + macros to make it easy to do the right thing(tm) ... And I know most people won't like this, but exceptions would also solve this. Note all the above suggestions are viable without exceptions, but exceptions are a fine solution in their own right. This really is just an area the standard library is utterly deficient in (much like logging), and the lack of macros, generics, and sum types makes elegant error handling much more difficult.
  - jimjimjim 10y ago
    I believe there are many languages that you could use that would have all of these features. That way you would have a language that would be more appropriate to your programming style without needing to add extra features to Go that possibly don't fit with Go's language style. (Note: post edited to remove snark.)
    - catnaroek 10y ago
      I don't see how `panic` and `recover` are any simpler than exceptions. Could you explain? P.D.: The downvote didn't come from me.
      - jimjimjim 10y ago
        Well, they are kind of more difficult than exceptions, but I think that's the point. Handling errors by use of an exception mechanism is discouraged in idiomatic go. A big part of the style and simplicity comes from not needing to worry about what possible exceptions could come bubbling up from a library stack and under what conditions. Even though people complain about the repetition of "if err !=nil" at least you know from the function declaration if an error is could occur.
        catnaroek 10y ago
        FWIW, I don't have anything against checking errors - I agree that it's preferable to exceptions, but the way it happens to be done in Go is rather distasteful. A sum type, as suggested by TheDong, would make things clearer, by statically forcing the alternatives to be: (0) A correct result (and no error object) (1) An error object (and no correct result) Ruling out: (0) Both a correct result and an error object (1) Neither a correct result nor an error object By “ruling out”, I mean “it's a compile error”, rather than “it's a pattern, and the community won't appreciate it if you deviate from it”. Panics as normal control flow are just as unidiomatic in Rust as they are in Go, but at least: (0) There's a decent (in fact, superior) alternative for error handling. (1) You don't risk leaking resources. RAII has that covered.
        fauigerzigerk 10y ago
        I like the conceptual clarity of sum types, but in some cases they may not be flexible enough (same as exceptions). Consider the case of io.Reader: https://golang.org/pkg/io/#Reader https://golang.org/pkg/io/#Reader Read always returns the number of bytes read, and if an error occurs after some bytes have been read (or at EOF) it returns that error as well. It does feel a little dirty, but it's also efficient and pragmatic.
  - catnaroek 10y ago
    Although sum types make it easier to handle errors (or, more generally, multiple classes of results), Go probably won't adopt them anytime soon, if ever. Even then, not supporting more informative and structured error objects in the standard library is indeed very difficult to defend.
4ad 10y ago
You don't need Dave's package to get stack traces. All you need is the runtime package. How do you think Dave's package works? There is no error package, perhaps you mean the errors package. But it doesn't matter how you create your error in Go as long as you assert for behavior, because errors are interfaces. That is the point really. If you import a package that uses Dave's error package, in general, you don't need to import Dave's package. That is the point of interfaces, to decouple things.
- nothrabannosir 10y ago
  There is no error package, perhaps you mean the errors package. Of course (s)he does. Why even point that out...?
  - xkarga00 10y ago
    Why not? It's cool to point things out.
- twic 10y ago
  I suspect you've misunderstood TheDong's complaints. Firstly, "to reliably get error stacks" doesn't mean "to get stack traces"; it means "to get stack traces attached to errors". You do indeed need Dave's package, or some equivalent, for that; it's not something the standard way of creating errors gives you. Secondly, "this solution doesn't solve random third-party packages you depend on all that much since they'll still use the utterly useless std-library error package" doesn't refer to the situation where "you import a package that uses Dave's error package", but the converse - if you import a package that uses the standard way of creating errors, you won't get stack traces on those errors. That's true, and it's why Dave's package provides the Wrap function as a partial mitigation.
  - tptacek 10y ago
    Go errors are not exceptions. It would not make sense to attach a stack trace to every one of them. If you want a stack trace, generate one when you return the error. It's hello-world simple.
fauigerzigerk 10y ago
What bothers me a lot more is that Go has somehow managed to get a trivial but crucial thing totally wrong: Some errors can only be identified by value and others can only be identified by type. There is no general and reliable way to handle specific types of errors. And there is no way to add instance specific information to errors in a later version of a package, because that would require using an error type instead of (or in addition to) the existing well known error value, which breaks callers.
- themartorana 10y ago
  That's not true. All errors adhere to the error interface [0] and can be parameterized as such. It's not significant different to subclassing a stdlib Error class in another language and using reflection or down-casting to get its sub-type. So almost all packages either return a base error, or a custom error type that adheres to the error interface. [0] https://golang.org/pkg/builtin/#error https://golang.org/pkg/builtin/#error
  - nemothekid 10y ago
    I think you missed his point. >Some errors can only be identified by value and others can only be identified by type Let's say I'm reading a csv file, and I need to check whether I'm done reading the file - I'd do something like so: if _, err := csvReader.Read(); err == io.ErrEOF { quit() } However (I can't think of an example off the top of my head), there are many library that make you check the error type to find out what went wrong. if c_err, ok := err.(LibError); ok { c_err.Code } I commonly see this version in libraries that need to return an error with a non-static message (such as the `hostname goolge.com could not be reached`).
- rakoo 10y ago
  > And there is no way to add instance specific information to errors in a later version of a package, because that would require using an error type instead of (or in addition to) the existing well known error value, which breaks callers. Yes you can. Errors are interfaces, not values. As long as you define an Errror() method your struct behaves as an error, but it can have anything a struct has, such as specific information. You should read Dave's previous presentation (http://dave.cheney.net/paste/gocon-spring-2016.pdf http://dave.cheney.net/paste/gocon-spring-2016.pdf) to understand a bit more about what's already possible today and why Dave's package makes it easier.
tptacek 10y ago
You've misunderstood what this library does. You do not need a third-party package to reliably get error stacks. You can get stack traces from any point in a program --- for errors, or for performance monitoring, or anything else --- any time you want in Go, which is probably the reason they're not attached to errors by default. Go errors are not exceptions.
- fidget 10y ago
  A stack trace is not an error stack. The GP was lamenting the fact that a 3rd party package is required to create error stacks, not stack traces.
  - nialo 10y ago
    Out of curiosity: what's the difference between an error stack and a stack trace?
    - tptacek 10y ago
      I'm unclear on whether they mean "errors with stack traces attached" (trivially available) or "errors that are stacks of errors" (requires small third party library". If the latter: fair enough, but it's not as if that's a common feature of other mainstream languages.

buro9 10y ago

I've been using my own errors package for a while to address a different issue with errors when building RESTful APIs or web applications... when the error occurs, the code that touches it first best knows the HTTP status code to ultimately return. My package in it's entirety is: package errors const ErrUnknown = 0 type RESTError struct { HTTPStatus int `json:"-"` Code int `json:"code,omitempty"` Message string `json:"error"` Err error `json:"-"` } func (e *RESTError) Error() string { return e.Message } func New(status int, code int, message string) *RESTError { return &RESTError{HTTPStatus: status, Code: code, Message: message} } Doc comments removed just to show the code and nothing else. When those errors finally get back to my web handlers, a standard renderError wraps all errors: func renderError(w http.ResponseWriter, err error) int { if e, ok := err.(*errors.RESTError); ok { return render.JSON(w, e.HTTPStatus, e) } e = errors.New(http.StatusInternalServerError, errors.ErrUnknown, err.Error()) return render.JSON(w, e.HTTPStatus, e) } I handle all errors as soon as possible and then immediately assign the HTTP status code that should be returned, the error code to enable an external system to look up the error (without string parsing), a sanitised error message (to allow devs to get an idea without looking up the code) and I do put the original error in the Err so that my log files can contain the raw underlying error. It effectively acts as a map between internal errors, and external communication of errors, and allows the place where the error occurred to say "this HTTP status". And because the Err is never stringified I am comforted that internal sensitive data (this is a billing system and unsanitised data should never be leaked even via error messages) does not get leaked. That's my goal, to handle the error in the way that allows it to best communicate to: 1. HTTP clients via the status 2. Developers via an error code and sanitised message (which they can use to communicate to users) 3. SREs via detail in the secure logs I found the native errors nice, but not the best for dealing with my 3 audiences.

oolongCat 10y ago
Hah, this is very similar to the one I use within my apps. type AppError struct { Code int Message string InternalMessage string Time time.Time HttpCode int } All my errors are handled in a http handler function. Before that these errors are produced in any functions called by my handler functions. I concatenate in to the InternalMessage var any error message I want to log, basically anything I want to show an admin. The message tag will have anything I want to show a user. Normally separated by a semicolon. Because of this I can give good errors to my users, and I have enough information when something goes wrong. The only negative might be that, the internal error logs are a bit verbose.

astrobe_ 10y ago

There are two types of errors: the ones you log, and the ones you report to the user. The difference is that the latter has to be translated to be useful (even when your user understands english).

idank 10y ago
What about the ones you retry?
- astrobe_ 10y ago
  This part was covered by Cheney already, isn't it?

justme24 10y ago

http://webcache.googleusercontent.com/search?q=cache%3Arobotwealth.com%2Fmachine-learning-financial-prediction-david-aronson%2F&oq=cache%3Arobotwealth.com%2Fmachine-learning-financial-prediction-david-aronson%2F&aqs=chrome..69i57j69i58.1735j0j4&sourceid=chrome&ie=UTF-8 http://webcache.googleusercontent.com/search?q=cache%3Arobot...

justme24 10y ago

Link below from google web cache since website is unavailable

nickcw 10y ago

Having been writing Go for 4 years now, I can say that Dave Cheney's package is exactly what I've been looking for. The thing that really sells it to me is the being able to wrap the error when necessary with errors.Wrap and find the underlying cause with errors.Cause. I've yet to experiment with the new formatting `%+v` but I can see that coming in useful too. There have been earlier attempts at something similar (eg https://github.com/juju/errors https://github.com/juju/errors) but none with the same clarity of thought. So thanks for a great package Dave and for taking the time to whittle it down into the simplest, most elegant thing.

themartorana 10y ago
We've been doing this via an in-house errors package for a while. It does other things, like log request ID with the error for tracing and whatnot, and uses a bit of code adopted from panicparse [0] to print pretty, easy-on-the-eyes, quickly-parsable stack traces. Honestly I don't know why stack traces aren't formatted for quick parsing. Many thanks to maruel [1] for the inspiration. [0] https://github.com/maruel/panicparse https://github.com/maruel/panicparse [1] https://github.com/maruel https://github.com/maruel

aboodman 10y ago

I don't follow the logic of avoiding error types. If I'm calling `foo.Bar()` directly, I already have a direct dependency on package `foo`, so what difference does it make to have another one to get `foo.BarError`? If I'm calling `foo.Bar()` indirectly, then it's true that checking `foo.ErrorType` introduces a novel dependency upon `foo`, but it's equally true that: a) my direct dependency is frequently going to check and wrap that error in some other type anyway, and b) I can still use the interface trick that Dave describes to avoid a dependency on `foo`. I don't see why forcing callers to declare helper interfaces every single time they want to inspect an error is preferable to them only having to do that as a rare workaround.