3 ms·

> The process is "panicking", but it is still reliable enough to post a message to Slack? Even writing a log entry Well.. most likely yes. If I invoke panic, a

> The process is "panicking", but it is still reliable enough to post a message to Slack? Even writing a log entry Well.. most likely yes. If I invoke panic, all state is exactly as it was just prior to that statement. And even if not, it's not gonna "launch the missiles" if that slack post or log-write fails now, is it? > Even writing a log entry might be too much "Too much", how? Either it succeeds in which case it'll help you, or it won't which has the same result as not attempting it in the first place: no log entry..

Jare 9y ago
There are many ways to "fail to do X", most of them much more complex (and possibly damaging) than "won't do X", especially in a program whose state is now corrupted. So limit that surface area as much as possible.
tedsuo 9y ago
The most common problem is that code executing at that point can hang in unexpected ways, preventing shutdown. It's a real bummer when that happens. I've even seen it happen with logging written the wrong way, where the code attempts to flush the logs to ensure they are written... and hangs. Meanwhile, the crazy code that panicked is still running it's other goroutines - remember, you called recover! - so maybe now the webserver still has an open port and is allowing your users to access whatever strange state is left inside... gonzo things really can happen if you let a corrupted program stay on rather than shutting it down immediately. Even if nothing bad is happening, you still are out of commission for that entire period. The point is, murphy's law always comes into play. So if we're talking about production best practices, consider that "most likely fine" means "definitely not fine at scale over time". Just make sure that whatever you're doing during shutdown can't block.
- unscaled 9y ago
  Would you suggest the same course of action for any language which uses exceptions? Because there is nothing special about the stack unwinding you get with panic/recover. I'm seriously wondering if you ran into any trouble with recovering panics in production, because that would imply all Java, C#, Python, JS and Ruby server code in production which is happily catching and logging exceptions in the main request handler is constantly running into corrupted state.
  - sleepydog 9y ago
    That's not an accurate comparison. The issue is not the stack unwinding, but the root cause of the panic. In all of those other languages, exceptions are used for normal error handling in addition to "fatal" events. In Go, panics, if used at all, are generally used for fatal runtime errors (memory, segfault, etc), or in place of assert, and indicates a fatal, unrecoverable flaw in your program. At every job I've worked at it was generally discouraged to handle MemoryError exceptions in python and OutOfMemoryError exceptions in Java. It's safer to assume there is nothing you can do on such occasions, because sometimes (usually the worst possible time) there isn't.
    - lomnakkus 9y ago
      Another relevant Java example might be VirtualMachineError (superclass of OutOfMemoryError) and the wonderfully named subclass UnknownError(!).
derefr 9y ago
If you invoke panic(), sure. If the runtime invokes panic(), it might be in a pretty awful state (e.g. unable to allocate memory.)

Jare 9y ago

There are many ways to "fail to do X", most of them much more complex (and possibly damaging) than "won't do X", especially in a program whose state is now corrupted. So limit that surface area as much as possible.

tedsuo 9y ago

The most common problem is that code executing at that point can hang in unexpected ways, preventing shutdown. It's a real bummer when that happens. I've even seen it happen with logging written the wrong way, where the code attempts to flush the logs to ensure they are written... and hangs. Meanwhile, the crazy code that panicked is still running it's other goroutines - remember, you called recover! - so maybe now the webserver still has an open port and is allowing your users to access whatever strange state is left inside... gonzo things really can happen if you let a corrupted program stay on rather than shutting it down immediately. Even if nothing bad is happening, you still are out of commission for that entire period. The point is, murphy's law always comes into play. So if we're talking about production best practices, consider that "most likely fine" means "definitely not fine at scale over time". Just make sure that whatever you're doing during shutdown can't block.

unscaled 9y ago
Would you suggest the same course of action for any language which uses exceptions? Because there is nothing special about the stack unwinding you get with panic/recover. I'm seriously wondering if you ran into any trouble with recovering panics in production, because that would imply all Java, C#, Python, JS and Ruby server code in production which is happily catching and logging exceptions in the main request handler is constantly running into corrupted state.
- sleepydog 9y ago
  That's not an accurate comparison. The issue is not the stack unwinding, but the root cause of the panic. In all of those other languages, exceptions are used for normal error handling in addition to "fatal" events. In Go, panics, if used at all, are generally used for fatal runtime errors (memory, segfault, etc), or in place of assert, and indicates a fatal, unrecoverable flaw in your program. At every job I've worked at it was generally discouraged to handle MemoryError exceptions in python and OutOfMemoryError exceptions in Java. It's safer to assume there is nothing you can do on such occasions, because sometimes (usually the worst possible time) there isn't.
  - lomnakkus 9y ago
    Another relevant Java example might be VirtualMachineError (superclass of OutOfMemoryError) and the wonderfully named subclass UnknownError(!).

derefr 9y ago

If you invoke panic(), sure. If the runtime invokes panic(), it might be in a pretty awful state (e.g. unable to allocate memory.)