4 ms·

The general principles for fault tolerance require Separation of Concerns vis. Error Encapsulation (make sure that the contagion doesn't spread), Fault Detectio

The general principles for fault tolerance require Separation of Concerns vis. Error Encapsulation (make sure that the contagion doesn't spread), Fault Detection (make sure that you know that someone is infected), and Fault Identification (you have ebola!). Error encapsulation (and this applies equally to modules, components, systems, architectures, organizations) is invariably best done at the lowest level possible, which invariably breaks #3 and #4 (fault detection and identification). what if you have different classes of exception handling mechanisms available for each of these cases? What about languages that may support exceptions, option types, tagged values, multiple return values, signals, continuations and/or whatever mix of them that exists? The problem with handling everything right away is that it lacks flexibility to handle things in what could be the optimal manner. "One true form of exception handling", to me, sounds as reductionist of an approach as "one true form of concurrency", "one true programming language paradigm", or whatever. Treating all error conditions / exceptions with the same mechanism will generally ensure that you pick similar stances on encapsulation vs. detection and identification for all error conditions / exceptions, unless you decide to be extra careful about all of that. Using multiple mechanisms will allow you to pick, case by case, which one you feel is worth breaking depending on the nature of the fault and what your specific application or system requires. I feel Go is doing a pretty bad job at this.

jb613 11y ago
"The problem with handling everything right away is that it lacks flexibility to handle things in what could be the optimal manner." You can pass the error (or another) up the stack. The language is flexible.
- mononcqc 11y ago
  "Passing up the stack" is fairly minimalistic in an environment where you could be running thousands of goroutines, some (many?) of which may share memory. If you do it that way, you seem to forego a lot of error encapsulation straight away and quickly find yourself into undefined territory.
  - jb613 11y ago
    doesn't matter whether no goroutines or thousands - you either handle errors or pass upwards. Anyhoo, just pointing out that it has the flexibility to do either.
    - parenthephobia 11y ago
      mononcqc is pointing out that those aren't the only options: other error-handling mechanisms exist. The options are not either "handle the error right now" or "make the caller handle the error". In fact, in go those aren't even the only options, since you can pass an error object up the stack using panic/recover. An alternative model could have error handlers registered somewhere and be invoked at the point the error occurs, either choosing to pass control to another part of the program, or resolve the issue and continue processing. That could, of course, be implemented in vanilla Go, but then everything has to agree to do that. Some systems are able to distinguish the point where you recover from an error from the point where you report an error: e.g. a batch processing system may recover from an error by skipping an item to be processed (and perhaps storing it in a list of failed items to be inspected later) and another "catch" handler further up may decide whether and how to display a diagnostic about the issue. Of course, this logic could be embodied as a "reporting" object which is passed down to the batch processing algorithm, but hooking into the exception handling logic means that you can use the existing error-reporting infrastructure. Common Lisp has a system of "restarts" where one can configure multiple ways for a handler to respond to errors other than simply passing it up or carrying on. For example, a batch processing system might set up restarts to allow items to be skipped, retried now, or retried after the batch finishes. The batch processor itself might have restarts for retrying the batch or rescheduling it on another node. Above the batch processing algorithm, an exception handler can look at the situation from a high level and decide what to do: e.g. it may determine that the batch has grown too big for the node and reschedule it on another one. Until the decision is made, the stack is not unwound, so if I decide to restart an item from the batch, I just jump right back into the still-running batch processing function. This design allows the mechanism for how to actually deal with an error condition to be separated from the decision as to which mechanism to use. It's very much like breaking into a debugger, except the program can debug itself. In Erlang, a program handles errors by keeling over, dead. Because an Erlang system is (meant to be) designed as a swarm of cooperating processes, an individual process can just die when something goes badly wrong and its compadres are expected to have registered an interest in knowing that this has happened. A common design pattern is to have a dedicated monitoring handling process which knows how to orchestrate things so that one broken process doesn't cause a cascading failure through the whole system. In the running example, a batch processing system might spawn a process for each batch, and a monitor process might check for batches which die and then decide whether and where to restart them. That said, Erlang still has a fairly traditional try/catch system for when that makes more sense. And errors are objects, so you can return them up the call stack as well. There are many more mechanisms for error handling which aren't simply "handle here or pass up".
- kazinator 11y ago
  > You can pass the error (or another) up the stack. (C++-like) exceptions pass the error up the stack. Directly to the right place, and ensuring that clean-up takes place along the way without tons of repeated test-and-bail code. Lisp-like are one better: the stack is searched for a binding to a function which handles the error, and then a new call is generated to that function. If the function returns, the search for the next possible one continues. Or the function can then look for a restart point somewhere the stack (the entire stack still being intact), and perform a dynamic non-local transfer (a big return) to that point. Dumb error return values make it very easy for an error to disappear without being handled. Oh, the caller has a few things to do before returning your value (things such as clean up that could be an unwind block!) and fumbles the ball somehow, and ends up returning a different value: the information is altered like in the telephone game. That's the problem: every function level between the source of the situation and the place where it is to be handled intercepts the handling effort by being involved in the control flow. When it should just be getting the heck out of the way (like cars pulling to the side when they hear an ambulance). Bubbling up error values is an excellent example of an "Anti Pattern".
  - jb613 11y ago
    Not handling an error from something you called into will lead you to be bypassed when it jumps "directly to the right place".
rakoo 11y ago
Actually the real idiom is not "handle errors where they happen" but rather "errors are values" (https://go-proverbs.github.io/ https://go-proverbs.github.io/). This means that there is no one true way to handle errors, but rather that it's up to you to build error management with the same tools you have to manage data. Maybe you want to process a list of stuff in parallel and report the first error that ever happens, the same way you'd want to use the first useful data that was processed by one of those parallel processes. Maybe you want to multiplex the error to multiple error processors, for some reason. Maybe you want to signal a watchdog but continue working. I believe bubbling up the error is a consequence of the Go community still being young and not having invested a lot in proper error management (especially since you don't always need something fancy) more than it is an idiomatic way of working.
- mononcqc 11y ago
  That's the thing though. Errors can be data in plenty of other languages. It's just that it's not the only tool you're given, and there's other things you can do without having to reimplement it all from scratch every time. Where a lot of people will say "go is opinionated", I keep feeling that Go is just not providing facilities I feel are part of what you should get out of the box (much like a standard library is one) and just shifts the burden to me to reimplement them all correctly. In doing so, you also lose the opportunity to have a well-designed approach that gets rather uniform adoption through the community.
  - spyspy 11y ago
    I know the feeling. Having moved to Go from the Python community I felt extremely scorned by the lack of a lot of features. Why should I have to write a for loop to traverse this list just to see if an element is present? What I've found is that the features that are missing are actually extremely simple to implement myself - to the point where it's comical that I ever complained about them. The things you gain, however, are much more powerful. There is an elegance to go code that can't be matched by dynamic languages. It has the readability of Python with the type safety of Java, and tooling built in that simply trumps both. I'm obviously a big fan, but it was a long and difficult journey to get there.

jb613 11y ago

"The problem with handling everything right away is that it lacks flexibility to handle things in what could be the optimal manner." You can pass the error (or another) up the stack. The language is flexible.

mononcqc 11y ago
"Passing up the stack" is fairly minimalistic in an environment where you could be running thousands of goroutines, some (many?) of which may share memory. If you do it that way, you seem to forego a lot of error encapsulation straight away and quickly find yourself into undefined territory.
- jb613 11y ago
  doesn't matter whether no goroutines or thousands - you either handle errors or pass upwards. Anyhoo, just pointing out that it has the flexibility to do either.
  - parenthephobia 11y ago
    mononcqc is pointing out that those aren't the only options: other error-handling mechanisms exist. The options are not either "handle the error right now" or "make the caller handle the error". In fact, in go those aren't even the only options, since you can pass an error object up the stack using panic/recover. An alternative model could have error handlers registered somewhere and be invoked at the point the error occurs, either choosing to pass control to another part of the program, or resolve the issue and continue processing. That could, of course, be implemented in vanilla Go, but then everything has to agree to do that. Some systems are able to distinguish the point where you recover from an error from the point where you report an error: e.g. a batch processing system may recover from an error by skipping an item to be processed (and perhaps storing it in a list of failed items to be inspected later) and another "catch" handler further up may decide whether and how to display a diagnostic about the issue. Of course, this logic could be embodied as a "reporting" object which is passed down to the batch processing algorithm, but hooking into the exception handling logic means that you can use the existing error-reporting infrastructure. Common Lisp has a system of "restarts" where one can configure multiple ways for a handler to respond to errors other than simply passing it up or carrying on. For example, a batch processing system might set up restarts to allow items to be skipped, retried now, or retried after the batch finishes. The batch processor itself might have restarts for retrying the batch or rescheduling it on another node. Above the batch processing algorithm, an exception handler can look at the situation from a high level and decide what to do: e.g. it may determine that the batch has grown too big for the node and reschedule it on another one. Until the decision is made, the stack is not unwound, so if I decide to restart an item from the batch, I just jump right back into the still-running batch processing function. This design allows the mechanism for how to actually deal with an error condition to be separated from the decision as to which mechanism to use. It's very much like breaking into a debugger, except the program can debug itself. In Erlang, a program handles errors by keeling over, dead. Because an Erlang system is (meant to be) designed as a swarm of cooperating processes, an individual process can just die when something goes badly wrong and its compadres are expected to have registered an interest in knowing that this has happened. A common design pattern is to have a dedicated monitoring handling process which knows how to orchestrate things so that one broken process doesn't cause a cascading failure through the whole system. In the running example, a batch processing system might spawn a process for each batch, and a monitor process might check for batches which die and then decide whether and where to restart them. That said, Erlang still has a fairly traditional try/catch system for when that makes more sense. And errors are objects, so you can return them up the call stack as well. There are many more mechanisms for error handling which aren't simply "handle here or pass up".
kazinator 11y ago
> You can pass the error (or another) up the stack. (C++-like) exceptions pass the error up the stack. Directly to the right place, and ensuring that clean-up takes place along the way without tons of repeated test-and-bail code. Lisp-like are one better: the stack is searched for a binding to a function which handles the error, and then a new call is generated to that function. If the function returns, the search for the next possible one continues. Or the function can then look for a restart point somewhere the stack (the entire stack still being intact), and perform a dynamic non-local transfer (a big return) to that point. Dumb error return values make it very easy for an error to disappear without being handled. Oh, the caller has a few things to do before returning your value (things such as clean up that could be an unwind block!) and fumbles the ball somehow, and ends up returning a different value: the information is altered like in the telephone game. That's the problem: every function level between the source of the situation and the place where it is to be handled intercepts the handling effort by being involved in the control flow. When it should just be getting the heck out of the way (like cars pulling to the side when they hear an ambulance). Bubbling up error values is an excellent example of an "Anti Pattern".
- jb613 11y ago
  Not handling an error from something you called into will lead you to be bypassed when it jumps "directly to the right place".

rakoo 11y ago

Actually the real idiom is not "handle errors where they happen" but rather "errors are values" (https://go-proverbs.github.io/ https://go-proverbs.github.io/). This means that there is no one true way to handle errors, but rather that it's up to you to build error management with the same tools you have to manage data. Maybe you want to process a list of stuff in parallel and report the first error that ever happens, the same way you'd want to use the first useful data that was processed by one of those parallel processes. Maybe you want to multiplex the error to multiple error processors, for some reason. Maybe you want to signal a watchdog but continue working. I believe bubbling up the error is a consequence of the Go community still being young and not having invested a lot in proper error management (especially since you don't always need something fancy) more than it is an idiomatic way of working.

mononcqc 11y ago
That's the thing though. Errors can be data in plenty of other languages. It's just that it's not the only tool you're given, and there's other things you can do without having to reimplement it all from scratch every time. Where a lot of people will say "go is opinionated", I keep feeling that Go is just not providing facilities I feel are part of what you should get out of the box (much like a standard library is one) and just shifts the burden to me to reimplement them all correctly. In doing so, you also lose the opportunity to have a well-designed approach that gets rather uniform adoption through the community.
- spyspy 11y ago
  I know the feeling. Having moved to Go from the Python community I felt extremely scorned by the lack of a lot of features. Why should I have to write a for loop to traverse this list just to see if an element is present? What I've found is that the features that are missing are actually extremely simple to implement myself - to the point where it's comical that I ever complained about them. The things you gain, however, are much more powerful. There is an elegance to go code that can't be matched by dynamic languages. It has the readability of Python with the type safety of Java, and tooling built in that simply trumps both. I'm obviously a big fan, but it was a long and difficult journey to get there.