4 ms·

What works even better is using dataflow instead of call/return, as the problem largely goes away by itself. With call/return, you have to return something, so

What works even better is using dataflow instead of call/return, as the problem largely goes away by itself. With call/return, you have to return something, so if you have nothing to return because of an error, you have to return that, or both the error and the normal return value (Go). This pollutes the happy path. With this polymorphic container through which you thread the remainder of the processing, you make the problem a little nicer, but it is still there. With dataflow, you simply don't send anything to the next filter stage, so your happy path is completely unaffected. You then send the error to some kind of standard error output, which can often be centralised for your application. Sounds to good to be true, but used it in Wunderlist, for example, and it worked like a charm. Oh, and the same technique that works for when you don't have a value (errors) works the same when you don't have a value yet (async).

rco8786 3y ago
Is this fundamentally different than returning the data or throwing an exception to be caught elsewhere?
- golergka 3y ago
  Type system doesn't force you to handle exceptions, they're spooky action at a distance.
  - naasking 3y ago
    Some do, like checked exceptions.
  - Karellen 3y ago
    Alternatively, they're just reducing boilerplate. Yes, anything might return an error. Do we need to really write `or error` or whatever everywhere that all we're going to do is pass the error immediately back up the stack? One way to improve this would be to have some kind of way of declaring the atypical case where you have a function that can never throw an exception... which is a feature of some exception-based programming languages.
    - rco8786 3y ago
      > Do we need to really write `or error` or whatever everywhere that all we're going to do is pass the error immediately back up the stack? Right, exactly. Like in Go: if (err != nil) return err You find this littered all over every Go codebase. And somehow people seem to not realize that they’re doing the exact same thing as throwing an exception (unwinding the stack until something handles the error), just manually and painfully.
    - kamray23 3y ago
      It can be important and reasonable. You might have things which never give errors. You might have things which sometimes do, and you attempt to use them in places which you don't believe to give errors. Here, the great thing about error functors is that the type system comes out and warns you very, very explicitly. Hell, we add that same boilerplate back in a lot of languages with things like "noexcept" and "throws", with the compiler's type checker doing the exact same thing and forcing error handling in a "noexcept" when it calls something which throws. You get an isomorphic system, it works the exact same way, it's effectively an error functor as far as the type checking is concerned. That being said, this article sells the whole concept of functors incredibly short. You use them to represent certain concepts, such as in this example where the concept being represented is "value, but not everywhere in the domain" or "value or a different type of value". On top of that, these are applicative functors, meaning that they also represent a way of combining two values in a specific manner, and they are monads, meaning that they represent a kind of composition. That means that these same concepts of railway tracks suddenly allow for a lot larger set of tools: you can represent as a functor the concept of having multiple values, you can consider lists to be applicative functors with the combination of two lists being an operation such as concatenation, zipping of sequences, or even cartesian products. You can even go as far as to consider lists monads, in which case their compositional behaviour could be one of non-determinism: each function takes in a single value and evaluates to multiple values – the composition would be running the function over every value in the list (over all possible states), then concatenating the lists that come out (creating a new list of possible states). Treating errors and non-determinism as special cases of a similar behaviour allows for extending many of the methods we use in error handling to this kind of application as well. Not that these two are the only uses either. It's a much larger toolbox, though going overboard with it will result in unreadable hellish code.
  - chriswarbo 3y ago
    Indeed, exception handlers are essentially the `COME FROM` madness from INTERCAL ;)
- mpweiher 3y ago
  Define "fundamentally". In the end it all comes down to NAND gates... ;-) But yes, it is very, very different. With an exception, which does achieve a similar effect of stopping further execution right there and then, you need to throw the exception and you need to check it. With dataflow, all you have to do is nothing. And exceptions are quite a tricky and heavyweight mechanism. What about the intermediate code? Does it need to run despite the exception? Will it just swallow the exception instead of passing it on? You can just pass the final error handler in and set it on the filter and there is no trickiness. And of course without the exception, there is a lot of code that needs to do error handling despite not really being involved. This becomes really, really noticeable when your async code is handled using callbacks. You just get two sets of callback everywhere. Just having the code be synchronous would be better, but with dataflow the whole issue just evaporated.
  - gpderetta 3y ago
    In the end it boils down to continuations: your function either either takes a single return continuation that takes result|error variant or it takes two (or more) continuations: one for the happy path, and one for the error path. The former maps well to result objects, the latter is maps better to non local (checked) exceptions, but it easy to see how to transform from one to the other.
    - mpweiher 3y ago
      > In the end it boils down to continuations No it does not. > your function Only if you deal exclusively with procedures/functions. With dataflow, that's not the case. You have two filters, and if the first filter detects an error, it just does nothing (i.e it does not pass the value to the second filter). Done. No need for exceptions or continuations, or weird return types.
kamray23 3y ago
That's got its downsides. Majorly, I feel like it's restrictive. Exception handling, especially asynchronous exception handling, can really disallow recovery from errors. Sure, when you have a straight data stream and you simply want to stop processing at an error, there's no data for the next stage and it never runs. But that greatly limits recovery from errors. If your reaction to sqrt(-1) is to "crash", ending the business part of the program and hopping straight to printing or logging or whatever for the error, it can be very difficult to, say, enlarge the domain. You might have the need to simply give some use-case specific value for negative numbers, and you can pretty simply do that in code, but it'll result in some coupling you don't want as now you have to test for being outside that domain. For sqrt the domain is simple to define, but for a lot of real-world logic, it isn't nearly that simple. You can, in some languages, trap the error. Dataflow machines often are not designed for that though, especially when dealing with copious asynchronicity. That's where explicit error types, such as Maybe come in. You get to write your happy path as if errors didn't exist, but function composition uses a different set of operators than you usually would. In cases where you do want to recover from an error, you handle both paths then and there, as part of the happy path, and possibly don't even allow for an error beyond that. Most importantly, to get any value which is not an abstract concept of "maybe value" out, you need to handle the existence of both paths gracefully. That can be very nice, and very useful. That being said, now that you're using explicit error types, you can escape the "happy path"/"error" dichotomy. No longer is there necessarily just a "no value". There can be a "no value, but". Or a "value, but". You can have several errors stack up in a chain of things which could be done in parallel and then collated as a result. You can even entirely give up on the concept of errors, since that's only a very special case. You can use it to encode non-determinism: each unit takes in a single value and processes it to several possible and different values. Combining them with the monadic bind operator, each unit outputting a list of values has those values concatenated at the end to the list of all outputs, then the next function runs for each of those values the set of outputs of them are joined for the next stage and so on. This can be very, very useful for things like traversing graphs. You can, as Haskell programmers often do, use it to haul around a bit of "state" in a technically pure manner (purity is in certain cases very desirable). Perhaps the most infamous use of applicatives and monads in programming is the Haskell IO Monad, which encodes no real paths at all. IO simply is a virus which latches on to everything you do with it, and getting out of IO requires touching the outside world in an impure manner, which in Haskell can "only" be done while evaluating the expression called "main", meaning that the expression called "main" becomes your only point of contact to the outside world and the only way to "unwrap" IO values. Once again, that is for (obsessive) purity reasons. Alternative applicatives even allow for things as simple as an alternative applicative functor based on zipping instead of nondeterminism. It's a surprisingly varied technique, going far beyond simple railroads, and offers a neat way to write "only" the happy path while staying pure (allowing simple equational static analysis and unit testing without the need for mocks to re-establish purity). It also offers you many other functorial tools which are all linked by a specific composition behaviour. Further, other applicatives provide extra tools when you just need a functor which represents a specific way to combine two values. Using alternative methods, you often run out of ways to extend existing code, create something completely unreadable yet somehow isomorphic, or you create something readable and extensible, but due to the lack of purity inherent in some exceptional business logic, it becomes very hard to test without extensive mocking. Toeing the line between purity, readability and extensibility can be done in many ways, but functors and especially monads are among the S-tier when it comes to it. That being said, Haskell can get a bit goofy with it, by no means is purity an absolute value to be always chased. I kind of wish more """mainstream""" languages contained better monadic toolboxes for those times when you see an issue and you know you could solve it better than what you have to otherwise do if you only had the tools for it.
jiehong 3y ago
Would you have a code example to help clarify this in my mind?

rco8786 3y ago

Is this fundamentally different than returning the data or throwing an exception to be caught elsewhere?

golergka 3y ago
Type system doesn't force you to handle exceptions, they're spooky action at a distance.
- naasking 3y ago
  Some do, like checked exceptions.
- Karellen 3y ago
  Alternatively, they're just reducing boilerplate. Yes, anything might return an error. Do we need to really write `or error` or whatever everywhere that all we're going to do is pass the error immediately back up the stack? One way to improve this would be to have some kind of way of declaring the atypical case where you have a function that can never throw an exception... which is a feature of some exception-based programming languages.
  - rco8786 3y ago
    > Do we need to really write `or error` or whatever everywhere that all we're going to do is pass the error immediately back up the stack? Right, exactly. Like in Go: if (err != nil) return err You find this littered all over every Go codebase. And somehow people seem to not realize that they’re doing the exact same thing as throwing an exception (unwinding the stack until something handles the error), just manually and painfully.
  - kamray23 3y ago
    It can be important and reasonable. You might have things which never give errors. You might have things which sometimes do, and you attempt to use them in places which you don't believe to give errors. Here, the great thing about error functors is that the type system comes out and warns you very, very explicitly. Hell, we add that same boilerplate back in a lot of languages with things like "noexcept" and "throws", with the compiler's type checker doing the exact same thing and forcing error handling in a "noexcept" when it calls something which throws. You get an isomorphic system, it works the exact same way, it's effectively an error functor as far as the type checking is concerned. That being said, this article sells the whole concept of functors incredibly short. You use them to represent certain concepts, such as in this example where the concept being represented is "value, but not everywhere in the domain" or "value or a different type of value". On top of that, these are applicative functors, meaning that they also represent a way of combining two values in a specific manner, and they are monads, meaning that they represent a kind of composition. That means that these same concepts of railway tracks suddenly allow for a lot larger set of tools: you can represent as a functor the concept of having multiple values, you can consider lists to be applicative functors with the combination of two lists being an operation such as concatenation, zipping of sequences, or even cartesian products. You can even go as far as to consider lists monads, in which case their compositional behaviour could be one of non-determinism: each function takes in a single value and evaluates to multiple values – the composition would be running the function over every value in the list (over all possible states), then concatenating the lists that come out (creating a new list of possible states). Treating errors and non-determinism as special cases of a similar behaviour allows for extending many of the methods we use in error handling to this kind of application as well. Not that these two are the only uses either. It's a much larger toolbox, though going overboard with it will result in unreadable hellish code.
- chriswarbo 3y ago
  Indeed, exception handlers are essentially the `COME FROM` madness from INTERCAL ;)
mpweiher 3y ago
Define "fundamentally". In the end it all comes down to NAND gates... ;-) But yes, it is very, very different. With an exception, which does achieve a similar effect of stopping further execution right there and then, you need to throw the exception and you need to check it. With dataflow, all you have to do is nothing. And exceptions are quite a tricky and heavyweight mechanism. What about the intermediate code? Does it need to run despite the exception? Will it just swallow the exception instead of passing it on? You can just pass the final error handler in and set it on the filter and there is no trickiness. And of course without the exception, there is a lot of code that needs to do error handling despite not really being involved. This becomes really, really noticeable when your async code is handled using callbacks. You just get two sets of callback everywhere. Just having the code be synchronous would be better, but with dataflow the whole issue just evaporated.
- gpderetta 3y ago
  In the end it boils down to continuations: your function either either takes a single return continuation that takes result|error variant or it takes two (or more) continuations: one for the happy path, and one for the error path. The former maps well to result objects, the latter is maps better to non local (checked) exceptions, but it easy to see how to transform from one to the other.
  - mpweiher 3y ago
    > In the end it boils down to continuations No it does not. > your function Only if you deal exclusively with procedures/functions. With dataflow, that's not the case. You have two filters, and if the first filter detects an error, it just does nothing (i.e it does not pass the value to the second filter). Done. No need for exceptions or continuations, or weird return types.

kamray23 3y ago

That's got its downsides. Majorly, I feel like it's restrictive. Exception handling, especially asynchronous exception handling, can really disallow recovery from errors. Sure, when you have a straight data stream and you simply want to stop processing at an error, there's no data for the next stage and it never runs. But that greatly limits recovery from errors. If your reaction to sqrt(-1) is to "crash", ending the business part of the program and hopping straight to printing or logging or whatever for the error, it can be very difficult to, say, enlarge the domain. You might have the need to simply give some use-case specific value for negative numbers, and you can pretty simply do that in code, but it'll result in some coupling you don't want as now you have to test for being outside that domain. For sqrt the domain is simple to define, but for a lot of real-world logic, it isn't nearly that simple. You can, in some languages, trap the error. Dataflow machines often are not designed for that though, especially when dealing with copious asynchronicity. That's where explicit error types, such as Maybe come in. You get to write your happy path as if errors didn't exist, but function composition uses a different set of operators than you usually would. In cases where you do want to recover from an error, you handle both paths then and there, as part of the happy path, and possibly don't even allow for an error beyond that. Most importantly, to get any value which is not an abstract concept of "maybe value" out, you need to handle the existence of both paths gracefully. That can be very nice, and very useful. That being said, now that you're using explicit error types, you can escape the "happy path"/"error" dichotomy. No longer is there necessarily just a "no value". There can be a "no value, but". Or a "value, but". You can have several errors stack up in a chain of things which could be done in parallel and then collated as a result. You can even entirely give up on the concept of errors, since that's only a very special case. You can use it to encode non-determinism: each unit takes in a single value and processes it to several possible and different values. Combining them with the monadic bind operator, each unit outputting a list of values has those values concatenated at the end to the list of all outputs, then the next function runs for each of those values the set of outputs of them are joined for the next stage and so on. This can be very, very useful for things like traversing graphs. You can, as Haskell programmers often do, use it to haul around a bit of "state" in a technically pure manner (purity is in certain cases very desirable). Perhaps the most infamous use of applicatives and monads in programming is the Haskell IO Monad, which encodes no real paths at all. IO simply is a virus which latches on to everything you do with it, and getting out of IO requires touching the outside world in an impure manner, which in Haskell can "only" be done while evaluating the expression called "main", meaning that the expression called "main" becomes your only point of contact to the outside world and the only way to "unwrap" IO values. Once again, that is for (obsessive) purity reasons. Alternative applicatives even allow for things as simple as an alternative applicative functor based on zipping instead of nondeterminism. It's a surprisingly varied technique, going far beyond simple railroads, and offers a neat way to write "only" the happy path while staying pure (allowing simple equational static analysis and unit testing without the need for mocks to re-establish purity). It also offers you many other functorial tools which are all linked by a specific composition behaviour. Further, other applicatives provide extra tools when you just need a functor which represents a specific way to combine two values. Using alternative methods, you often run out of ways to extend existing code, create something completely unreadable yet somehow isomorphic, or you create something readable and extensible, but due to the lack of purity inherent in some exceptional business logic, it becomes very hard to test without extensive mocking. Toeing the line between purity, readability and extensibility can be done in many ways, but functors and especially monads are among the S-tier when it comes to it. That being said, Haskell can get a bit goofy with it, by no means is purity an absolute value to be always chased. I kind of wish more """mainstream""" languages contained better monadic toolboxes for those times when you see an issue and you know you could solve it better than what you have to otherwise do if you only had the tools for it.

jiehong 3y ago

Would you have a code example to help clarify this in my mind?