8 ms·

The author followed up this post with another one a few years later titled "Against Railway Oriented Programming": https://fsharpforfunandprofit.com/posts/agai

The author followed up this post with another one a few years later titled "Against Railway Oriented Programming": https://fsharpforfunandprofit.com/posts/against-railway-oriented-programming/ https://fsharpforfunandprofit.com/posts/against-railway-orie... Railway-oriented programming is an interesting concept and it does have its use cases, but it does need to come with a massive health warning. I've often seen it used in practice to reinvent exception handling badly, and this is something I consider particularly ill advised because exceptions, when understood and used correctly, provide a much cleaner and more effective way of handling error conditions in most cases. The thing about exceptions is that in most cases, they make the safe option the default. An error condition is an indication that your code can not do what its specification says that it does, and in that case you need to stop what you are doing, because to continue regardless means that your code will be operating under assumptions that are incorrect, potentially corrupting data. Error conditions can happen for a wide variety of reasons, many of which you do not anticipate and can not plan for, and in those cases the only safe option is to clean up if necessary and then propagate the error up to the caller. Exceptions do this automatically for you by default (you need to explicitly override it with a try/catch block) but alternative approaches, such as railway oriented programming, require you to add in a whole lot of extra boilerplate code that is easy to forget and easy to get wrong. If you can't handle the error condition on the way up the call stack, you would then log it at the top level and report a generic error to the user. Having said that I see two particular use cases for this kind of technique. The first is situations where you need to handle specific, well defined and anticipated errors right at the point at which they occur. Validation is one example that comes to mind; another example is where you are trying to fetch a file or database record that does not exist. The second is situations where exception handling is not available for whatever reason. Asynchronous code using promises (for example with jQuery) are pretty much an exact implementation of railway oriented programming, but since modern JavaScript now has async/await, we can now use exception handling in these scenarios.

fsloth 3y ago
"The first is situations where you need to handle specific, well defined and anticipated errors right at the point at which they occur" Barring system level errors can you give an example of an error state that's not like that, that would then rather merit an exception? I would like to understand your point of view, is it due to the nature of the problem, or the constraints of runtime that make exceptions preferable. In the C++ code I need to write, we can 1. check data for error conditions in the beginning 2. if we fail the error check, let application crash 3. use the found error state to debug and fix the error in the initial checking code. The data my code needs to process is fairly straightforward - data abiding by some known CAD data format or given geometric topology, so the error conditions are "quite easy" to tackle in the sense that there is an understanding what correct data looks like in the first place.
- xmcqdpt2 3y ago
  C++/Rust are different because exceptions in those languages are expensive and culturally counter indicated. For the runtime-hosted languages the author is talking about (JVM, CLR, Python etc.), optionally throwing an exception is much cheaper than constantly creating and unwrapping Result objects. Your example is a perfect case where one would prefer to throw: say you have a parser that parses your file and the parser is expensive because the files are large. You are better off throwing out of your parsing iteration then doing a Result.map in your hot loop. (However you might want to wrap the top level of the parser in a Result and return that.)
- jammycakes 3y ago
  Missing dependencies. External services having gone offline. Timeouts. Foreign key violations. Data corruption. Invalid user input. Incorrect assumptions about how a third party library works. Incorrectly configured firewalls. Bugs in your code. Subtle incompatibilities between libraries, frameworks or protocols. Botched deployments. Hacking attacks. The list is endless. Probably not so much of an issue if you're dealing with well validated CAD data and most of your processing is in-memory using your own code. But if you're working with enterprise applications talking to each other via microservices written by different teams with different levels of competence, legacy code (sometimes spanning back decades), complex and poorly documented third party libraries and frameworks, design decisions that are more political than technical, and so on and so forth, it can quickly mount up.
  - catiopatio 3y ago
    External services having gone offline, timeouts, and invalid user input are expected conditions you should handle locally. Almost everything else you listed represents a bug in your software that should terminate execution. I’m more than a little shocked that you think yeeting exceptions up the call stack is appropriate for these cases.
    - jammycakes 3y ago
      > External services having gone offline, timeouts, and invalid user input are expected conditions you should handle locally. Not necessarily. You should only handle expected conditions locally if there is a specific action that you need to take in response to them -- for example, correcting the condition that caused the error, retrying, falling back to an alternative, or cleaning up before reporting failure. Even if you do know what all the different failure modes are, you will only need to do this in a minority of cases, and those will be determined by your user stories, your acceptance criteria, your business priorities and your budgetary constraints. That is what I mean by "expected conditions." Ones that are (or that in theory could be) called out on your Jira tickets or your specification documents. For anything else, the correct course of action is to assume that your own method is not able to fulfil its contract and to report that particular fact to its caller. Which is what "yeeting exceptions up the call stack" actually does. > Almost everything else you listed represents a bug in your software that should terminate execution. Well of course it represents a bug in your software, but you most certainly do not terminate execution altogether. You perform any cleanup that may be necessary, you record an event in your error log, and you show a generic error message to whoever needs to know about it, whether that be the end user or your support team. Again, what action you need to do in these cases will depend on your user stories, your acceptance criteria, your business priorities and your budgetary constraints. But it is usually done right at the top level of your code in a single location. That is why "yeeting exceptions up the call stack" is appropriate for these cases. You only terminate execution altogether if your process is so deeply diseased that for it to continue would cause even more damage. For example, memory corruption or failures of safety-critical systems. > I’m more than a little shocked that you think yeeting exceptions up the call stack is appropriate for these cases. I hope I've clarified what "yeeting exceptions up the call stack" actually does. The alternative to "yeeting exceptions up the call stack" when you don't have any specific cleanup or corrective action that you can do is to continue execution regardless. This is almost never the correct thing to do as it means your code is running under assumptions that are incorrect. And that is a recipe for data corruption and all sorts of other nasties.
piaste 3y ago
> Exceptions do this automatically for you by default (you need to explicitly override it with a try/catch block) but alternative approaches, such as railway oriented programming, require you to add in a whole lot of extra boilerplate code that is easy to forget and easy to get wrong. The unfortunately missing part of exceptions (in mainstream languages) is that they handle this invisibly. Figuring out, at compile time, what sort of exceptions can appear inside a given function is not obvious. That's the big payoff of ROP: you can look at any function signature and immediately know what sort of errors can come out of it. Mitigating the downside of ROP (boilerplate) can be done to various extents, depending on the language. Haskell has do-notation. In F#, using the result computation expression [0] can make your code extremely clean: type LoginError = InvalidUser | InvalidPwd | Unauthorized of AuthError let login (username : string) (password : string) : Result<AuthToken, LoginError> = result { // requireSome unwraps a Some value or gives the specified error if None let! user = username |> tryGetUser |> Result.requireSome InvalidUser // requireTrue gives the specified error if false do! user |> isPwdValid password |> Result.requireTrue InvalidPwd // Error value is wrapped/transformed (Unauthorized has signature AuthError -> LoginError) do! user |> authorize |> Result.mapError Unauthorized return user |> createAuthToken } Could we do the reverse, i.e. mitigate the downside of exceptions? Is there a linter, code analyzer, or some other compile-time tool that can integrate with a Java IDE and automatically display the uncaught exceptions that might be thrown by a given line of code? [0] https://demystifyfp.gitbook.io/fstoolkit-errorhandling/fstoolkit.errorhandling/result/ce https://demystifyfp.gitbook.io/fstoolkit-errorhandling/fstoo...
- bertylicious 3y ago
  Java has/had a compiler check that forced you to write catch blocks or `throws` annotations in/on functions that call other functions which might throw. The feature is called "checked exceptions" and I believe it has been discarded for its inconvenience by now. Sometimes it feels like developers are going in circles while trying to find the most optimal way to handle errors.
  - brabel 3y ago
    People are now realizing that having the errors a function can cause right in the type system may actually have been a good idea, but when you point out that Result is not the only way and that Java checked Exceptions do the exact same thing (and so does the Zig error handling mechanism which is a third variant of the idea), they come up with all sorts of easily dismissable nonsense to explain why the two are very different.
bertylicious 3y ago
I think Rust has shown very well how ROP with first-class syntax support pretty much eliminates all boilerplate code. IMHO Rust nailed error handling with the `Result` type/trait.
- scotty79 3y ago
  It came to my mind too but then I got confused, what if the type of the Result changes along the function call chain and you want to propagate Errors with minimal effort? Then I saw this stackoverflow question and it seems that ? operator does quite smart thing and is as easy to use as possible. https://stackoverflow.com/questions/31172451/is-there-a-non-messy-way-to-chain-the-results-of-functions-that-return-option-va#31172699 https://stackoverflow.com/questions/31172451/is-there-a-non-...
  - mprovost 3y ago
    The Rust community is converging on the anyhow crate to chain multiple error types without having to enumerate each.
    - benecollyridam 3y ago
      It looks like thiserror (6.6M downloads/month) is more popular even though anyhow (5.7M downloads/month) is listed as #1 https://lib.rs/keywords/error https://lib.rs/keywords/error
      - dathinab 3y ago
        they are not competing they handle two different cases anyhow is for type erased errors, which is mainly used for the kind of errors you mainly propagate upward without handling them in any fine grained way. It's mainly used in applications (instead of libraries). For example in a web server anyhow errors will likely yield Internal Serer errors. thiserror provides a derive (codegen) to easily create your own error. It's much more often used by libraries, but if an application doesn't want to handle this errors they will likely be converted into anyhow errors. A very common use case is to apply it on an enum which represent "one of many errors" e.g. as a dump example `enum Error { BadArgument(...), ConstraintViolation(...), ... }` and it's no absurd in some cases to have a mixture e.g. an enum variant `Unexpected(anyhow::Error)` which represents various very unpexted errors which likely could be bugs and you might have considered panicing there but decided to propagate them instead to avoid panic related problems
sasmithjr 3y ago
In my mind, the difference between errors-as-values and exceptions is most useful when describing domain-specific errors and other issues that you have to handle in support of the domain/problem space. To me, domain errors make sense as errors-as-values, but your database being unreachable is unrelated to the domain and makes sense as an exception. > another example is where you are trying to fetch a file or database record that does not exist I think this depends on whether or not you expect the file/record to exist. Handling a request from a user where the user provided the id used for lookup? The lookup itself is validation of the user input. But if you retrieved a DB record that has a blob name associated with it and your blob storage says that a blob doesn't exist by that name? I find that to be a great situation for an exception. The errors-or-exception line is fuzzy and going to be dependent on your team and the problems you're solving, but I've found that it's a decent rule of thumb.
pharmakom 3y ago
> An error condition is an indication that your code can not do what its specification says that it does Meanwhile, in real code-bases, exceptions are used for all sorts of expected but less common situations, like ProductNotFoundException
jmull 3y ago
I think the conventional way exceptions are implemented is pretty bad. First, a lot of languages make you use an awkward, unnecessary scope to catch an exception. e.g., you want to declare and initialize a variable to the value of a function that can throw (and assign some other value if it does. Well, you've got to split the declaration and initialization, putting the declaration outside the scopes try and catch create. That one's an unforced error -- languages don't have to do that to use exceptions, but for some reason many do. It's pretty weird to have to add homespun utilities for fundamental control flow scenarios. But the bigger issue is that you really want to handle the error conditions at the lowest level where you have enough context to do so correctly. That's usually pretty low, but exceptions default to "send it all the way to the top". The default is either invisible or invisible in practice, depending on the language, and wrong, so programs end up riddled with these issues. You tend to end up with these higher-level functions that can throw all kinds of exceptions, many of which are meaningless to the caller. E.g. someone adds a file cache one day and all of a sudden some higher-level HandleRequest function can through a IO exception... because the cache code didn't handle it... because they never even realized it was a possibility. You couldn't design a better mechanism for creating leaky abstractions. I think anything a function might return needs to be an explicit part of its signature, and a caller needs to handle it explicitly, even if just to indicate, pass it up the line. The langue doesn't need to require a lot of boilerplate to do this. That's just my experience from having lived through it.
- thuridas 3y ago
  If you catch them at the lowest level you end with N different exceptions that are coupled with implementation. I only need different exceptions if I am treating them different. Oh! And you typically catch it next to the failure ( maybe wrapping it in another exception) out in a global exception handler
klysm 3y ago
I disagree that exceptions are better in most cases. Exceptions aren't captured effectively in most type systems so it's hard to ensure you've covered all your bases. When used effectively, discriminated unions for return types force you to handle all the cases and the result is much more robust in my experience.

fsloth 3y ago

"The first is situations where you need to handle specific, well defined and anticipated errors right at the point at which they occur" Barring system level errors can you give an example of an error state that's not like that, that would then rather merit an exception? I would like to understand your point of view, is it due to the nature of the problem, or the constraints of runtime that make exceptions preferable. In the C++ code I need to write, we can 1. check data for error conditions in the beginning 2. if we fail the error check, let application crash 3. use the found error state to debug and fix the error in the initial checking code. The data my code needs to process is fairly straightforward - data abiding by some known CAD data format or given geometric topology, so the error conditions are "quite easy" to tackle in the sense that there is an understanding what correct data looks like in the first place.

xmcqdpt2 3y ago
C++/Rust are different because exceptions in those languages are expensive and culturally counter indicated. For the runtime-hosted languages the author is talking about (JVM, CLR, Python etc.), optionally throwing an exception is much cheaper than constantly creating and unwrapping Result objects. Your example is a perfect case where one would prefer to throw: say you have a parser that parses your file and the parser is expensive because the files are large. You are better off throwing out of your parsing iteration then doing a Result.map in your hot loop. (However you might want to wrap the top level of the parser in a Result and return that.)
jammycakes 3y ago
Missing dependencies. External services having gone offline. Timeouts. Foreign key violations. Data corruption. Invalid user input. Incorrect assumptions about how a third party library works. Incorrectly configured firewalls. Bugs in your code. Subtle incompatibilities between libraries, frameworks or protocols. Botched deployments. Hacking attacks. The list is endless. Probably not so much of an issue if you're dealing with well validated CAD data and most of your processing is in-memory using your own code. But if you're working with enterprise applications talking to each other via microservices written by different teams with different levels of competence, legacy code (sometimes spanning back decades), complex and poorly documented third party libraries and frameworks, design decisions that are more political than technical, and so on and so forth, it can quickly mount up.
- catiopatio 3y ago
  External services having gone offline, timeouts, and invalid user input are expected conditions you should handle locally. Almost everything else you listed represents a bug in your software that should terminate execution. I’m more than a little shocked that you think yeeting exceptions up the call stack is appropriate for these cases.
  - jammycakes 3y ago
    > External services having gone offline, timeouts, and invalid user input are expected conditions you should handle locally. Not necessarily. You should only handle expected conditions locally if there is a specific action that you need to take in response to them -- for example, correcting the condition that caused the error, retrying, falling back to an alternative, or cleaning up before reporting failure. Even if you do know what all the different failure modes are, you will only need to do this in a minority of cases, and those will be determined by your user stories, your acceptance criteria, your business priorities and your budgetary constraints. That is what I mean by "expected conditions." Ones that are (or that in theory could be) called out on your Jira tickets or your specification documents. For anything else, the correct course of action is to assume that your own method is not able to fulfil its contract and to report that particular fact to its caller. Which is what "yeeting exceptions up the call stack" actually does. > Almost everything else you listed represents a bug in your software that should terminate execution. Well of course it represents a bug in your software, but you most certainly do not terminate execution altogether. You perform any cleanup that may be necessary, you record an event in your error log, and you show a generic error message to whoever needs to know about it, whether that be the end user or your support team. Again, what action you need to do in these cases will depend on your user stories, your acceptance criteria, your business priorities and your budgetary constraints. But it is usually done right at the top level of your code in a single location. That is why "yeeting exceptions up the call stack" is appropriate for these cases. You only terminate execution altogether if your process is so deeply diseased that for it to continue would cause even more damage. For example, memory corruption or failures of safety-critical systems. > I’m more than a little shocked that you think yeeting exceptions up the call stack is appropriate for these cases. I hope I've clarified what "yeeting exceptions up the call stack" actually does. The alternative to "yeeting exceptions up the call stack" when you don't have any specific cleanup or corrective action that you can do is to continue execution regardless. This is almost never the correct thing to do as it means your code is running under assumptions that are incorrect. And that is a recipe for data corruption and all sorts of other nasties.

piaste 3y ago

> Exceptions do this automatically for you by default (you need to explicitly override it with a try/catch block) but alternative approaches, such as railway oriented programming, require you to add in a whole lot of extra boilerplate code that is easy to forget and easy to get wrong. The unfortunately missing part of exceptions (in mainstream languages) is that they handle this invisibly. Figuring out, at compile time, what sort of exceptions can appear inside a given function is not obvious. That's the big payoff of ROP: you can look at any function signature and immediately know what sort of errors can come out of it. Mitigating the downside of ROP (boilerplate) can be done to various extents, depending on the language. Haskell has do-notation. In F#, using the result computation expression [0] can make your code extremely clean: type LoginError = InvalidUser | InvalidPwd | Unauthorized of AuthError let login (username : string) (password : string) : Result<AuthToken, LoginError> = result { // requireSome unwraps a Some value or gives the specified error if None let! user = username |> tryGetUser |> Result.requireSome InvalidUser // requireTrue gives the specified error if false do! user |> isPwdValid password |> Result.requireTrue InvalidPwd // Error value is wrapped/transformed (Unauthorized has signature AuthError -> LoginError) do! user |> authorize |> Result.mapError Unauthorized return user |> createAuthToken } Could we do the reverse, i.e. mitigate the downside of exceptions? Is there a linter, code analyzer, or some other compile-time tool that can integrate with a Java IDE and automatically display the uncaught exceptions that might be thrown by a given line of code? [0] https://demystifyfp.gitbook.io/fstoolkit-errorhandling/fstoolkit.errorhandling/result/ce https://demystifyfp.gitbook.io/fstoolkit-errorhandling/fstoo...

bertylicious 3y ago
Java has/had a compiler check that forced you to write catch blocks or `throws` annotations in/on functions that call other functions which might throw. The feature is called "checked exceptions" and I believe it has been discarded for its inconvenience by now. Sometimes it feels like developers are going in circles while trying to find the most optimal way to handle errors.
- brabel 3y ago
  People are now realizing that having the errors a function can cause right in the type system may actually have been a good idea, but when you point out that Result is not the only way and that Java checked Exceptions do the exact same thing (and so does the Zig error handling mechanism which is a third variant of the idea), they come up with all sorts of easily dismissable nonsense to explain why the two are very different.

bertylicious 3y ago

I think Rust has shown very well how ROP with first-class syntax support pretty much eliminates all boilerplate code. IMHO Rust nailed error handling with the `Result` type/trait.

scotty79 3y ago
It came to my mind too but then I got confused, what if the type of the Result changes along the function call chain and you want to propagate Errors with minimal effort? Then I saw this stackoverflow question and it seems that ? operator does quite smart thing and is as easy to use as possible. https://stackoverflow.com/questions/31172451/is-there-a-non-messy-way-to-chain-the-results-of-functions-that-return-option-va#31172699 https://stackoverflow.com/questions/31172451/is-there-a-non-...
- mprovost 3y ago
  The Rust community is converging on the anyhow crate to chain multiple error types without having to enumerate each.
  - benecollyridam 3y ago
    It looks like thiserror (6.6M downloads/month) is more popular even though anyhow (5.7M downloads/month) is listed as #1 https://lib.rs/keywords/error https://lib.rs/keywords/error
    - dathinab 3y ago
      they are not competing they handle two different cases anyhow is for type erased errors, which is mainly used for the kind of errors you mainly propagate upward without handling them in any fine grained way. It's mainly used in applications (instead of libraries). For example in a web server anyhow errors will likely yield Internal Serer errors. thiserror provides a derive (codegen) to easily create your own error. It's much more often used by libraries, but if an application doesn't want to handle this errors they will likely be converted into anyhow errors. A very common use case is to apply it on an enum which represent "one of many errors" e.g. as a dump example `enum Error { BadArgument(...), ConstraintViolation(...), ... }` and it's no absurd in some cases to have a mixture e.g. an enum variant `Unexpected(anyhow::Error)` which represents various very unpexted errors which likely could be bugs and you might have considered panicing there but decided to propagate them instead to avoid panic related problems

sasmithjr 3y ago

In my mind, the difference between errors-as-values and exceptions is most useful when describing domain-specific errors and other issues that you have to handle in support of the domain/problem space. To me, domain errors make sense as errors-as-values, but your database being unreachable is unrelated to the domain and makes sense as an exception. > another example is where you are trying to fetch a file or database record that does not exist I think this depends on whether or not you expect the file/record to exist. Handling a request from a user where the user provided the id used for lookup? The lookup itself is validation of the user input. But if you retrieved a DB record that has a blob name associated with it and your blob storage says that a blob doesn't exist by that name? I find that to be a great situation for an exception. The errors-or-exception line is fuzzy and going to be dependent on your team and the problems you're solving, but I've found that it's a decent rule of thumb.

pharmakom 3y ago

> An error condition is an indication that your code can not do what its specification says that it does Meanwhile, in real code-bases, exceptions are used for all sorts of expected but less common situations, like ProductNotFoundException

jmull 3y ago

I think the conventional way exceptions are implemented is pretty bad. First, a lot of languages make you use an awkward, unnecessary scope to catch an exception. e.g., you want to declare and initialize a variable to the value of a function that can throw (and assign some other value if it does. Well, you've got to split the declaration and initialization, putting the declaration outside the scopes try and catch create. That one's an unforced error -- languages don't have to do that to use exceptions, but for some reason many do. It's pretty weird to have to add homespun utilities for fundamental control flow scenarios. But the bigger issue is that you really want to handle the error conditions at the lowest level where you have enough context to do so correctly. That's usually pretty low, but exceptions default to "send it all the way to the top". The default is either invisible or invisible in practice, depending on the language, and wrong, so programs end up riddled with these issues. You tend to end up with these higher-level functions that can throw all kinds of exceptions, many of which are meaningless to the caller. E.g. someone adds a file cache one day and all of a sudden some higher-level HandleRequest function can through a IO exception... because the cache code didn't handle it... because they never even realized it was a possibility. You couldn't design a better mechanism for creating leaky abstractions. I think anything a function might return needs to be an explicit part of its signature, and a caller needs to handle it explicitly, even if just to indicate, pass it up the line. The langue doesn't need to require a lot of boilerplate to do this. That's just my experience from having lived through it.

thuridas 3y ago
If you catch them at the lowest level you end with N different exceptions that are coupled with implementation. I only need different exceptions if I am treating them different. Oh! And you typically catch it next to the failure ( maybe wrapping it in another exception) out in a global exception handler

klysm 3y ago

I disagree that exceptions are better in most cases. Exceptions aren't captured effectively in most type systems so it's hard to ensure you've covered all your bases. When used effectively, discriminated unions for return types force you to handle all the cases and the result is much more robust in my experience.