6 ms·

So I'm not talking about exceptions, I'm talking about talking about errors. You can easily generate thousands of errors per second if you're simply validating

by steego 8y ago

So I'm not talking about exceptions, I'm talking about talking about errors. You can easily generate thousands of errors per second if you're simply validating and routing large streams of data.

jstimpfle 8y ago
It's nice to get concrete and leave the abstract shit talk (of which I've been guilty many times). So let's focus on that problem you mention and not argue errors vs exceptions terminology. In the case you mention I would clearly make two streams of data. In the simplest case, two append-only dynamically allocated arrays, one for parsing errors and one for parsed results. This is very likely possible to do - unless one needs an exact ordering, in which case it could be still possible to add positions to each of the streams. It's intuitively clear to me that this approach will be significantly faster than a Result<>/Either based approach. Because control flow is much simpler. There are two consumers, one for the error stream, and one for the parsed stream. There is no switching between the cases on the consuming side, and complexity is much reduced because each part can do one thing and do it well. You also get the freedom to process the streams independently, e.g. at different points in time, or not at all, etc. It's much simpler - if you try this route once, you will ask yourself afterwards, how have you ever endured this complex algebraic data types stuff. Yes, you could still add a consumer of those Result<> values that demultiplexes them in two. But nobody does that. Because, why did we introduce Result values in the first place then? This is just one of the many examples I encounter daily and that demonstrate how advanced type systems and ideas like OOP or FP encourage bad program structure. They are just not good ideas.
- mdpopescu 8y ago
  > Yes, you could still add a consumer of those Result<> values that demultiplexes them in two. But nobody does that. What? That's exactly how you consume those values. In pseudocode: match result when success: ... when error: ...
  - jstimpfle 8y ago
    This is not demultiplexing into two different data streams, but handling both clumped together. Exactly what I was speaking of: Avoiding this clumping and processing the cases as separate streams (and independently!) brings huge payoffs. Just try it; you will realize how much simpler your application gets - so many problems simply go away if you make more homogeneous data streams, and not fewer heterogeneous ones. And so many possibilities magically open up. Very briefly, here is the difference: def process_both_cases(stream): for item in stream: if case1: do_complex_stuff_1(item) elif case2: do_complex_stuff_2(item) else: assert(False) vs def process_case1(stream): for item in stream: do_complex_stuff_1(item) def process_case2(stream): for item in stream: do_complex_stuff_2(item) I think it's immediately clear how much better the second version is. It's less complicated: The conditionals are not needed. Processing the cases is decoupled.
    - KirinDave 8y ago
      You... do realize that it's quite easy to make the former out of the later, and that's how every FP parser combinator library works. Using a combinator to combine two undecorated funtions into one, you don't need to result to two streams, you can significantly simplify your runtime by only needing one. It's been written for you: http://hackage.haskell.org/package/base-4.11.1.0/docs/Control-Arrow.html#g:5 http://hackage.haskell.org/package/base-4.11.1.0/docs/Contro... Please consider that the incredible strength of ADTs is that you can just write generic code to combine those things without resorting to the crabtastic pattern matches that you're proposing, AND avoid the ugly implicitly unpredictably ordered world that you're describing. Heck, we don't even need to resort to combinators for this. It's just you making up a complex case by saying, "What if I hide half of example 2 by handwaving it?" You still are going to have a case1/case2 decomposition in your second version. It's universally worse.
      - jstimpfle 8y ago
        I don't see a pattern match anywhere in my example. And it's not in the slightest "unpredictably ordered" and has much simpler control flow. > You still are going to have a case1/case2 decomposition in your second version. It's universally worse. No. There is simply none. The producer makes them as two, and the two are processed independently. There is no problem if you don't make one. If you had ever tried the approach I propose out you'd know the flexibility that comes with it. (I'm currently working on language tooling, and there it is absolutely the case that it's very easy do without ASTs, and process the cases independently). The only thing I "handwaved away" is the calls: s1, s2 = process_data() process_case1(s1) process_case2(s2) There is no pattern match there, sorry to disappoint. Where I need to relate the individual items of s1 and s2 in a common ordering, I've had great success with this approach: s1, s2, forks = process_data() process_case1(s1) process_case2(s2) process_forks(forks) There, "forks" does contain an ADT of pointers into s1 and s2. But it contains nothing more, so it's very minimalistic. The advantage from the first case is still there: Most code profits from the homogeneity of s1 and s2, and from the actual, physical decoupling of the cases, and so much pattern matching just goes away. (Btw. I'm not interested in fancy FP constructs. You will not convince me that you can process ADTs without pattern matches. You might be able to avoid writing them down in the source code, but I care about what actually happens on the hardware.) (And actually I think avoiding to write down the pattern match in the source code and only generating it magically with higher level constructs is a bad, bad idea. It has poor readability. And it encourages the mindset "let's just do the worst shit on the actual machine. We can avoid writing it in the source code, and we don't care what happens on the machine and how slow it is. And we're too lazy to think about actual solutions that actually avoid real work".)
- steego 8y ago
  I'm sorry, I ignored the second part of your question. I was responding to: > What is an example of exception that happens many times per second? I made a distinction between exceptions and errors because exceptions are a specific mechanism for propagating errors. I wasn't trying to be pedantic, I was trying to point out that the answer to your first question is more obvious if you subsitute out the word exception for error. The answer is simply: Because you're probably the one generating the errors and you're going to generate a lot of them. With regards to the second question: > And where it could not be easily avoided to step on the error path at all? I'm not sure how to interpret this question, but I'll throw out an answer anyway. You don't need to take a Railway Oriented Approach. There are other ways. Your reply was an example. For a streaming scenario, applying a function with the signature ('a -> Result<'b,'e>) to a Stream.map is pretty trivial. It may not be the most performant or scalable, but it's easy to do and easy to reason about. However, one could also easily create two streams as you suggested with a partition type function with the signature: (Stream<'a> -> ('a -> Result<'b,'e>) -> (Stream<'b> * Stream<'e>) You're free to use the Result<> type as much or as little as it makes sense. How one should best solve the problem depends on the scope of the problem and what you need to do downstream, right?
  - jstimpfle 8y ago
    > I wasn't trying to be pedantic, I was trying to point out... I agree! > but it's easy to do and easy to reason about I agree it's an obvious/intuitive approach to take, just like OOP. However, from a computational and software maintainability standpoint, I think it rarely pays out. > How one should best solve the problem depends on the scope of the problem and what you need to do downstream That's right, however ADTs are very rarely occurring naturally in what I do (I've dabbled in quite a few areas). I've found them to occur naturally in parsers (because languages have forks in their syntaxes), but even there I've had great gains by avoiding ASTs altogether, and making multiple streams, supported by a ADT stream of simple data that does not contain real data but only "joins" the other ones (it contains only a type-flag and a pointer into one of the other streams). I think ADTs are the right choice whenever the ordering between a set of things with heterogeneous types is very important. If that's not the case, code can be vastly simplified by sorting the data by their types and processing them as separate bins - separately and independently.

by steego 8y ago

So I'm not talking about exceptions, I'm talking about talking about errors. You can easily generate thousands of errors per second if you're simply validating and routing large streams of data.

jstimpfle 8y ago

It's nice to get concrete and leave the abstract shit talk (of which I've been guilty many times). So let's focus on that problem you mention and not argue errors vs exceptions terminology. In the case you mention I would clearly make two streams of data. In the simplest case, two append-only dynamically allocated arrays, one for parsing errors and one for parsed results. This is very likely possible to do - unless one needs an exact ordering, in which case it could be still possible to add positions to each of the streams. It's intuitively clear to me that this approach will be significantly faster than a Result<>/Either based approach. Because control flow is much simpler. There are two consumers, one for the error stream, and one for the parsed stream. There is no switching between the cases on the consuming side, and complexity is much reduced because each part can do one thing and do it well. You also get the freedom to process the streams independently, e.g. at different points in time, or not at all, etc. It's much simpler - if you try this route once, you will ask yourself afterwards, how have you ever endured this complex algebraic data types stuff. Yes, you could still add a consumer of those Result<> values that demultiplexes them in two. But nobody does that. Because, why did we introduce Result values in the first place then? This is just one of the many examples I encounter daily and that demonstrate how advanced type systems and ideas like OOP or FP encourage bad program structure. They are just not good ideas.

mdpopescu 8y ago
> Yes, you could still add a consumer of those Result<> values that demultiplexes them in two. But nobody does that. What? That's exactly how you consume those values. In pseudocode: match result when success: ... when error: ...
- jstimpfle 8y ago
  This is not demultiplexing into two different data streams, but handling both clumped together. Exactly what I was speaking of: Avoiding this clumping and processing the cases as separate streams (and independently!) brings huge payoffs. Just try it; you will realize how much simpler your application gets - so many problems simply go away if you make more homogeneous data streams, and not fewer heterogeneous ones. And so many possibilities magically open up. Very briefly, here is the difference: def process_both_cases(stream): for item in stream: if case1: do_complex_stuff_1(item) elif case2: do_complex_stuff_2(item) else: assert(False) vs def process_case1(stream): for item in stream: do_complex_stuff_1(item) def process_case2(stream): for item in stream: do_complex_stuff_2(item) I think it's immediately clear how much better the second version is. It's less complicated: The conditionals are not needed. Processing the cases is decoupled.
  - KirinDave 8y ago
    You... do realize that it's quite easy to make the former out of the later, and that's how every FP parser combinator library works. Using a combinator to combine two undecorated funtions into one, you don't need to result to two streams, you can significantly simplify your runtime by only needing one. It's been written for you: http://hackage.haskell.org/package/base-4.11.1.0/docs/Control-Arrow.html#g:5 http://hackage.haskell.org/package/base-4.11.1.0/docs/Contro... Please consider that the incredible strength of ADTs is that you can just write generic code to combine those things without resorting to the crabtastic pattern matches that you're proposing, AND avoid the ugly implicitly unpredictably ordered world that you're describing. Heck, we don't even need to resort to combinators for this. It's just you making up a complex case by saying, "What if I hide half of example 2 by handwaving it?" You still are going to have a case1/case2 decomposition in your second version. It's universally worse.
    - jstimpfle 8y ago
      I don't see a pattern match anywhere in my example. And it's not in the slightest "unpredictably ordered" and has much simpler control flow. > You still are going to have a case1/case2 decomposition in your second version. It's universally worse. No. There is simply none. The producer makes them as two, and the two are processed independently. There is no problem if you don't make one. If you had ever tried the approach I propose out you'd know the flexibility that comes with it. (I'm currently working on language tooling, and there it is absolutely the case that it's very easy do without ASTs, and process the cases independently). The only thing I "handwaved away" is the calls: s1, s2 = process_data() process_case1(s1) process_case2(s2) There is no pattern match there, sorry to disappoint. Where I need to relate the individual items of s1 and s2 in a common ordering, I've had great success with this approach: s1, s2, forks = process_data() process_case1(s1) process_case2(s2) process_forks(forks) There, "forks" does contain an ADT of pointers into s1 and s2. But it contains nothing more, so it's very minimalistic. The advantage from the first case is still there: Most code profits from the homogeneity of s1 and s2, and from the actual, physical decoupling of the cases, and so much pattern matching just goes away. (Btw. I'm not interested in fancy FP constructs. You will not convince me that you can process ADTs without pattern matches. You might be able to avoid writing them down in the source code, but I care about what actually happens on the hardware.) (And actually I think avoiding to write down the pattern match in the source code and only generating it magically with higher level constructs is a bad, bad idea. It has poor readability. And it encourages the mindset "let's just do the worst shit on the actual machine. We can avoid writing it in the source code, and we don't care what happens on the machine and how slow it is. And we're too lazy to think about actual solutions that actually avoid real work".)
steego 8y ago
I'm sorry, I ignored the second part of your question. I was responding to: > What is an example of exception that happens many times per second? I made a distinction between exceptions and errors because exceptions are a specific mechanism for propagating errors. I wasn't trying to be pedantic, I was trying to point out that the answer to your first question is more obvious if you subsitute out the word exception for error. The answer is simply: Because you're probably the one generating the errors and you're going to generate a lot of them. With regards to the second question: > And where it could not be easily avoided to step on the error path at all? I'm not sure how to interpret this question, but I'll throw out an answer anyway. You don't need to take a Railway Oriented Approach. There are other ways. Your reply was an example. For a streaming scenario, applying a function with the signature ('a -> Result<'b,'e>) to a Stream.map is pretty trivial. It may not be the most performant or scalable, but it's easy to do and easy to reason about. However, one could also easily create two streams as you suggested with a partition type function with the signature: (Stream<'a> -> ('a -> Result<'b,'e>) -> (Stream<'b> * Stream<'e>) You're free to use the Result<> type as much or as little as it makes sense. How one should best solve the problem depends on the scope of the problem and what you need to do downstream, right?
- jstimpfle 8y ago
  > I wasn't trying to be pedantic, I was trying to point out... I agree! > but it's easy to do and easy to reason about I agree it's an obvious/intuitive approach to take, just like OOP. However, from a computational and software maintainability standpoint, I think it rarely pays out. > How one should best solve the problem depends on the scope of the problem and what you need to do downstream That's right, however ADTs are very rarely occurring naturally in what I do (I've dabbled in quite a few areas). I've found them to occur naturally in parsers (because languages have forks in their syntaxes), but even there I've had great gains by avoiding ASTs altogether, and making multiple streams, supported by a ADT stream of simple data that does not contain real data but only "joins" the other ones (it contains only a type-flag and a pointer into one of the other streams). I think ADTs are the right choice whenever the ordering between a set of things with heterogeneous types is very important. If that's not the case, code can be vastly simplified by sorting the data by their types and processing them as separate bins - separately and independently.