8 ms·

> Yes, you could still add a consumer of those Result<> values that demultiplexes them in two. But nobody does that. What? That's exactly how you consume those

> Yes, you could still add a consumer of those Result<> values that demultiplexes them in two. But nobody does that. What? That's exactly how you consume those values. In pseudocode: match result when success: ... when error: ...

jstimpfle 8y ago
This is not demultiplexing into two different data streams, but handling both clumped together. Exactly what I was speaking of: Avoiding this clumping and processing the cases as separate streams (and independently!) brings huge payoffs. Just try it; you will realize how much simpler your application gets - so many problems simply go away if you make more homogeneous data streams, and not fewer heterogeneous ones. And so many possibilities magically open up. Very briefly, here is the difference: def process_both_cases(stream): for item in stream: if case1: do_complex_stuff_1(item) elif case2: do_complex_stuff_2(item) else: assert(False) vs def process_case1(stream): for item in stream: do_complex_stuff_1(item) def process_case2(stream): for item in stream: do_complex_stuff_2(item) I think it's immediately clear how much better the second version is. It's less complicated: The conditionals are not needed. Processing the cases is decoupled.
- KirinDave 8y ago
  You... do realize that it's quite easy to make the former out of the later, and that's how every FP parser combinator library works. Using a combinator to combine two undecorated funtions into one, you don't need to result to two streams, you can significantly simplify your runtime by only needing one. It's been written for you: http://hackage.haskell.org/package/base-4.11.1.0/docs/Control-Arrow.html#g:5 http://hackage.haskell.org/package/base-4.11.1.0/docs/Contro... Please consider that the incredible strength of ADTs is that you can just write generic code to combine those things without resorting to the crabtastic pattern matches that you're proposing, AND avoid the ugly implicitly unpredictably ordered world that you're describing. Heck, we don't even need to resort to combinators for this. It's just you making up a complex case by saying, "What if I hide half of example 2 by handwaving it?" You still are going to have a case1/case2 decomposition in your second version. It's universally worse.
  - jstimpfle 8y ago
    I don't see a pattern match anywhere in my example. And it's not in the slightest "unpredictably ordered" and has much simpler control flow. > You still are going to have a case1/case2 decomposition in your second version. It's universally worse. No. There is simply none. The producer makes them as two, and the two are processed independently. There is no problem if you don't make one. If you had ever tried the approach I propose out you'd know the flexibility that comes with it. (I'm currently working on language tooling, and there it is absolutely the case that it's very easy do without ASTs, and process the cases independently). The only thing I "handwaved away" is the calls: s1, s2 = process_data() process_case1(s1) process_case2(s2) There is no pattern match there, sorry to disappoint. Where I need to relate the individual items of s1 and s2 in a common ordering, I've had great success with this approach: s1, s2, forks = process_data() process_case1(s1) process_case2(s2) process_forks(forks) There, "forks" does contain an ADT of pointers into s1 and s2. But it contains nothing more, so it's very minimalistic. The advantage from the first case is still there: Most code profits from the homogeneity of s1 and s2, and from the actual, physical decoupling of the cases, and so much pattern matching just goes away. (Btw. I'm not interested in fancy FP constructs. You will not convince me that you can process ADTs without pattern matches. You might be able to avoid writing them down in the source code, but I care about what actually happens on the hardware.) (And actually I think avoiding to write down the pattern match in the source code and only generating it magically with higher level constructs is a bad, bad idea. It has poor readability. And it encourages the mindset "let's just do the worst shit on the actual machine. We can avoid writing it in the source code, and we don't care what happens on the machine and how slow it is. And we're too lazy to think about actual solutions that actually avoid real work".)
    - KirinDave 8y ago
      > Where I need to relate the individual items of s1 and s2 in a common ordering, I've had great success with this approach: Sorry I misunderstood your use of the phrase "two streams." In my world multiple streams effectively severs causality between the streams. Or did I? You wrote: def process_case1(stream): for item in stream: do_complex_stuff_1(item) def process_case2(stream): for item in stream: do_complex_stuff_2(item) If this genuinely relates to your example above then the only possible consistent ordering without any async/multithreading is to exhaust stream1, then process your errors. If you don't do that and the implication is that the streams won't close on errors but rather continue to emit, then you really can't guarantee any ordering to the processing. Given this inconsistency, I'm going to continue the post with your intent of causality being preserved and assuming assuming that process_data() offers the next value (or null) and we're calling these in a loop. Because otherwise you have a whole heap of problems and your example doesn't even faintly resemble any subject matter from the post, or what most folks were talking about here. And if you believe that it doesn't destroy causality, you're either writing a lot of code to recombine your streams in lockstep (in which case, splitting is quite the extreme step to avoid matching) or you just don't understand the implications of stream semantics in this case. > There is no pattern match there, sorry to disappoint. However you provided a destructuring pattern match in your code! s1, s2 = process_data() Your process data is pattern matched on a generic product there. Sorry to disappoint. The actual type of the value is: (Either Null S1Type, Either Null S2Type) (And I assume it must be an `Either Null a` because we're discussing a case where either case1 or case2 will exist, so there's no valid value for one when the other exists.) This is sort of why folks who know about ADTs roll their eyes at the sizzling invective about their complexity. You often end up using the same solutions, you just don't name them. It seems like the naming itself is objectionable. You've written about 4 posts I can see here about how much you hate ADTs, but then you're happy for an implicit `Either Null a` on every value, no opting out, and `a,b = f(x)` is fine because the name of the ADT is a punctuation mark, I guess? As for the first example, Pattern Matches as Case statements are related in that a pattern match is a hyperactive case statement. So when you have one, you have the other to a varying degree. > (Btw. I'm not interested in fancy FP constructs. You will not convince me that you can process ADTs without pattern matches. You might be able to avoid writing them down in the source code, but I care about what actually happens on the hardware.) You seem to be making a huge deal out of pointer chasing. However, you're also suggesting a detour through the function contexts of process_case1 and process_case2 just to offer a null in one case. Your plan seems a bit unfair, while we're on the subject. You're arguing that a pattern match is "heavy overhead", but you're completely ignoring the actual use case of "railway programming": to make a single clean codepath obvious and localize error handling at the end. But the thing you've offered doesn't really do that. You're processing S1 and S2 (which I'll rename E1 for error), but what happens when I want to compose it? Extending the metaphor, your new code might look like: s1, e1 = process_data() process_ecase(e1) // If you don't put an if( s1 != null ) here you're going // to have to do a ton of function calls AND guarantee // your handlers check their args for null. if( s1 != null ) { s2, e2 = process_case(s1) process_ecase(e2) if( s2 != null ) { // ... } } You could perhaps change the ordering of these ops (or push the null checks into your code which has a bunch of problems). And most of all, your compiler is totally powerless to help you work out if you have an error or not. You might also instead try to embed the dispatch to the next step in the def blocks of each individual step (an approach that could never work with the stream example, as you need to define your stream network (or at least a global namespace for it) before streaming events through it. > And it encourages the mindset "let's just do the worst shit on the actual machine. Okay, your example extended above. The Haskell-style of this combinator pattern (which DOES require all errors have the same base type, let me be transparent about that) is: -- Assuming the steps are: a -> Either Error b let pipeline = process_data >>= step2 >>= step3 pipeline () `catchError` errorHandler The errorHandler is just a normal function taking your shared error type. Each step is just a function that returns an Either Error or its value. The values can vary so long as the types between steps line up (and the compiler checks this in F#, ML and Haskell). It's easy to read, too. It's easy to extend, and it's predictable, and it's made from very simple decorated functions. We can even safely make it interact with exceptions, if we need to, without overly complicating the callers. > And it encourages the mindset "let's just do the worst shit on the actual machine. We can avoid writing it in the source code What's completely confusing to me about your claim here is that you're doing dynamic dispatches and perhaps excess function calls, demanding your process functions do null checks, you have no clear looping construct, and your compiler has 0 opportunity for any sort of inlining but the most naive, cache busting type. But you're essentially dunking on what becomes BNE statements at the machine code level. At runtime, all the types are erased in the best case or preserved in exactly the same way as any normal object with a dispatch table would. The actual cost is, in most cases, indistinguishable from your if-statement rails. I actually think you like lots of elements of the FP style, you just seem to dislike the names. Perhaps if you could hold your nose with a clothespin and play through the stink of math-ish names, you might find more you like?

jstimpfle 8y ago

This is not demultiplexing into two different data streams, but handling both clumped together. Exactly what I was speaking of: Avoiding this clumping and processing the cases as separate streams (and independently!) brings huge payoffs. Just try it; you will realize how much simpler your application gets - so many problems simply go away if you make more homogeneous data streams, and not fewer heterogeneous ones. And so many possibilities magically open up. Very briefly, here is the difference: def process_both_cases(stream): for item in stream: if case1: do_complex_stuff_1(item) elif case2: do_complex_stuff_2(item) else: assert(False) vs def process_case1(stream): for item in stream: do_complex_stuff_1(item) def process_case2(stream): for item in stream: do_complex_stuff_2(item) I think it's immediately clear how much better the second version is. It's less complicated: The conditionals are not needed. Processing the cases is decoupled.

KirinDave 8y ago
You... do realize that it's quite easy to make the former out of the later, and that's how every FP parser combinator library works. Using a combinator to combine two undecorated funtions into one, you don't need to result to two streams, you can significantly simplify your runtime by only needing one. It's been written for you: http://hackage.haskell.org/package/base-4.11.1.0/docs/Control-Arrow.html#g:5 http://hackage.haskell.org/package/base-4.11.1.0/docs/Contro... Please consider that the incredible strength of ADTs is that you can just write generic code to combine those things without resorting to the crabtastic pattern matches that you're proposing, AND avoid the ugly implicitly unpredictably ordered world that you're describing. Heck, we don't even need to resort to combinators for this. It's just you making up a complex case by saying, "What if I hide half of example 2 by handwaving it?" You still are going to have a case1/case2 decomposition in your second version. It's universally worse.
- jstimpfle 8y ago
  I don't see a pattern match anywhere in my example. And it's not in the slightest "unpredictably ordered" and has much simpler control flow. > You still are going to have a case1/case2 decomposition in your second version. It's universally worse. No. There is simply none. The producer makes them as two, and the two are processed independently. There is no problem if you don't make one. If you had ever tried the approach I propose out you'd know the flexibility that comes with it. (I'm currently working on language tooling, and there it is absolutely the case that it's very easy do without ASTs, and process the cases independently). The only thing I "handwaved away" is the calls: s1, s2 = process_data() process_case1(s1) process_case2(s2) There is no pattern match there, sorry to disappoint. Where I need to relate the individual items of s1 and s2 in a common ordering, I've had great success with this approach: s1, s2, forks = process_data() process_case1(s1) process_case2(s2) process_forks(forks) There, "forks" does contain an ADT of pointers into s1 and s2. But it contains nothing more, so it's very minimalistic. The advantage from the first case is still there: Most code profits from the homogeneity of s1 and s2, and from the actual, physical decoupling of the cases, and so much pattern matching just goes away. (Btw. I'm not interested in fancy FP constructs. You will not convince me that you can process ADTs without pattern matches. You might be able to avoid writing them down in the source code, but I care about what actually happens on the hardware.) (And actually I think avoiding to write down the pattern match in the source code and only generating it magically with higher level constructs is a bad, bad idea. It has poor readability. And it encourages the mindset "let's just do the worst shit on the actual machine. We can avoid writing it in the source code, and we don't care what happens on the machine and how slow it is. And we're too lazy to think about actual solutions that actually avoid real work".)
  - KirinDave 8y ago
    > Where I need to relate the individual items of s1 and s2 in a common ordering, I've had great success with this approach: Sorry I misunderstood your use of the phrase "two streams." In my world multiple streams effectively severs causality between the streams. Or did I? You wrote: def process_case1(stream): for item in stream: do_complex_stuff_1(item) def process_case2(stream): for item in stream: do_complex_stuff_2(item) If this genuinely relates to your example above then the only possible consistent ordering without any async/multithreading is to exhaust stream1, then process your errors. If you don't do that and the implication is that the streams won't close on errors but rather continue to emit, then you really can't guarantee any ordering to the processing. Given this inconsistency, I'm going to continue the post with your intent of causality being preserved and assuming assuming that process_data() offers the next value (or null) and we're calling these in a loop. Because otherwise you have a whole heap of problems and your example doesn't even faintly resemble any subject matter from the post, or what most folks were talking about here. And if you believe that it doesn't destroy causality, you're either writing a lot of code to recombine your streams in lockstep (in which case, splitting is quite the extreme step to avoid matching) or you just don't understand the implications of stream semantics in this case. > There is no pattern match there, sorry to disappoint. However you provided a destructuring pattern match in your code! s1, s2 = process_data() Your process data is pattern matched on a generic product there. Sorry to disappoint. The actual type of the value is: (Either Null S1Type, Either Null S2Type) (And I assume it must be an `Either Null a` because we're discussing a case where either case1 or case2 will exist, so there's no valid value for one when the other exists.) This is sort of why folks who know about ADTs roll their eyes at the sizzling invective about their complexity. You often end up using the same solutions, you just don't name them. It seems like the naming itself is objectionable. You've written about 4 posts I can see here about how much you hate ADTs, but then you're happy for an implicit `Either Null a` on every value, no opting out, and `a,b = f(x)` is fine because the name of the ADT is a punctuation mark, I guess? As for the first example, Pattern Matches as Case statements are related in that a pattern match is a hyperactive case statement. So when you have one, you have the other to a varying degree. > (Btw. I'm not interested in fancy FP constructs. You will not convince me that you can process ADTs without pattern matches. You might be able to avoid writing them down in the source code, but I care about what actually happens on the hardware.) You seem to be making a huge deal out of pointer chasing. However, you're also suggesting a detour through the function contexts of process_case1 and process_case2 just to offer a null in one case. Your plan seems a bit unfair, while we're on the subject. You're arguing that a pattern match is "heavy overhead", but you're completely ignoring the actual use case of "railway programming": to make a single clean codepath obvious and localize error handling at the end. But the thing you've offered doesn't really do that. You're processing S1 and S2 (which I'll rename E1 for error), but what happens when I want to compose it? Extending the metaphor, your new code might look like: s1, e1 = process_data() process_ecase(e1) // If you don't put an if( s1 != null ) here you're going // to have to do a ton of function calls AND guarantee // your handlers check their args for null. if( s1 != null ) { s2, e2 = process_case(s1) process_ecase(e2) if( s2 != null ) { // ... } } You could perhaps change the ordering of these ops (or push the null checks into your code which has a bunch of problems). And most of all, your compiler is totally powerless to help you work out if you have an error or not. You might also instead try to embed the dispatch to the next step in the def blocks of each individual step (an approach that could never work with the stream example, as you need to define your stream network (or at least a global namespace for it) before streaming events through it. > And it encourages the mindset "let's just do the worst shit on the actual machine. Okay, your example extended above. The Haskell-style of this combinator pattern (which DOES require all errors have the same base type, let me be transparent about that) is: -- Assuming the steps are: a -> Either Error b let pipeline = process_data >>= step2 >>= step3 pipeline () `catchError` errorHandler The errorHandler is just a normal function taking your shared error type. Each step is just a function that returns an Either Error or its value. The values can vary so long as the types between steps line up (and the compiler checks this in F#, ML and Haskell). It's easy to read, too. It's easy to extend, and it's predictable, and it's made from very simple decorated functions. We can even safely make it interact with exceptions, if we need to, without overly complicating the callers. > And it encourages the mindset "let's just do the worst shit on the actual machine. We can avoid writing it in the source code What's completely confusing to me about your claim here is that you're doing dynamic dispatches and perhaps excess function calls, demanding your process functions do null checks, you have no clear looping construct, and your compiler has 0 opportunity for any sort of inlining but the most naive, cache busting type. But you're essentially dunking on what becomes BNE statements at the machine code level. At runtime, all the types are erased in the best case or preserved in exactly the same way as any normal object with a dispatch table would. The actual cost is, in most cases, indistinguishable from your if-statement rails. I actually think you like lots of elements of the FP style, you just seem to dislike the names. Perhaps if you could hold your nose with a clothespin and play through the stink of math-ish names, you might find more you like?