> Where I need to relate the individual items of s1 and s2 in a common ordering, I've had great success with this approach:
Sorry I misunderstood your use of the phrase "two streams." In my world multiple streams effectively severs causality between the streams. Or did I? You wrote:
def process_case1(stream):
for item in stream:
do_complex_stuff_1(item)
def process_case2(stream):
for item in stream:
do_complex_stuff_2(item)
If this genuinely relates to your example above then the only possible consistent ordering without any async/multithreading is to exhaust stream1, then process your errors. If you don't do that and the implication is that the streams won't close on errors but rather continue to emit, then you really can't guarantee any ordering to the processing.
Given this inconsistency, I'm going to continue the post with your intent of causality being preserved and assuming assuming that process_data() offers the next value (or null) and we're calling these in a loop. Because otherwise you have a whole heap of problems and your example doesn't even faintly resemble any subject matter from the post, or what most folks were talking about here. And if you believe that it doesn't destroy causality, you're either writing a lot of code to recombine your streams in lockstep (in which case, splitting is quite the extreme step to avoid matching) or you just don't understand the implications of stream semantics in this case.
> There is no pattern match there, sorry to disappoint.
However you provided a destructuring pattern match in your code!
s1, s2 = process_data()
Your process data is pattern matched on a generic product there. Sorry to disappoint. The actual type of the value is:
(Either Null S1Type, Either Null S2Type)
(And I assume it must be an `Either Null a` because we're discussing a case where either case1 or case2 will exist, so there's no valid value for one when the other exists.)
This is sort of why folks who know about ADTs roll their eyes at the sizzling invective about their complexity. You often end up using the same solutions, you just don't name them. It seems like the naming itself is objectionable. You've written about 4 posts I can see here about how much you hate ADTs, but then you're happy for an implicit `Either Null a` on every value, no opting out, and `a,b = f(x)` is fine because the name of the ADT is a punctuation mark, I guess?
As for the first example, Pattern Matches as Case statements are related in that a pattern match is a hyperactive case statement. So when you have one, you have the other to a varying degree.
> (Btw. I'm not interested in fancy FP constructs. You will not convince me that you can process ADTs without pattern matches. You might be able to avoid writing them down in the source code, but I care about what actually happens on the hardware.)
You seem to be making a huge deal out of pointer chasing. However, you're also suggesting a detour through the function contexts of process_case1 and process_case2 just to offer a null in one case.
Your plan seems a bit unfair, while we're on the subject. You're arguing that a pattern match is "heavy overhead", but you're completely ignoring the actual use case of "railway programming": to make a single clean codepath obvious and localize error handling at the end. But the thing you've offered doesn't really do that.
You're processing S1 and S2 (which I'll rename E1 for error), but what happens when I want to compose it? Extending the metaphor, your new code might look like:
s1, e1 = process_data()
process_ecase(e1)
// If you don't put an if( s1 != null ) here you're going
// to have to do a ton of function calls AND guarantee
// your handlers check their args for null.
if( s1 != null ) {
s2, e2 = process_case(s1)
process_ecase(e2)
if( s2 != null ) {
// ...
}
}
You could perhaps change the ordering of these ops (or push the null checks into your code which has a bunch of problems). And most of all, your compiler is totally powerless to help you work out if you have an error or not. You might also instead try to embed the dispatch to the next step in the def blocks of each individual step (an approach that could never work with the stream example, as you need to define your stream network (or at least a global namespace for it) before streaming events through it.
> And it encourages the mindset "let's just do the worst shit on the actual machine.
Okay, your example extended above. The Haskell-style of this combinator pattern (which DOES require all errors have the same base type, let me be transparent about that) is:
-- Assuming the steps are: a -> Either Error b
let pipeline = process_data >>= step2 >>= step3
pipeline () `catchError` errorHandler
The errorHandler is just a normal function taking your shared error type. Each step is just a function that returns an Either Error or its value. The values can vary so long as the types between steps line up (and the compiler checks this in F#, ML and Haskell).
It's easy to read, too. It's easy to extend, and it's predictable, and it's made from very simple decorated functions. We can even safely make it interact with exceptions, if we need to, without overly complicating the callers.
> And it encourages the mindset "let's just do the worst shit on the actual machine. We can avoid writing it in the source code
What's completely confusing to me about your claim here is that you're doing dynamic dispatches and perhaps excess function calls, demanding your process functions do null checks, you have no clear looping construct, and your compiler has 0 opportunity for any sort of inlining but the most naive, cache busting type. But you're essentially dunking on what becomes BNE statements at the machine code level. At runtime, all the types are erased in the best case or preserved in exactly the same way as any normal object with a dispatch table would. The actual cost is, in most cases, indistinguishable from your if-statement rails.
I actually think you like lots of elements of the FP style, you just seem to dislike the names. Perhaps if you could hold your nose with a clothespin and play through the stink of math-ish names, you might find more you like?
I think you're assuming I'm less experienced than I actually am!
> If this genuinely relates to your example above then the only possible consistent ordering without any async/multithreading is to exhaust stream1, then process your errors. [...]
Unless we know there are dependencies between the processing of s1 and s2 (and we don't; we're talking very abstract here) I'm totally unconcerned about the processing order, and I even stated explicitly that in the simplest case they're just append-only arrays. And the processing of the items in s1 and s2 can be done in any order, but why not just do it in a single thread, first process "data" completely, then "s1", then "s2". In many cases of course, it will be possible to run in 3 parallel threads, where the additional two process s1 and s2 as they are emitted from the first thread. But that's just a small optimization and a lot of complexity and not relevant here.
> Your process data is pattern matched on a generic product there. Sorry to disappoint. The actual type of the value is:
You're purposefully twisting words here. And no, that's not "the type", because I was (obviously) just using straight forward pseudocode whose semantics I assumed would be intuitively clear to everyone. That's not the kind of pattern matching we'd been talking about. In my code s1 and s2 are simply two known variables holding the streams. That's something different than a single variable holding either of two distinct cases. I assume you know that and just want to sell the FP cool-aid or troll me; please refrain from this style of discussion.
> This is sort of why folks who know about ADTs roll their eyes at the sizzling invective about their complexity. You often end up using the same solutions, you just don't name them. It seems like the naming itself is objectionable. You've written about 4 posts I can see here about how much you hate ADTs, but then you're happy for an implicit `Either Null a` on every value, no opting out, and `a,b = f(x)` is fine because the name of the ADT is a punctuation mark, I guess?
There is so much hair-pulling in here and what you say is simply wrong. I was just returning two friggin' variables. And why are you talking about "Null"? That's of no interest. There is no use in my pseudocode for any kind of thing resembling what's called "Null" in any programming language. As a Haskell enthusiast you should know that there is always a canonical stream/array, and that's the empty one. Why are you haunting me with Nulls, that's not fair, I don't want them, and I can do without them. (And just to be clear, I don't do OOP either; I think it's at least as bad as FP)
> What's completely confusing to me about your claim here is that you're doing dynamic dispatches and perhaps excess function calls,
No I'm frigging not. I don't get it why you're talking of dynamic dispatches now. What I have given is some pseudo code of the simplest kind, that easily translates to any procedural programming language, including assembly code. Why do you make it seem like it was something more complicated? You must be deliberately hair-pulling just to win a stupid argument. I'm not doing any "virtual object-oriented blah blah" or any kind of types craziness.
s1 is a stream (or let's just say (again), an "array", to hopefully prevent any misunderstandings) of homogeneous things. Let's just say all items in s1 have type S1 and all items of s2 have type S2. It's the simplest kind of problem.
> -- Assuming the steps are: [..]
Congrats. I learned Haskell once, too. Until at some point it became clear that it didn't bring anything practical to the table and only complicates program designs. This "uniform error handling" idea (actually it seems to be more of a necessity/accident) just makes for bad code. Don't do that. Just do the first processing step, handle errors accordingly. Then do the second step, and handle errors accordingly. The second step in general requires different error handling than the first, (and often the only thing they have in common is that they might "die"), and there is no need and only disadvantages in restricting them to have uniform error handling.
What's wrong with my procedural pseudo code? Fill in the required error handling between the steps as soon as you have a concrete instance for my abstract example.
> dynamic dispatches
Nope
> excess function calls
What's that?
> demanding your process functions do null checks
No, I don't have any "Nulls" of any kind, since my data is simple and normalized.
> no clear looping construct
I used a pseudo-code for loop. Every 1-month beginner to programming has an immediate grasp of that. I never heard anyone complain that was somehow not "clear". It's plain, procedural code. You're trolling me so hard it hurts.
> I actually think you like lots of elements of the FP style, you just seem to dislike the names. Perhaps if you could hold your nose with a clothespin and play through the stink of math-ish names, you might find more you like?
I've been there and I know to use FP-ish constructs when they make sense (very rarely. Much easier to restrict to procedural altogether).