5 ms·

Flow-based programming and Erlang-style message passing

69 points by samuell 11y ago

jerf 11y ago
"E.g. backpressure is a thing that is not easily supported implicitly in Erlang/Elixir in the same way as in FBP languages where the bounded-buffer channels provide implicit backpressure by blocking the sends on the outport they are connected to when the buffer is full." This is one thing that I've learned from using both Go and Erlang now... I like asynchronous message passing as a better default, but if I had to choose between the core language supporting either synchronous message passing with Go-like channels or async message passing like Erlang, I'll take the Go case, because it's way easier to implement async on top of what Go provides than it is to implement sync on top of what Erlang provides. And while sync may be the exception, when you want it, you really want it, and Erlang doesn't offer anything for it that I can see, at least at user level. As the phrasing above may suggest, though, what I'd like is both. Of course sync can only be used in the local OS process and async works across networks... that's just something you'd have to work with as a fundamental constraint. That's life. Even with a conventional manifest type system (i.e., C++/Java), if you treat the two types of communication as different types, you won't mix the two up. If were going to try to do flow-based with Erlang, I'd look at moving code to the data rather than the other way around. That has its own negative tradeoffs too, but I think in the end they'd be less bad than trying to send data through Erlang's system. As good as Erlang is at message passing, it's very not designed for pushing lots of data repeatedly through the messaging system. If you use granular flow constructs than involve pushing data through a couple dozen flow elements and you model those as processes, you're going to get hammered on all the copying that implies. (For anything other than binaries, but what kind of flows do you have that involve sending a binary through it such than none of the flow elements modify it?) Though I'd say in the end, even with moving code around, the conclusion stands and Erlang isn't really good at flow-based itself.
- samuell 11y ago
  That's an interesting way to put it! And yes, I agree one would like both, optimally. Btw, Robert Aloi linked some kind of simpler back-pressure implementation available already today in Erlang, on G+: https://plus.google.com/u/0/+SamuelLampa/posts/Q1XLTpk4KQC https://plus.google.com/u/0/+SamuelLampa/posts/Q1XLTpk4KQC ... though as he says, might not be sufficient for whats typically needed in flow-based, or CSP-style programs.
  - samuell 11y ago
    And yea, I very loosely treat Go as more or less a Flow-based programming system ... especially after I realized one can write more or less FBP network semantics in vanilla, framework-less Go [1]. [1] http://blog.gopheracademy.com/composable-pipelines-pattern http://blog.gopheracademy.com/composable-pipelines-pattern :)
- e_proxus 11y ago
  What do you mean by sync being hard to implement in Erlang? Simple synchronous requests can easily be implemented: Server ! {request, Request}, receive {reply, Reply} -> {ok, Reply} end. This is also the default way when using the built-in gen_server library.
  - jerf 11y ago
    In Go, sending a message on a channel is actually a sync point between the sender and the receiver; a channel blocks until it has both. (Buffered channels may not necessarily block, but you have to treat them as if they do anyhow in general.) Thus, the send itself is synchronous, not just the fact that we're "waiting until we get an answer". If the sender proceeds past the line that sends on a channel, we know a receiver has received it. I don't know of an efficient way to emulate this in Erlang. Suppose you have a pool of 10 undistinguished server processes in Erlang and you just want "a" process to answer. If you use Go channels, you have all 10 listen on a channel, and you automatically get the behavior where as long as one is available, that one will pick up the request. You also automatically get backpressure; if none are available, the attempt to send will hang. (There are ways of turning that into a timeout if you need to.) In Erlang, you could try to create an 11th coordinator process, but you'll create a lot of additional traffic and take a latency hit on every request. (Plus, some of the naive ways you might implement that will get you in further performance trouble with mailbox scanning. This is going to be a non-trivial thing to build correctly.) Or you could just randomly pick and send, but if you pick one that's currently answering a slow request while others are available, you get spurious latency. It hasn't been a killer in my system, but it's been an annoyance. And to be clear, let me reiterate that having using both systems, I actually still like Erlang's behavior as a better default. It's really hard to deadlock Erlang... it's much easier to deadlock Go. The async message passing is, in my experience, much easier to use correctly without much thinking. But it would be nice to also have a "channel" in Erlang, despite the limitations it would have to have with not being able to go across nodes. In fact, technically speaking, subject to that limitation (and there are already a couple of other functions limited to the current node), there's no reason I could see why this couldn't be added to Erlang. See for instance: https://github.com/inaka/worker_pool https://github.com/inaka/worker_pool , particularly the section discussion "strategy", and the discussion about "available_worker" and its discussion on performance implications, and its specific references to the other tricky edge cases to consider. Of course, programming languages being programming languages, someone else can do the work and you can just use it, but in Go, this behavior is quite trivial. (By contrast, in Go, proper asynchronous messaging in the Erlang style is a challenge. Whacking together a "slice" and some "messages" isn't that hard, but you've got other edge cases to consider... to say nothing of network transparency!)
- rozap 11y ago
  Yes, you don't get sync for free when using the ! operator in Erlang, but that's why you use the OTP constructs like handle_call (sync) and handle_cast (async) to model that sort of behavior... Throwing the ! operator around haphazardly can be useful, but generally it's a code smell, as things can usually be encapsulated more effectively using OTP constructs. Edit: saw you addressed this in another comment.
- srean 11y ago
  Curious. Have you used systems that provide fibres ? The idea is that blocks of code exchange control whenever a fibre is written to or read through. I have not used Erlang or Go so am far from being an expert in these things. You might find this interesting http://felix-lang.org/share/src/web/tut/fibres_index.fdoc http://felix-lang.org/share/src/web/tut/fibres_index.fdoc the site is undergoing reorganization so expect (and pardon) broken links. BTW I am not the one who develops Felix.
  - samuell 11y ago
    I wonder if fibres are not quite very similar to the CSP (communicating sequential processes) paradigm of Go..
    - srean 11y ago
      Indeed they are. Compared to Felix, Go is quite new though.
murbard2 11y ago
An interesting approach is join calculus. This tutorial builds on the same "chemical" metaphor used in the article https://sites.google.com/site/winitzki/tutorial-on-join-calculus-and-its-implementation-in-ocaml-jocaml https://sites.google.com/site/winitzki/tutorial-on-join-calc...
- samuell 11y ago
  Thanks, will check this!
tunesmith 11y ago
Dealing with back pressure in general is a huge subject. How do you deal with a producer that produces more than a consumer can handle? I had just finished reading an excellent interview with the collaborators of the Reactive Streams API that talked about wrestling with this: https://medium.com/@viktorklang/reactive-streams-1-0-0-interview-faaca2c00bec https://medium.com/@viktorklang/reactive-streams-1-0-0-inter...
- josevalim 11y ago
  Indeed! I have been following the efforts behind Reactive Streams with a special focus on how they are tackling back-pressure.

69 points by samuell 11y ago

jerf 11y ago

"E.g. backpressure is a thing that is not easily supported implicitly in Erlang/Elixir in the same way as in FBP languages where the bounded-buffer channels provide implicit backpressure by blocking the sends on the outport they are connected to when the buffer is full." This is one thing that I've learned from using both Go and Erlang now... I like asynchronous message passing as a better default, but if I had to choose between the core language supporting either synchronous message passing with Go-like channels or async message passing like Erlang, I'll take the Go case, because it's way easier to implement async on top of what Go provides than it is to implement sync on top of what Erlang provides. And while sync may be the exception, when you want it, you really want it, and Erlang doesn't offer anything for it that I can see, at least at user level. As the phrasing above may suggest, though, what I'd like is both. Of course sync can only be used in the local OS process and async works across networks... that's just something you'd have to work with as a fundamental constraint. That's life. Even with a conventional manifest type system (i.e., C++/Java), if you treat the two types of communication as different types, you won't mix the two up. If were going to try to do flow-based with Erlang, I'd look at moving code to the data rather than the other way around. That has its own negative tradeoffs too, but I think in the end they'd be less bad than trying to send data through Erlang's system. As good as Erlang is at message passing, it's very not designed for pushing lots of data repeatedly through the messaging system. If you use granular flow constructs than involve pushing data through a couple dozen flow elements and you model those as processes, you're going to get hammered on all the copying that implies. (For anything other than binaries, but what kind of flows do you have that involve sending a binary through it such than none of the flow elements modify it?) Though I'd say in the end, even with moving code around, the conclusion stands and Erlang isn't really good at flow-based itself.

samuell 11y ago
That's an interesting way to put it! And yes, I agree one would like both, optimally. Btw, Robert Aloi linked some kind of simpler back-pressure implementation available already today in Erlang, on G+: https://plus.google.com/u/0/+SamuelLampa/posts/Q1XLTpk4KQC https://plus.google.com/u/0/+SamuelLampa/posts/Q1XLTpk4KQC ... though as he says, might not be sufficient for whats typically needed in flow-based, or CSP-style programs.
- samuell 11y ago
  And yea, I very loosely treat Go as more or less a Flow-based programming system ... especially after I realized one can write more or less FBP network semantics in vanilla, framework-less Go [1]. [1] http://blog.gopheracademy.com/composable-pipelines-pattern http://blog.gopheracademy.com/composable-pipelines-pattern :)
e_proxus 11y ago
What do you mean by sync being hard to implement in Erlang? Simple synchronous requests can easily be implemented: Server ! {request, Request}, receive {reply, Reply} -> {ok, Reply} end. This is also the default way when using the built-in gen_server library.
- jerf 11y ago
  In Go, sending a message on a channel is actually a sync point between the sender and the receiver; a channel blocks until it has both. (Buffered channels may not necessarily block, but you have to treat them as if they do anyhow in general.) Thus, the send itself is synchronous, not just the fact that we're "waiting until we get an answer". If the sender proceeds past the line that sends on a channel, we know a receiver has received it. I don't know of an efficient way to emulate this in Erlang. Suppose you have a pool of 10 undistinguished server processes in Erlang and you just want "a" process to answer. If you use Go channels, you have all 10 listen on a channel, and you automatically get the behavior where as long as one is available, that one will pick up the request. You also automatically get backpressure; if none are available, the attempt to send will hang. (There are ways of turning that into a timeout if you need to.) In Erlang, you could try to create an 11th coordinator process, but you'll create a lot of additional traffic and take a latency hit on every request. (Plus, some of the naive ways you might implement that will get you in further performance trouble with mailbox scanning. This is going to be a non-trivial thing to build correctly.) Or you could just randomly pick and send, but if you pick one that's currently answering a slow request while others are available, you get spurious latency. It hasn't been a killer in my system, but it's been an annoyance. And to be clear, let me reiterate that having using both systems, I actually still like Erlang's behavior as a better default. It's really hard to deadlock Erlang... it's much easier to deadlock Go. The async message passing is, in my experience, much easier to use correctly without much thinking. But it would be nice to also have a "channel" in Erlang, despite the limitations it would have to have with not being able to go across nodes. In fact, technically speaking, subject to that limitation (and there are already a couple of other functions limited to the current node), there's no reason I could see why this couldn't be added to Erlang. See for instance: https://github.com/inaka/worker_pool https://github.com/inaka/worker_pool , particularly the section discussion "strategy", and the discussion about "available_worker" and its discussion on performance implications, and its specific references to the other tricky edge cases to consider. Of course, programming languages being programming languages, someone else can do the work and you can just use it, but in Go, this behavior is quite trivial. (By contrast, in Go, proper asynchronous messaging in the Erlang style is a challenge. Whacking together a "slice" and some "messages" isn't that hard, but you've got other edge cases to consider... to say nothing of network transparency!)
rozap 11y ago
Yes, you don't get sync for free when using the ! operator in Erlang, but that's why you use the OTP constructs like handle_call (sync) and handle_cast (async) to model that sort of behavior... Throwing the ! operator around haphazardly can be useful, but generally it's a code smell, as things can usually be encapsulated more effectively using OTP constructs. Edit: saw you addressed this in another comment.
srean 11y ago
Curious. Have you used systems that provide fibres ? The idea is that blocks of code exchange control whenever a fibre is written to or read through. I have not used Erlang or Go so am far from being an expert in these things. You might find this interesting http://felix-lang.org/share/src/web/tut/fibres_index.fdoc http://felix-lang.org/share/src/web/tut/fibres_index.fdoc the site is undergoing reorganization so expect (and pardon) broken links. BTW I am not the one who develops Felix.
- samuell 11y ago
  I wonder if fibres are not quite very similar to the CSP (communicating sequential processes) paradigm of Go..
  - srean 11y ago
    Indeed they are. Compared to Felix, Go is quite new though.

murbard2 11y ago

An interesting approach is join calculus. This tutorial builds on the same "chemical" metaphor used in the article https://sites.google.com/site/winitzki/tutorial-on-join-calculus-and-its-implementation-in-ocaml-jocaml https://sites.google.com/site/winitzki/tutorial-on-join-calc...

samuell 11y ago
Thanks, will check this!

tunesmith 11y ago

Dealing with back pressure in general is a huge subject. How do you deal with a producer that produces more than a consumer can handle? I had just finished reading an excellent interview with the collaborators of the Reactive Streams API that talked about wrestling with this: https://medium.com/@viktorklang/reactive-streams-1-0-0-interview-faaca2c00bec https://medium.com/@viktorklang/reactive-streams-1-0-0-inter...

josevalim 11y ago
Indeed! I have been following the efforts behind Reactive Streams with a special focus on how they are tackling back-pressure.