5 ms·

The core of the FBP principles are the holy grail of true componentized function architecture. Instead of losing yourself in ever more complex syntax convoluti

The core of the FBP principles are the holy grail of true componentized function architecture. Instead of losing yourself in ever more complex syntax convolutions as has happened in a lot of functional programming, you make the components (even long running stateful ones) self contained with ports for data input and output as the sole means of communicating with them, over buffered channels to allow asynchronous computation, and most importantly keep the network definition separate. Just this idea in itself is just brilliant. Hats off to Mr Morrison for that! It allows to decouple complex software into reusable components, without clever FP syntax. (Though, FP is perfect for implementing the components themselves. It just doesn't really scale all too well for whole program architecture, in my experience). One point to note: The visual component of many fbp systems is completely optional, as is the idea of using novel DSLs. You can as well define your networks and components in pure code. See GoFlow https://github.com/trustmaster/goflow https://github.com/trustmaster/goflow) and my own little experiment FlowBase (https://flowbase.org https://flowbase.org) for examples of that, in Go. I successfully built a rather complex little app to convert from Semantic web RDF format to (semantic) mediawiki XML dump format, in two weeks straight, of linear development time: for each component (of ca 7), implement, test, go to the next component (See: https://github.com/rdfio/rdf2smw https://github.com/rdfio/rdf2smw) The same implementation in procedural PHP took months, and still doesn't have all bugs and strange behaviours filed out.

the_duke 6y ago
> you make the components self contained with ports for data input and output as the sole means of communicating with them, over buffered channels to allow asynchronous computation This concept sounds exactly like actor systems like Erlang/OTP and Akka, only with a different set of terminology. The submitted site and your comment don't mention those anywhere. Are there appreciable differences between actor systems and FBP?
- megameter 6y ago
  FBP is most analogous to the unit record machines of yore: https://en.m.wikipedia.org/wiki/Unit_record_equipment https://en.m.wikipedia.org/wiki/Unit_record_equipment What you are programming is the graph of how these machines connect and flow data(keypunch cards) through each other, while the machines themselves and their function are abstracted out. Data does not stay "at rest", it's presumed to reach a terminating point where it exits the graph. A buildup of unprocessed data in a machine's inbox results in an overflow. It's a very useful model for making a debuggable asynchronous system since it imposes static constraints everywhere that you can map to your real hardware resources, versus the emphasis on dynamism seen in the actor model(actors hold private data, modify their state, create new actors - all explicitly disallowed in FBP).
- samuell 6y ago
  Actor systems differ in their means of communication. While actor systems use the actors as the main units that you connect, with fire and forget messaging to each actor's inbox, FBP proposes communication between named ports over channels with bounded buffers, very much like the CSP implementation in Go (just that you have long running components as in actor systems - something that can easily be implemented in Go with structs with go channels in fields for the ports, as shown in the FlowBase library I mentioned in another comment). The FBP model provides implicit backpressure, because of the bounded buffers on the channels. The actor model on the other hand is more loosely coupled and flexible. This actually means FBP systems are slightly more optimal for efficient within-one-node, in-memory parallellisation, whereas actor systems shine more for distributed systems. They shine on different scales, in other words. I wrote a post many years ago on this, that seems to have aligned with the experience of a number of people, based on the reactions in the comments: https://rillabs.com/posts/flowbased-vs-erlang-message-passing https://rillabs.com/posts/flowbased-vs-erlang-message-passin...
  - Nullabillity 6y ago
    Akka Streams adds named ports and defined backpressure, as well as a graphical-ish DSL for connecting everything.[0] [0]: https://doc.akka.io/docs/akka/current/stream/stream-graphs.html https://doc.akka.io/docs/akka/current/stream/stream-graphs.h...
    - samuell 6y ago
      That is nice to see! I was evaluating Akka a bit some years ago.
      - rad_gruchalski 6y ago
        AkkaHttp is written on top of Streams. There are many more things written using it but Http is a good example to learn from.
    - jghn 6y ago
      It is worth noting that Akka Streams is an implementation of the Reactive Streams [0] spec, using Akka [0]: https://www.reactive-streams.org https://www.reactive-streams.org
dustingetz 6y ago
FP decouples the AST (symbolic functions with input ports and an output port) from the evaluation context, which might be async, or sync, backpressured, stateful, exceptional, incremental/reactive ... mix-n-match whatever behaviors you want, all for the same abstract AST
- samuell 6y ago
  I actually didn't want to make this too much of an FP-bashing comment, as I find great use of many FP concepts, daily. I have yet to see an FP concept for composition though, that is as simple and generic in its implementation, as the FBP principles. I have sometimes thought FBP networks provide roughly the same function (pun not intended) as a monad, although I never seem to fully grasp what a monad is, so I can't tell for sure :o)
  - dustingetz 6y ago
    Yes, dataflow with dynamic topologies (self-adjusting computation) forms a monad. If the "flowchart" is merely static, that forms an applicative. https://blog.janestreet.com/breaking-down-frp/ https://blog.janestreet.com/breaking-down-frp/ In my opinion, understanding the monad is essential to understanding how to fully separate an AST from its evaluation behavior. That is basically what a monad is, the monad is the computational structure that does that.
    - samuell 6y ago
      Thanks, that clarifies a lot!
- dmbarbour 6y ago
  I think most FP languages don't decouple AST from evaluation context. E.g. I cannot perform abstract interpretation on a function's AST, nor rewrite a function's AST for incremental computation, nor perform explicit partial evaluation during function composition by composing the ASTs. I only can access opaque `Input -> Output`. Also, FP algorithms developed for one evaluation context cannot easily be ported to another. They are implicitly entangled with assumptions. For example, I cannot evaluate a lazy algorithm in an eager evaluation context without paying a huge price. Adding back-pressure for list processing where a program was not designed for it would easily result in deadlock. Adding exceptions without adding unwind (or bracket, try/finally, etc.) to the existing program expressions will easily result in buggy code. I think your assertion might be true for some specific integrations of FP nodes into FBP. Is this what you mean?
codetrotter 6y ago
Your flowbase.org domain redirects to a different domain. I thought that perhaps you had a typo in the URL or forgot to renew your registration of the domain, but upon a closer look I think it’s probably just a misconfiguration in your webserver config causing it to redirect to another one of your projects. My guess is that you forgot to put the definition of the domain sans www in your webserver config. If my guess is correct then this link should work: https://www.flowbase.org https://www.flowbase.org Edit: doesn’t work either. Perhaps you don’t have a TLS cert for that domain? In which case maybe this link will work instead: http://www.flowbase.org http://www.flowbase.org Edit 2: Without https it works. And like the person responding to this comment said it redirects to the GH repo.
- wizzwizz4 6y ago
  The link's trying to point you here: https://github.com/flowbase/flowbase https://github.com/flowbase/flowbase
  - samuell 6y ago
    Thank you!
- samuell 6y ago
  Ouch, sorry, and thanks! Wrote on mobile on the go and didn't manage to double check the link.
FigmentEngine 6y ago
Sounds like XProc 3.0 https://xproc.org/ https://xproc.org/
megameter 6y ago
Having studied and implemented FBP systems in the past, one major takeaway I've gleaned is that most automation problems start off as a linear sequence of processes, and so the branching-graph of FBP looks unwieldy and superfluous. But this is deceptive; you probably don't want to have a huge number of branches in the design, but you will want them as an optimization step or a way to combine inputs. So it's useful to design with FBP in mind but with a linear interface as the entry point. Another aspect of this is that FBP graphs are static but you may have a need to reconfigure them frequently; that is, you may want to have a graph compilation step drawn from a source language, rather than manually wiring it up. A way of making that graph compilation more than a syntax is to include a formal constraint solver: Excel, for example, flows the data after determining a solution for how cells relate to each other. The power of the spreadsheet paradigm really lies in these combinations of concepts. Lastly, there isn't really magic in the algorithmic/implementation aspects of FBP. It grew out of 1960's mainframe types of problems, and so it can be implemented in a low level way with static pools of memory and pieces of assembly code. But it remains conceptually just as relevant to today's massive distributed systems.
goliatone 6y ago
The flowbase.org link is redirecting me to a polish website, not sure what’s going on there
- deleted 6y ago
  [deleted]
- samuell 6y ago
  Sorry about this, I was typing in a hurry. The links should redirect to https://github.com/flowbase/flowbase https://github.com/flowbase/flowbase but seems to be misconfigured for https. Looking into it.
vitalhead 6y ago
Visual aspect of FBP is also brilliant and helps to navigate/comprehend inherently parallel execution flow. The same complexity network expressed in linear text would require more effort to grasp. And it allows you to leverage your brain's visual capacity which is quite powerful - humans are visual creatures (https://www.seyens.com/humans-are-visual-creatures https://www.seyens.com/humans-are-visual-creatures). For example in Excel the update dependency network is completely hidden. The maintenance of complex Excel models is really hard. For same reasons we start looking at FBP and functional reactive programing to simplify the design and maintenance of complex interactive UIs. End up implementing Kelp (https://kelp.app https://kelp.app) with visual FBP editor and reactive framework (https://kefirjs.github.io/kefir/ https://kefirjs.github.io/kefir/).

the_duke 6y ago

> you make the components self contained with ports for data input and output as the sole means of communicating with them, over buffered channels to allow asynchronous computation This concept sounds exactly like actor systems like Erlang/OTP and Akka, only with a different set of terminology. The submitted site and your comment don't mention those anywhere. Are there appreciable differences between actor systems and FBP?

megameter 6y ago
FBP is most analogous to the unit record machines of yore: https://en.m.wikipedia.org/wiki/Unit_record_equipment https://en.m.wikipedia.org/wiki/Unit_record_equipment What you are programming is the graph of how these machines connect and flow data(keypunch cards) through each other, while the machines themselves and their function are abstracted out. Data does not stay "at rest", it's presumed to reach a terminating point where it exits the graph. A buildup of unprocessed data in a machine's inbox results in an overflow. It's a very useful model for making a debuggable asynchronous system since it imposes static constraints everywhere that you can map to your real hardware resources, versus the emphasis on dynamism seen in the actor model(actors hold private data, modify their state, create new actors - all explicitly disallowed in FBP).
samuell 6y ago
Actor systems differ in their means of communication. While actor systems use the actors as the main units that you connect, with fire and forget messaging to each actor's inbox, FBP proposes communication between named ports over channels with bounded buffers, very much like the CSP implementation in Go (just that you have long running components as in actor systems - something that can easily be implemented in Go with structs with go channels in fields for the ports, as shown in the FlowBase library I mentioned in another comment). The FBP model provides implicit backpressure, because of the bounded buffers on the channels. The actor model on the other hand is more loosely coupled and flexible. This actually means FBP systems are slightly more optimal for efficient within-one-node, in-memory parallellisation, whereas actor systems shine more for distributed systems. They shine on different scales, in other words. I wrote a post many years ago on this, that seems to have aligned with the experience of a number of people, based on the reactions in the comments: https://rillabs.com/posts/flowbased-vs-erlang-message-passing https://rillabs.com/posts/flowbased-vs-erlang-message-passin...
- Nullabillity 6y ago
  Akka Streams adds named ports and defined backpressure, as well as a graphical-ish DSL for connecting everything.[0] [0]: https://doc.akka.io/docs/akka/current/stream/stream-graphs.html https://doc.akka.io/docs/akka/current/stream/stream-graphs.h...
  - samuell 6y ago
    That is nice to see! I was evaluating Akka a bit some years ago.
    - rad_gruchalski 6y ago
      AkkaHttp is written on top of Streams. There are many more things written using it but Http is a good example to learn from.
  - jghn 6y ago
    It is worth noting that Akka Streams is an implementation of the Reactive Streams [0] spec, using Akka [0]: https://www.reactive-streams.org https://www.reactive-streams.org

dustingetz 6y ago

FP decouples the AST (symbolic functions with input ports and an output port) from the evaluation context, which might be async, or sync, backpressured, stateful, exceptional, incremental/reactive ... mix-n-match whatever behaviors you want, all for the same abstract AST

samuell 6y ago
I actually didn't want to make this too much of an FP-bashing comment, as I find great use of many FP concepts, daily. I have yet to see an FP concept for composition though, that is as simple and generic in its implementation, as the FBP principles. I have sometimes thought FBP networks provide roughly the same function (pun not intended) as a monad, although I never seem to fully grasp what a monad is, so I can't tell for sure :o)
- dustingetz 6y ago
  Yes, dataflow with dynamic topologies (self-adjusting computation) forms a monad. If the "flowchart" is merely static, that forms an applicative. https://blog.janestreet.com/breaking-down-frp/ https://blog.janestreet.com/breaking-down-frp/ In my opinion, understanding the monad is essential to understanding how to fully separate an AST from its evaluation behavior. That is basically what a monad is, the monad is the computational structure that does that.
  - samuell 6y ago
    Thanks, that clarifies a lot!
dmbarbour 6y ago
I think most FP languages don't decouple AST from evaluation context. E.g. I cannot perform abstract interpretation on a function's AST, nor rewrite a function's AST for incremental computation, nor perform explicit partial evaluation during function composition by composing the ASTs. I only can access opaque `Input -> Output`. Also, FP algorithms developed for one evaluation context cannot easily be ported to another. They are implicitly entangled with assumptions. For example, I cannot evaluate a lazy algorithm in an eager evaluation context without paying a huge price. Adding back-pressure for list processing where a program was not designed for it would easily result in deadlock. Adding exceptions without adding unwind (or bracket, try/finally, etc.) to the existing program expressions will easily result in buggy code. I think your assertion might be true for some specific integrations of FP nodes into FBP. Is this what you mean?

codetrotter 6y ago

Your flowbase.org domain redirects to a different domain. I thought that perhaps you had a typo in the URL or forgot to renew your registration of the domain, but upon a closer look I think it’s probably just a misconfiguration in your webserver config causing it to redirect to another one of your projects. My guess is that you forgot to put the definition of the domain sans www in your webserver config. If my guess is correct then this link should work: https://www.flowbase.org https://www.flowbase.org Edit: doesn’t work either. Perhaps you don’t have a TLS cert for that domain? In which case maybe this link will work instead: http://www.flowbase.org http://www.flowbase.org Edit 2: Without https it works. And like the person responding to this comment said it redirects to the GH repo.

wizzwizz4 6y ago
The link's trying to point you here: https://github.com/flowbase/flowbase https://github.com/flowbase/flowbase
- samuell 6y ago
  Thank you!
samuell 6y ago
Ouch, sorry, and thanks! Wrote on mobile on the go and didn't manage to double check the link.

FigmentEngine 6y ago

Sounds like XProc 3.0 https://xproc.org/ https://xproc.org/

megameter 6y ago

Having studied and implemented FBP systems in the past, one major takeaway I've gleaned is that most automation problems start off as a linear sequence of processes, and so the branching-graph of FBP looks unwieldy and superfluous. But this is deceptive; you probably don't want to have a huge number of branches in the design, but you will want them as an optimization step or a way to combine inputs. So it's useful to design with FBP in mind but with a linear interface as the entry point. Another aspect of this is that FBP graphs are static but you may have a need to reconfigure them frequently; that is, you may want to have a graph compilation step drawn from a source language, rather than manually wiring it up. A way of making that graph compilation more than a syntax is to include a formal constraint solver: Excel, for example, flows the data after determining a solution for how cells relate to each other. The power of the spreadsheet paradigm really lies in these combinations of concepts. Lastly, there isn't really magic in the algorithmic/implementation aspects of FBP. It grew out of 1960's mainframe types of problems, and so it can be implemented in a low level way with static pools of memory and pieces of assembly code. But it remains conceptually just as relevant to today's massive distributed systems.

goliatone 6y ago

The flowbase.org link is redirecting me to a polish website, not sure what’s going on there

deleted 6y ago
[deleted]
samuell 6y ago
Sorry about this, I was typing in a hurry. The links should redirect to https://github.com/flowbase/flowbase https://github.com/flowbase/flowbase but seems to be misconfigured for https. Looking into it.

vitalhead 6y ago

Visual aspect of FBP is also brilliant and helps to navigate/comprehend inherently parallel execution flow. The same complexity network expressed in linear text would require more effort to grasp. And it allows you to leverage your brain's visual capacity which is quite powerful - humans are visual creatures (https://www.seyens.com/humans-are-visual-creatures https://www.seyens.com/humans-are-visual-creatures). For example in Excel the update dependency network is completely hidden. The maintenance of complex Excel models is really hard. For same reasons we start looking at FBP and functional reactive programing to simplify the design and maintenance of complex interactive UIs. End up implementing Kelp (https://kelp.app https://kelp.app) with visual FBP editor and reactive framework (https://kefirjs.github.io/kefir/ https://kefirjs.github.io/kefir/).