12 ms·

How Flow-Based Programming Could Save The Sanity Of Web Developers

148 points by robbyking 13y ago

knz42 13y ago
The title is sensationalist. The "coding method" is simply data flow programming. Many languages provide data flow facilities nowadays. The reason why graphical UIs for data flow languages have not become popular is that large programs become unwieldy to visualize and edit, whereas everyone know how to use text editors.
- AsymetricCom 13y ago
  It makes me wonder at which point JavaScript will become more or equally unwieldy that using such a system will be a competitive trade-off. Of course, this is in detriment to the actual security and extensibility of so-called web apps. It's a great opportunity to coopt developers onto a platform with only industry-accepted functionality.
jbellis 13y ago
"The paradigm was so disruptive that it was suppressed by computer scientists for decades." It didn't really get better from there.
- hga 13y ago
  Yeah, I must have imagined the 1-2? computer science professors at MIT who had active data flow research programs when I showed up in 1979. The general opinion on campus was "this looks very interesting" but no one had yet figured out how to make it really work and be comprehendable.
  - gruseom 13y ago
    And still haven't. The most interesting attempts, like Ed Ashcroft's and Bill Wadge's Lucid, seem to have withered. It's not clear to me whether that was because they didn't offer enough that was different, or it was too hard to make the implementations efficient, or both. And yet it all does still "look very interesting".
- zeteo 13y ago
  I kept reading and reading through poetic descriptions of how valuable the method is, hoping to get to at least a short description of what the method actually is. Excitedly I clicked on "the company published FBP as a technical disclosure bulletin", only to get to the Wikipedia entry for "IBM Technical Disclosure Bulletin". Reluctantly I must conclude that the article is pure and content-less bullshit instead.
- nathan_long 13y ago
  Well, we all know how reluctant scientists are to promote groundbreaking ideas, since all prestige in science comes from doing what's been done before. No wait, scratch that. That's just in BizzaroWorld.
- deleted 13y ago
  [deleted]
jaimebuelta 13y ago
This is the kind of idea that looks fantastic, but that it's not that useful (at least, so far). The equivalent of "Minority Report" interfaces. The article is nothing more that a collection of cliches and hype...
deleted 13y ago
[deleted]
jgrahamc 13y ago
Here's the deal (and I'm sure that this is the same for most programmers): release something that uses this paradigm and let's me get something done and I'll use it. It's getting something done that matters. That's the same motivation for me using Go (easy concurrency), Perl (easy text mangling), Lua (easy embedded stuff), and on and on. If Flow-Based Programming is the next big thing then great. Let's get at it. But please don't sell me bullshit like: "The paradigm was so disruptive that it was suppressed by computer scientists for decades."
- sanderjd 13y ago
  Completely agree - I see links to frameworks and tools in the article but none to projects and products made with them.
- doorhammer 13y ago
  I haven't read the rest of the article, but I got to that phrase and my disposition instantly turned from "huh, this is interesting" to "snake oil", whether deserved or not. Reminds me of the recent "One weird trick!" article [0] and lines like, "the simple solution dietitians don't want you to know!" Content aside, that kind of thing immediately sets off alarms for me. [0] http://www.slate.com/articles/business/moneybox/2013/07/how_one_weird_trick_conquered_the_internet_what_happens_when_you_click_on.single.html http://www.slate.com/articles/business/moneybox/2013/07/how_...
  - jerf 13y ago
    For me it's the fact that I've seen this whole spiel a lot of times already, in EVERY DETAIL. Visual programming will save us all. Nonprogrammers should be able to program. Suppressed, blah blah blah. For something so revolutionary, it sure is some awfully-well-trodden ground. And I don't just mean that this is an old paradigm, I mean this spiel is well-trodden ground. In a nutshell, it doesn't work. Using Fred Brook's venerable terminology, the essential complexity of programming is too difficult for nonprogrammers. Even if you entirely get rid of the accidental complexity... and visual layout does not do that, it's full of incidental complexity involved in the layout itself... you still can't get non-programmers into a programming situation, and trying to get them into a concurrent programming situation is just sheer insanity. And we know it doesn't work because it has been tried. Over and over. Ad naseum, as in, yeah, I'm sick of hearing these ideas as The Answer as if nobody has ever heard of these before. Tell me what's different about your solution and all the other people who have tried this.
meemoo 13y ago
How does flow-based programming compare with functional reactive programming, like Elm [1] and Bacon.js [2]? Could those be made visual with a graph editor like this? Bacon.js has some flow diagrams [3] to describe some of its advanced concepts. 1. http://elm-lang.org http://elm-lang.org 2. https://github.com/baconjs/bacon.js https://github.com/baconjs/bacon.js 3. https://github.com/baconjs/bacon.js/wiki/Diagrams https://github.com/baconjs/bacon.js/wiki/Diagrams
- michaelsbradley 13y ago
  Also see Flapjax: http://www.flapjax-lang.org/docs/ http://www.flapjax-lang.org/docs/ https://github.com/brownplt/flapjax/ https://github.com/brownplt/flapjax/ The programming behind the JavaScript library is really solid, and was most actively developed from 2006 through 2009. It never caught on, possibly because the mental model required for successfully hacking with "event streams" and "behaviors" is decidedly a functional one, and those concepts are quite abstract to begin with (more so than objects and prototypes). Also, studying the implementation is not for the faint of heart. On top of that, I don't think any big names got behind it and there wasn't exactly a marketing campaign. Ahead of its time? Definitely. And it's probably still worth looking into as an alternative when trying to choose among the various reactive libraries that have popped up in the last year or so.
- johnpmayer 13y ago
  Elm could do this, and it would also benefit from a graph editor that knew about the types
- chipsy 13y ago
  FRP is very similar in nature to FBP. (even the acronyms are similar) The main difference is in packaging. The original FBP systems were still 70s-style near-metal code within the component design, but the architecture described a protocol just sufficient for statically and asynchronously connecting the components together, and a runtime method that is amenable to the simplistic approach of having each component run round-robin until all return a "finished" signal. It's very much an "industrial engineering" perspective. FRP, on the other hand, comes from the traditions of functional programming, and so the evaluation order computation is given a more academic treatment, and the languages are given more explicit syntax, where FBP doesn't aim to describe itself much more deeply than the flowcharts. Both FRP and FBP have purity and immutability as core concepts. So as I see it: different starting point, different packaging, same conclusions.
jliechti1 13y ago
>> What we need is not more programmers. What we need is to enable non-programmers to participate in the creation process, not just the ideation process,” says Kenneth Kan, CTO of a company called Pixbi and a recent convert to flow-based programming. I disagree with this. This might work fine for small programs, but will quickly fall apart for anything of significant size. National Instrument's LabVIEW [0] offers a graphical programming language called "G". G makes it easier for non-programmers (usually other types of engineers) to create the software they need. However, as their programs grow, they can easily get out of hand and become a dangled mess if they don't understand good program design. I am all for getting more non-programmers programming, but this method does not solve the need to understand programming fundamentals. In my experience with LabVIEW, it was really great for making things like concurrency easy, and it was really great for quickly building GUIs to control hardware. I like to think that graphical programming makes some "hard" things easy, and some "easy" things hard (it's a bit manually intensive to make complex mathematical equations - though there are blocks you can use that let you drop in C code into the interface). Like anything else, the methodology has its advantages and disadvantages. [0]: http://en.wikipedia.org/wiki/LabVIEW http://en.wikipedia.org/wiki/LabVIEW
- bergie 13y ago
  If LabVIEW was perfect we might not have to build all this. Execution matters :-) I've written about this in http://bergie.iki.fi/blog/noflo-kickstarter-launch/ http://bergie.iki.fi/blog/noflo-kickstarter-launch/
- albeec13 13y ago
  "I am all for getting more non-programmers programming..." May I ask why? I completely disagree with that statement and all the initiatives pushing that agenda.
  - jliechti1 13y ago
    Sure - let me answer that with my post for the "Everyone does not need to learn to code" thread. When I say getting more non-programmers programming, I mean it from more of a practical standpoint. My previous post: "From a practical standpoint, I really wish everyone was taught enough programming to know how to automate basic manipulation of text data. I see way too many people editing long lists of data by hand, when there are much better ways to go about it (even Excel works great for this kind of stuff). Just knowing this setup below can let you do some really power automation of manual tasks and can save you a lot of time. Example template in Python: import csv with open('mydata.csv', 'r') as csvfile: reader = csv.reader(csvfile, delimiter=',', quotechar='"') for row in reader: # Data manipulation here " Link: https://news.ycombinator.com/item?id=6237430 https://news.ycombinator.com/item?id=6237430 I'd also recommend looking at Jach's post (the child post of my post in the link) - I thought he made some really good points/critiques about 'tool-based' vs 'concept-based' teaching.
albeec13 13y ago
"'What we need is not more programmers. What we need is to enable non-programmers to participate in the creation process, not just the ideation process,' says Kenneth Kan, CTO of a company called Pixbi and a recent convert to flow-based programming." Could not disagree more. I'm so tired of these initiatives to "get everyone coding!" Everyone can't be a programmer, just like everyone can't be a doctor and everyone can't be a mechanic. There are certain skills, mindsets, and desires that drive people to do what they do.
- smokel 13y ago
  I agree that it is tiresome to see some of these initiatives, but you may consider the analogue that many people now can write, even though not everyone writes literature.
  - albeec13 13y ago
    The important distinction is that knowing how to read and write is a basic skill everyone needs in order to be an effective human being. Similarly, everyone needs to know math and problem solving skills (the major basis of programming) to be effective human beings. In other words, I fully support the push for STEM, but force-feeding programming into the agenda is both unnecessary (as those interested in math, problem solving and computers will tend to go into programming fields) and potentially counterproductive (having many unskilled "programmers" trying to touch important software could be catastrophic in many cases.) It's kind of like the groups of script-kiddies who consider themselves "hackers" but really aren't doing anything but utilizing tools real hackers have created. This is how NoFlo (from the article) strikes me.
    - dragonwriter 13y ago
      > The important distinction is that knowing how to read and write is a basic skill everyone needs in order to be an effective human being. I don't think that's true. Its something people may need to be (or, at least, greatly benefit from in being) effective members of modern society, but all the people that weren't part of the literate minority when literacy was rare (or before writing existed) were not "ineffective human heings".
codegeek 13y ago
"His company is busy resurrecting flow-based programming with a framework called NoFlo, an implementation of FBP for NodeJS" Something interest to note. The name of the framework is NoFlo that sounds like NoFlow And it is a framework for flow based programming. Wonder why they came up with that name.
- bergie 13y ago
  Now NoFlo works on both Node.js and browsers... But when I started it, I targeted only the former. So, Node.js Flow -> NoFlo
- meemoo 13y ago
  It comes from "Node Flow." Or reverse psychology?
gruseom 13y ago
I have two questions. First, what exactly is Flow-Based Programming? Emphasis on exactly. I know what dataflow is and I know who J. Paul Morrison is and I've even browsed through his book. I still can't figure it out, and the book (at least the parts I looked at) had too many diagrams and not enough code to clarify this most basic of questions. Every discussion on it seems hopelessly hand-wavey. What I want is the diffs between FBP and other computational models. Just the diffs, please! Or is it just dataflow? In that case, which version of dataflow? There have been many. Second, we know from the history of these things that it's easy to sell managers and journalists on boxes-and-lines visual programming. The pictures look a lot easier to understand than reams of source code do, but that's always because the examples are trivial—invariably some variant of a box with 2 in it, another box with two in it, arrows leading from those to a box with + in it, and then an arrow leading to a box with 4 in it. These always turn out to be a siren song because they don't scale to application complexity. How exactly is FBP different? Emphasis on exactly.
- enraged_camel 13y ago
  >>I have two questions. First, what exactly is Flow-Based Programming? Emphasis on exactly. Let the confusion flow through you!
- drcode 13y ago
  The crude/simple explanation I think is to imagine your program like an Excel spreadsheet: Some cells on the spreadsheet contain your client-side data (i.e. your "model"). Other cells are used by a javascript library to decide what's in the DOM (i.e. your "view") The "flow-based programming" model involves creating all the formulas that shuffle the data from the model to the view. At least that's what I can gather from what I've read (and I agree it's always been a big fail in the past, though there may be some merit to this idea eventually in limited roles.)
dragonwriter 13y ago
Aren't there like eleventy-bajillion enterprise tools that include exactly this kind of visual, dataflow-oriented methods of constructing large systems by wiring together individual components? Its not exactly an "arcane" model that's been lost in the mists of time since its use in "1970s banking software". And its very much not something that's been "suppressed by computer scientists for decades".
brianpaxe 13y ago
I thought this quote from the article was interesting: Kuhn’s book on scientific revolutions includes a famous quote from physicist Max Planck about what really causes paradigm shifts: "A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it."
- RogerL 13y ago
  Except, of course, scientists change their minds all the time based on the evidence. Look at all the work in QM in the 20s and 30s. These guys had competing hypothesizes, and some won out based on the evidence. The people that held the opposing view? They quickly accepted the errors of their ways. This is known as the bozo the clown argument. You know, somebody says "My perpetual motion machine will work! They laughed at Einstein, you know". And the reply is "yes, and they also laughed at bozo the clown". Almost always, if everyone is against your idea, having tried it, you are probably wrong. There is probably no industry on the planet more open to change. We've gone from procedural to OO, functional programming is having a huge resurgence, static typed to dynamic typed, waterfall to extreme/agile/whatever, and so on. And, despite the claims of the article, data flow programming has been alive and well the whole time. It hasn't been used much because of all the things that we have raised each time the topic is brought up, with no real response other than claims that we just don't get it, that people disagreeing is a sign that you are doing something right, and so on. I don't find it convincing at all. It's time to retire that Kuhn/Planck quote.
  - gruseom 13y ago
    It's time to retire that Kuhn/Planck quote Oh, I don't think so. As its presence in Kuhn's book suggests, the kind of 'scientific truth' Planck was talking about is not just any hypothesis, but the much larger thing that we now (post-Kuhn) call a paradigm. For a counterexample, you'd need to show a paradigm shift occurring within the individual careers of the most established scientists in a community. QM isn't an example of that, for two reasons: it was the work of a new generation, and the debates you refer to were taking place within the new paradigm [1]. Your example is ironic, since it was Planck who put the Q in QM. If the history of QM refuted the quote as easily as all that, he'd never have said it in the first place. (I do agree with you that most claims of a new paradigm turn out to be false.) [1] That's not quite correct; they also took place before the new paradigm had crystallized—a time when (as Kuhn describes it) old models have broken down and lots of big things are up for grabs. Under such circumstances, minds change more fluidly. But such transitional circumstances are relatively rare and Planck was talking about the normal ones.
deleted 13y ago
[deleted]
protomyth 13y ago
So, I should use BizTalk Server[1][2] for my website. I'm not saying it's a bad idea, but I get the feeling it will be a hard sell. 1) http://www.microsoft.com/en-us/biztalk/default.aspx http://www.microsoft.com/en-us/biztalk/default.aspx 2) http://msdn.microsoft.com/en-us/library/aa577881.aspx http://msdn.microsoft.com/en-us/library/aa577881.aspx
dangoor 13y ago
For people looking for something more substantial, you might enjoy this c2 page about FBP vs. Actors: http://c2.com/cgi/wiki?ActorsAndFlowBasedProgrammingDiscussion http://c2.com/cgi/wiki?ActorsAndFlowBasedProgrammingDiscussi... The basic idea with FBP is that you have multiple processes that communicate over channels (kind of like actors, but the communication is unidirectional across those channels). I'm not a fan of GUIs like this generally, but the idea of FBP is interesting in the same way that the actor model is interesting.
a0 13y ago
I think programmers are always thinking about the flow of programs. Jumping through blocks of statements, following the chains of function calls and analysing their definitions when needed. The _flow_ is there, but it's not visual. We are always imagining what would the compiler or interpreter do with our code. On the other hand, we must be able to quickly navigate in the source tree. That's why text editors like Vim, that understand the contents of the programs (buffers, text objects, definitions), are so powerful. The same happens with Xcode, for example. The Interface Builder is a graphical tool that defines the code components as visually manageable objects. Maybe the proposed paradigm could extend this ideas of representing the code in flows, as graphs of objects. I'm really curious to try it. Mainly because it seems to feet really well with handheld devices.
pcunite 13y ago
These guys should talk to the creators of VNOS 1.1 and ask how that went. Visual programming does have its place. My mom was able to make a website using Weebly without any help. However, as things get complex I personally can visualize (in my mind) the interconnects. In my opinion, we need easier to use libraries, not visual tools. But I genuinely want to be corrected.
deleted 13y ago
[deleted]
shortcj 13y ago
This read like a long winded video sales pitch on the unbeatable game plan for seducing women.
it_learnses 13y ago
"The paradigm was so disruptive that it was suppressed by computer scientists for decades." Stopped reading after that.
keeran 13y ago
There are multiple communities of 'creative coders' who already work with this paradigm using tools like vvvv[0], MaxMSP[1], PureData[2] & others. Closer(ish) to home we have QuartzComposer[3] available for OSX. I like all of these tools but can't help feeling I'd be more effective if I could just get at the underlying code half the time (in cases where that's not an option). [0] http://vvvv.org/ http://vvvv.org/ [1] http://cycling74.com/ http://cycling74.com/ [2] http://puredata.info/ http://puredata.info/ [3] http://developer.apple.com/graphicsimaging/quartz/quartzcomposer.html http://developer.apple.com/graphicsimaging/quartz/quartzcomp...
- deleted 13y ago
  [deleted]
- lstamour 13y ago
  Seconding Max/MSP, you can write C code in it thanks to "gen".
- heuermh 13y ago
  > I'd be more effective if I could just get at the underlying code For something similar to MaxMSP/PureData but with writing code in a text editor, you may wish to take a look at ChucK [1]. Instead of graphical edges or links between between UGens ChucK provides the '=>' operator. E.g. adc => Chorus c => LPF lpf => Echo e => Delay d => dac; [1] http://chuck.cs.princeton.edu http://chuck.cs.princeton.edu
  - melloclello 13y ago
    I took a class in ChucK once. It is moderately terrible and half-finished but I managed to write a patch once while blackout drunk, so I'd say it's pretty intuitive.
- aufreak3 13y ago
  > I like all of these tools but can't help feeling I'd be more effective if I could just get at the underlying code half the time (in cases where that's not an option). I've seen novices do cool stuff with Max without thinking that. Perhaps this feeling comes more from familiarity with textual coding than something fundamental about textual coding. On the other hand, we keep saying "a picture is worth a thousand words", but here we are writing blog posts in gobs! That suggests that text is what we train for, and is what we might be most effective with in the future too.
jimfuller 13y ago
ya, my eyes gloss over when I read 'rediscoveries' like this ... I think the finer point is more about workflow then programming. That is state based workflow is fundamentally harder for developers to grok, also maintaining state is a kind of fallacy when scaling out and/or up. Flow based workflows, while probably can't cover easily the breadth of exotic corner cases that FSM can achieve, are much easier to understand and state is byproduct of the system, not to mention that a full record of mutation occurs with a copy of data at every step (similar to MVCC - Multi-value-consistency-control) This is similar to how http://www.w3.org/TR/xproc/ http://www.w3.org/TR/xproc/ XProc works, which has a serious implementation in http://xmlcalabash.com/ http://xmlcalabash.com/ and slowly gaining adoption in XML circles. Its nice to see yet another XML technology, which of course is based on older established concepts in computing, being emulated/replicated in the JS universe ... shame about the visual programming antipattern/lunacy.
falcolas 13y ago
Based solely on my previous experience with a "flow-based" programming system, which endeavored to make programming financial decision logic a matter of connecting boxes, it's the 80/20 problem. It makes 80% of the work dirt simple and able to be completed. The remaining 20% still needs to be implemented in code, and takes most of the time. It got so bad that they actually ended up implementing "code" objects, just so the code could be edited and incorporated into directly into the tool instead of around it.
- meemoo 13y ago
  I'm a JS hacker working on the NoFlo UI, and one of our primary design goals is to make it easy to code new modules on the fly (in the same environment, with tests) when that makes more sense.
oakwhiz 13y ago
The advantage of this method of programming is that complex parallel processing tasks can be reduced to a directed acyclic graph from information sources, through filters, and into information sinks. You can then take this graph and either interpret it by creating threads for each of the nodes in the graph and shuffling information between them using FIFO queues, or try to compile the graph to a better performing, but "flattened" native program. The main problems with this approach are how to properly represent timing, latency, and node state, since you can send one piece of information through the graph at a time, or you can try to improve efficiency by pipelining the graph - using the graph like a shift register, and making sure every node has work to do at all times. Of course it's not a cure-all solution and doesn't map well to every problem domain, but it is generally easier to reason about for complex parallel processing tasks, and can be deployed over multiple machines with minimal "glue" code.
- zhemao 13y ago
  So, hardware description languages, which are essentially flow-based programming languages, solve this using clocks and sensitivity lists. An FBPL for web programming would probably use events instead of edge/level triggers and timers instead of clocks.
lukifer 13y ago
You know what costs us sanity? It's not code. It's people. Scope creep. Last-minute specifications. Pointy-hairs who don't understand technical debt, and need everything done yesterday. Design by committee. Too many cooks in too many kitchens. Legacy processes, legacy data, legacy user habits. Unrealistic expectations, unrealistic timelines, and unrealistic budgets. Marginally competent coworkers. Office politics. Compared to those, callback hell and and classitis are a dream come true. And while all the problems above are reducable with the right processes, you'll never shrink them all to zero. "Algorithms may have mathematical underpinnings, but computer programming is a behavioral science." - @brixen
jwatte 13y ago
I tried a flow based tool for the Mac in late '80s. It really didn't make things easier - and it had terrible support for things that text languages take for granted, like foreign library import. Computer scientists have not "hidden away" the idea. It simply hasn't provided close to the value that text based languages have in real life. The new tool is as much snake oil as any tool before it that uses the same language in its advertising.
chris_mahan 13y ago
being a good programmer is not about knowing how to handle the minute details; it is about know how to handle complexity.
spellboots 13y ago
"Let's get non-musicians making music! We'll just give them a nice MIDI keyboard instead of that hard-to-use violin" Sure, this will skip over some tricky bits of getting started with programming, just as a MIDI keyboard can make it easier to make violin sounds without having to learn all that pesky fingering. However, as with playing the violin, those bits aren't really the hard part to learn; the easiest to use keyboard in the world won't make you a composer and flow based programming is not going to make you a programmer.
- meemoo 13y ago
  MIDI tools enabled more people to make new kinds of music. Not everybody will be a programmer, but hopefully a slightly larger percentage of the population will be able to discover the power of algorithmic thinking, and hopefully they will work on some interesting problems.
6ren 13y ago
The fundamental problem with visual programming methods as opposed to linguistic methods is that abstraction doesn't work as well. I agree that we'll get glue programmers, who just stick modules together. We already do: they are those programmers who use APIs. i.e. all of us to some extent. So far, it's still programming.
dclowd9901 13y ago
At this point, I see the use of Flow-based programming to be analogous to something like Titanium (the JS framework that lets you build iOS apps). It's interesting in concept, and will most assuredly open the doors to allow a larger swath of people to participate, but for anything of consequence, you've still gotta buckle down and write some code. The reason is almost entirely about performance optimization, which is something you just can't get from compiling from GUI.
wwweston 13y ago
> What we need is not more programmers. What we need is to enable non-programmers to participate in the creation process, not just the ideation process Why don't we need more programmers participating in the ideation process, not just the creation process?
stormbrew 13y ago
> The paradigm was so disruptive that it was suppressed by computer scientists for decades. Did someone not get their "suppress the disruptive programming technique" kickback cheque in the mail this month?
richardjordan 13y ago
People waiting for "the breakthrough" that suddenly makes programming easy and available to all are kidding themselves. As others have said there's inherent complexity to programming and solving problems that can't be wished away with paradigm changing strategerized facilitations utilized by flow-optimized programmerators or whatever the secret code knowledge suppressed by scientists was prescribing. On the broader point - and the comments by folks saying we should have made things better by now: we have. I started out learning to program with zero help, no tools, few manuals on an Apple II in some flavor of BASIC or machine code by trial and error. Decades later having not coded a lick in more than 15 years I picked up Ruby On Rails quickly and easily (enough to get out MVP) and with the help of Stack Overflow and Google have spent the last three years getting better and better at it. I can do dramatically more today than I could have imagined in my days alone with an Apple II and a poor english translation of a poor chinese translation of a pirated Apple manual. It's all better. The worst programmers have super powers available to them (they don't always use them admittedly) compared to prior days. Things are getting better but they're earning it, not because "THEY" don't want you to know "SECRET KNOWLEDGE".
- melloclello 13y ago
  > People waiting for "the breakthrough" that suddenly makes programming easy and available to all are kidding themselves. As others have said there's inherent complexity to programming and solving problems that can't be wished away with paradigm changing strategerized facilitations utilized by flow-optimized programmerators or whatever the secret code knowledge suppressed by scientists was prescribing. I totally agree with you and would go so far as to say there's probably a kind of upper bound to the rate at which a human can translate an original idea in their head into an unambiguous logical construct of the kind which can be executed by a computer. However, I would also say that the narrowest bottleneck in this process is the rate at which someone can comprehend the current state and structure of a given program, and I would say that anything that tries to improve on the status quo of text files in a directory is pretty cool.
  - Roboprog 13y ago
    I miss call trees. I used to have tools to generate call tree diagrams for procedural languages, back in the 80s. With virtual/dynamic methods, IDEs don't bother to try to make such a feature anymore.
drivebyacct2 13y ago
Polymer Project.
hcarvalhoalves 13y ago
> Today’s web programmers grapple with problems that people in the early days never had to deal with. They’re building complex UIs, juggling a bunch of APIs, and running multiple processes at the same time. All of these tasks require mastering the flow of data between application components in real-time, something which even the most advanced developers struggle with. No. The problem is not moving data around. The biggest problem on web development is the drive from the industry to shoe-horne applications on top of platforms that weren't meant for it.
now_what 13y ago
FWIW the visual graph style IDE is used to write scripts for 3d design programs Rhino 3d (Grasshopper plugin [0]) and Revit (Dynamo). It's pretty popular among architects and the like. [0] http://en.wikipedia.org/wiki/Grasshopper_3d http://en.wikipedia.org/wiki/Grasshopper_3d
drcode 13y ago
FYI, the Clojure Pedestal Client-side framework has a similar update model to this.
MarcScott 13y ago
I was using Xpresso in Cinema 4D years ago, which looks remarkably similar. http://www.maxon.net/products/cinema-4d-prime/customizing.html http://www.maxon.net/products/cinema-4d-prime/customizing.ht... This is nothing new and neither has it been "suppressed".
- Raphmedia 13y ago
  I agree. A lot of software use that kind of visualization. It's nothing new.
marshray 13y ago
Scratch, Lego Mindstorms, DirectShow audio graph editor, any number of lab and process control systems.
abawany 13y ago
This reminded me in some way of FASTech CELLworks (sic): software used in manufacturing automation to define process flows. It had some interesting features such as message passing via a mailbox abstraction, concurrent execution, and a memory-based data store.
dbond 13y ago
We're building something very similar, aiming to have demos and beta by new year. (Bad) signup page here if interested: http://www.lexasapp.com/ http://www.lexasapp.com/
romansanchez 13y ago
Has any body heard of TIBCO? They've been doing this (flow-based programming) for years(>10).
shepbook 13y ago
There are two points I don't get with such a system. 1) The logic/functions needed in each "black box" still need to be written. How is that handled? Is that where the line of "programmer vs non-programmer" is crossed? 2) Programming is, more or less, the skill of breaking down a problem into the parts needed for its solution and providing the solution. The value of a programmer isn't the ability to type in code, so much as the ability to logically break down the problem and solve it. How would such a programming paradigm actually solve that problem?
- meemoo 13y ago
  The NoFlo team is coders designing this for our own work first. We are free to put as much code in a black box as we like. We'll be running tests with ourselves and our early community to make some best practices, but there is no rule that keeps you from writing most of your logic in one block. Our theory is that the graph-based view will encourage encapsulation in different ways that make sense for the different domains that adopt the tool. One of the big design differences (vs Quartz Composer, Lab View, Pure Data) is that it will be easy to dive in and edit component code and make new components when needed.
zhemao 13y ago
Okay, yeah, so I agree with most of the criticisms here. Flow-based languages didn't catch on because no-one came up with a really compelling general-purpose implementation that scales up to large, complex projects. It's not because it was actively suppressed by the programmer Illuminati to keep our salaries high. I think the main reason FBP hasn't gone mainstream is because most implementations are built around a graphical programming environments (e.g. LabView, Pure Data, Scratch) in order to cater to non-programmers. But programmers love their text files, and for plenty of good reasons. Code represented as text can be edited by any general-purpose text editor, work well with version control systems, and can (usually) be easily split into separate files. It's probably instructive, then, to look at a domain in which textual flow-based programming languages have become mainstream. I'm speaking, of course, about Hardware Description Languages, which are used by digital circuit designers to simulate and prototype complex circuits, like CPUs. The two dominant HDLs are VHDL and Verilog. Both of these languages use a flow-based model, in which logical blocks called entities are connected to each other through input and output ports. The reason these languages work is because the domain is a good fit for FBP and because they are designed for the domain experts (circuit designers) to use, and not for some non-technical manager to be able to understand. Taking a similar approach to web programming may work. Logical blocks connected through ports. Except instead of ones and zeroes going through these ports, you have I/O events and data. I haven't looked at NoFlo, but it might be interesting. Note that I'm not saying that VHDL and Verilog are good languages, even for their intended domain. They have plenty of warts, and a FBP language for software that basically copies one or the other would be terrible (especially VHDL; oh god, please nobody make VHDL for web programming). But the entity-port model is essentially sound, and I would be quite interested in a general-purpose language that adapts that model in a well thought-out way.
keithnoizu 13y ago
I doubt flow based programming is going to let the nonprogrammer program but I do think there is some potential in the age of the tablet to get stuff done visually with flow based development and touch friendly tools provided you can easily switch back and forth to source. E.g. something sort of like windows worfklow on speed.
leke 13y ago
When I saw this I immediately thought of [Drakon](https://en.wikipedia.org/wiki/DRAKON https://en.wikipedia.org/wiki/DRAKON).
rbanffy 13y ago
It's a real tragedy people who don't know history can't learn from it.
Too 13y ago
I looked at all the pictures in the article and couldn't imagine how to draw something as fundamental as an if/else-statement. Think about that for a moment. http://www.maxon.net/uploads/pics/xpresso_17.jpg http://www.maxon.net/uploads/pics/xpresso_17.jpg shows an alternative that i imagine could be used to lazily evaluate only certain connected boxes but still...just look at that mess. Please don't remind me how if-statments in labview are represented. It gives me nightmares.

148 points by robbyking 13y ago

knz42 13y ago

The title is sensationalist. The "coding method" is simply data flow programming. Many languages provide data flow facilities nowadays. The reason why graphical UIs for data flow languages have not become popular is that large programs become unwieldy to visualize and edit, whereas everyone know how to use text editors.

AsymetricCom 13y ago
It makes me wonder at which point JavaScript will become more or equally unwieldy that using such a system will be a competitive trade-off. Of course, this is in detriment to the actual security and extensibility of so-called web apps. It's a great opportunity to coopt developers onto a platform with only industry-accepted functionality.

jbellis 13y ago

"The paradigm was so disruptive that it was suppressed by computer scientists for decades." It didn't really get better from there.

hga 13y ago
Yeah, I must have imagined the 1-2? computer science professors at MIT who had active data flow research programs when I showed up in 1979. The general opinion on campus was "this looks very interesting" but no one had yet figured out how to make it really work and be comprehendable.
- gruseom 13y ago
  And still haven't. The most interesting attempts, like Ed Ashcroft's and Bill Wadge's Lucid, seem to have withered. It's not clear to me whether that was because they didn't offer enough that was different, or it was too hard to make the implementations efficient, or both. And yet it all does still "look very interesting".
zeteo 13y ago
I kept reading and reading through poetic descriptions of how valuable the method is, hoping to get to at least a short description of what the method actually is. Excitedly I clicked on "the company published FBP as a technical disclosure bulletin", only to get to the Wikipedia entry for "IBM Technical Disclosure Bulletin". Reluctantly I must conclude that the article is pure and content-less bullshit instead.
nathan_long 13y ago
Well, we all know how reluctant scientists are to promote groundbreaking ideas, since all prestige in science comes from doing what's been done before. No wait, scratch that. That's just in BizzaroWorld.
deleted 13y ago
[deleted]

jaimebuelta 13y ago

This is the kind of idea that looks fantastic, but that it's not that useful (at least, so far). The equivalent of "Minority Report" interfaces. The article is nothing more that a collection of cliches and hype...

deleted 13y ago

[deleted]

jgrahamc 13y ago

Here's the deal (and I'm sure that this is the same for most programmers): release something that uses this paradigm and let's me get something done and I'll use it. It's getting something done that matters. That's the same motivation for me using Go (easy concurrency), Perl (easy text mangling), Lua (easy embedded stuff), and on and on. If Flow-Based Programming is the next big thing then great. Let's get at it. But please don't sell me bullshit like: "The paradigm was so disruptive that it was suppressed by computer scientists for decades."

sanderjd 13y ago
Completely agree - I see links to frameworks and tools in the article but none to projects and products made with them.
doorhammer 13y ago
I haven't read the rest of the article, but I got to that phrase and my disposition instantly turned from "huh, this is interesting" to "snake oil", whether deserved or not. Reminds me of the recent "One weird trick!" article [0] and lines like, "the simple solution dietitians don't want you to know!" Content aside, that kind of thing immediately sets off alarms for me. [0] http://www.slate.com/articles/business/moneybox/2013/07/how_one_weird_trick_conquered_the_internet_what_happens_when_you_click_on.single.html http://www.slate.com/articles/business/moneybox/2013/07/how_...
- jerf 13y ago
  For me it's the fact that I've seen this whole spiel a lot of times already, in EVERY DETAIL. Visual programming will save us all. Nonprogrammers should be able to program. Suppressed, blah blah blah. For something so revolutionary, it sure is some awfully-well-trodden ground. And I don't just mean that this is an old paradigm, I mean this spiel is well-trodden ground. In a nutshell, it doesn't work. Using Fred Brook's venerable terminology, the essential complexity of programming is too difficult for nonprogrammers. Even if you entirely get rid of the accidental complexity... and visual layout does not do that, it's full of incidental complexity involved in the layout itself... you still can't get non-programmers into a programming situation, and trying to get them into a concurrent programming situation is just sheer insanity. And we know it doesn't work because it has been tried. Over and over. Ad naseum, as in, yeah, I'm sick of hearing these ideas as The Answer as if nobody has ever heard of these before. Tell me what's different about your solution and all the other people who have tried this.

meemoo 13y ago

How does flow-based programming compare with functional reactive programming, like Elm [1] and Bacon.js [2]? Could those be made visual with a graph editor like this? Bacon.js has some flow diagrams [3] to describe some of its advanced concepts. 1. http://elm-lang.org http://elm-lang.org 2. https://github.com/baconjs/bacon.js https://github.com/baconjs/bacon.js 3. https://github.com/baconjs/bacon.js/wiki/Diagrams https://github.com/baconjs/bacon.js/wiki/Diagrams

michaelsbradley 13y ago
Also see Flapjax: http://www.flapjax-lang.org/docs/ http://www.flapjax-lang.org/docs/ https://github.com/brownplt/flapjax/ https://github.com/brownplt/flapjax/ The programming behind the JavaScript library is really solid, and was most actively developed from 2006 through 2009. It never caught on, possibly because the mental model required for successfully hacking with "event streams" and "behaviors" is decidedly a functional one, and those concepts are quite abstract to begin with (more so than objects and prototypes). Also, studying the implementation is not for the faint of heart. On top of that, I don't think any big names got behind it and there wasn't exactly a marketing campaign. Ahead of its time? Definitely. And it's probably still worth looking into as an alternative when trying to choose among the various reactive libraries that have popped up in the last year or so.
johnpmayer 13y ago
Elm could do this, and it would also benefit from a graph editor that knew about the types
chipsy 13y ago
FRP is very similar in nature to FBP. (even the acronyms are similar) The main difference is in packaging. The original FBP systems were still 70s-style near-metal code within the component design, but the architecture described a protocol just sufficient for statically and asynchronously connecting the components together, and a runtime method that is amenable to the simplistic approach of having each component run round-robin until all return a "finished" signal. It's very much an "industrial engineering" perspective. FRP, on the other hand, comes from the traditions of functional programming, and so the evaluation order computation is given a more academic treatment, and the languages are given more explicit syntax, where FBP doesn't aim to describe itself much more deeply than the flowcharts. Both FRP and FBP have purity and immutability as core concepts. So as I see it: different starting point, different packaging, same conclusions.

jliechti1 13y ago

>> What we need is not more programmers. What we need is to enable non-programmers to participate in the creation process, not just the ideation process,” says Kenneth Kan, CTO of a company called Pixbi and a recent convert to flow-based programming. I disagree with this. This might work fine for small programs, but will quickly fall apart for anything of significant size. National Instrument's LabVIEW [0] offers a graphical programming language called "G". G makes it easier for non-programmers (usually other types of engineers) to create the software they need. However, as their programs grow, they can easily get out of hand and become a dangled mess if they don't understand good program design. I am all for getting more non-programmers programming, but this method does not solve the need to understand programming fundamentals. In my experience with LabVIEW, it was really great for making things like concurrency easy, and it was really great for quickly building GUIs to control hardware. I like to think that graphical programming makes some "hard" things easy, and some "easy" things hard (it's a bit manually intensive to make complex mathematical equations - though there are blocks you can use that let you drop in C code into the interface). Like anything else, the methodology has its advantages and disadvantages. [0]: http://en.wikipedia.org/wiki/LabVIEW http://en.wikipedia.org/wiki/LabVIEW

bergie 13y ago
If LabVIEW was perfect we might not have to build all this. Execution matters :-) I've written about this in http://bergie.iki.fi/blog/noflo-kickstarter-launch/ http://bergie.iki.fi/blog/noflo-kickstarter-launch/
albeec13 13y ago
"I am all for getting more non-programmers programming..." May I ask why? I completely disagree with that statement and all the initiatives pushing that agenda.
- jliechti1 13y ago
  Sure - let me answer that with my post for the "Everyone does not need to learn to code" thread. When I say getting more non-programmers programming, I mean it from more of a practical standpoint. My previous post: "From a practical standpoint, I really wish everyone was taught enough programming to know how to automate basic manipulation of text data. I see way too many people editing long lists of data by hand, when there are much better ways to go about it (even Excel works great for this kind of stuff). Just knowing this setup below can let you do some really power automation of manual tasks and can save you a lot of time. Example template in Python: import csv with open('mydata.csv', 'r') as csvfile: reader = csv.reader(csvfile, delimiter=',', quotechar='"') for row in reader: # Data manipulation here " Link: https://news.ycombinator.com/item?id=6237430 https://news.ycombinator.com/item?id=6237430 I'd also recommend looking at Jach's post (the child post of my post in the link) - I thought he made some really good points/critiques about 'tool-based' vs 'concept-based' teaching.

albeec13 13y ago

"'What we need is not more programmers. What we need is to enable non-programmers to participate in the creation process, not just the ideation process,' says Kenneth Kan, CTO of a company called Pixbi and a recent convert to flow-based programming." Could not disagree more. I'm so tired of these initiatives to "get everyone coding!" Everyone can't be a programmer, just like everyone can't be a doctor and everyone can't be a mechanic. There are certain skills, mindsets, and desires that drive people to do what they do.

smokel 13y ago
I agree that it is tiresome to see some of these initiatives, but you may consider the analogue that many people now can write, even though not everyone writes literature.
- albeec13 13y ago
  The important distinction is that knowing how to read and write is a basic skill everyone needs in order to be an effective human being. Similarly, everyone needs to know math and problem solving skills (the major basis of programming) to be effective human beings. In other words, I fully support the push for STEM, but force-feeding programming into the agenda is both unnecessary (as those interested in math, problem solving and computers will tend to go into programming fields) and potentially counterproductive (having many unskilled "programmers" trying to touch important software could be catastrophic in many cases.) It's kind of like the groups of script-kiddies who consider themselves "hackers" but really aren't doing anything but utilizing tools real hackers have created. This is how NoFlo (from the article) strikes me.
  - dragonwriter 13y ago
    > The important distinction is that knowing how to read and write is a basic skill everyone needs in order to be an effective human being. I don't think that's true. Its something people may need to be (or, at least, greatly benefit from in being) effective members of modern society, but all the people that weren't part of the literate minority when literacy was rare (or before writing existed) were not "ineffective human heings".

codegeek 13y ago

"His company is busy resurrecting flow-based programming with a framework called NoFlo, an implementation of FBP for NodeJS" Something interest to note. The name of the framework is NoFlo that sounds like NoFlow And it is a framework for flow based programming. Wonder why they came up with that name.

bergie 13y ago
Now NoFlo works on both Node.js and browsers... But when I started it, I targeted only the former. So, Node.js Flow -> NoFlo
meemoo 13y ago
It comes from "Node Flow." Or reverse psychology?

gruseom 13y ago

I have two questions. First, what exactly is Flow-Based Programming? Emphasis on exactly. I know what dataflow is and I know who J. Paul Morrison is and I've even browsed through his book. I still can't figure it out, and the book (at least the parts I looked at) had too many diagrams and not enough code to clarify this most basic of questions. Every discussion on it seems hopelessly hand-wavey. What I want is the diffs between FBP and other computational models. Just the diffs, please! Or is it just dataflow? In that case, which version of dataflow? There have been many. Second, we know from the history of these things that it's easy to sell managers and journalists on boxes-and-lines visual programming. The pictures look a lot easier to understand than reams of source code do, but that's always because the examples are trivial—invariably some variant of a box with 2 in it, another box with two in it, arrows leading from those to a box with + in it, and then an arrow leading to a box with 4 in it. These always turn out to be a siren song because they don't scale to application complexity. How exactly is FBP different? Emphasis on exactly.

enraged_camel 13y ago
>>I have two questions. First, what exactly is Flow-Based Programming? Emphasis on exactly. Let the confusion flow through you!
drcode 13y ago
The crude/simple explanation I think is to imagine your program like an Excel spreadsheet: Some cells on the spreadsheet contain your client-side data (i.e. your "model"). Other cells are used by a javascript library to decide what's in the DOM (i.e. your "view") The "flow-based programming" model involves creating all the formulas that shuffle the data from the model to the view. At least that's what I can gather from what I've read (and I agree it's always been a big fail in the past, though there may be some merit to this idea eventually in limited roles.)

dragonwriter 13y ago

Aren't there like eleventy-bajillion enterprise tools that include exactly this kind of visual, dataflow-oriented methods of constructing large systems by wiring together individual components? Its not exactly an "arcane" model that's been lost in the mists of time since its use in "1970s banking software". And its very much not something that's been "suppressed by computer scientists for decades".

brianpaxe 13y ago

I thought this quote from the article was interesting: Kuhn’s book on scientific revolutions includes a famous quote from physicist Max Planck about what really causes paradigm shifts: "A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it."

RogerL 13y ago
Except, of course, scientists change their minds all the time based on the evidence. Look at all the work in QM in the 20s and 30s. These guys had competing hypothesizes, and some won out based on the evidence. The people that held the opposing view? They quickly accepted the errors of their ways. This is known as the bozo the clown argument. You know, somebody says "My perpetual motion machine will work! They laughed at Einstein, you know". And the reply is "yes, and they also laughed at bozo the clown". Almost always, if everyone is against your idea, having tried it, you are probably wrong. There is probably no industry on the planet more open to change. We've gone from procedural to OO, functional programming is having a huge resurgence, static typed to dynamic typed, waterfall to extreme/agile/whatever, and so on. And, despite the claims of the article, data flow programming has been alive and well the whole time. It hasn't been used much because of all the things that we have raised each time the topic is brought up, with no real response other than claims that we just don't get it, that people disagreeing is a sign that you are doing something right, and so on. I don't find it convincing at all. It's time to retire that Kuhn/Planck quote.
- gruseom 13y ago
  It's time to retire that Kuhn/Planck quote Oh, I don't think so. As its presence in Kuhn's book suggests, the kind of 'scientific truth' Planck was talking about is not just any hypothesis, but the much larger thing that we now (post-Kuhn) call a paradigm. For a counterexample, you'd need to show a paradigm shift occurring within the individual careers of the most established scientists in a community. QM isn't an example of that, for two reasons: it was the work of a new generation, and the debates you refer to were taking place within the new paradigm [1]. Your example is ironic, since it was Planck who put the Q in QM. If the history of QM refuted the quote as easily as all that, he'd never have said it in the first place. (I do agree with you that most claims of a new paradigm turn out to be false.) [1] That's not quite correct; they also took place before the new paradigm had crystallized—a time when (as Kuhn describes it) old models have broken down and lots of big things are up for grabs. Under such circumstances, minds change more fluidly. But such transitional circumstances are relatively rare and Planck was talking about the normal ones.

deleted 13y ago

[deleted]

protomyth 13y ago

So, I should use BizTalk Server[1][2] for my website. I'm not saying it's a bad idea, but I get the feeling it will be a hard sell. 1) http://www.microsoft.com/en-us/biztalk/default.aspx http://www.microsoft.com/en-us/biztalk/default.aspx 2) http://msdn.microsoft.com/en-us/library/aa577881.aspx http://msdn.microsoft.com/en-us/library/aa577881.aspx

dangoor 13y ago

For people looking for something more substantial, you might enjoy this c2 page about FBP vs. Actors: http://c2.com/cgi/wiki?ActorsAndFlowBasedProgrammingDiscussion http://c2.com/cgi/wiki?ActorsAndFlowBasedProgrammingDiscussi... The basic idea with FBP is that you have multiple processes that communicate over channels (kind of like actors, but the communication is unidirectional across those channels). I'm not a fan of GUIs like this generally, but the idea of FBP is interesting in the same way that the actor model is interesting.

a0 13y ago

I think programmers are always thinking about the flow of programs. Jumping through blocks of statements, following the chains of function calls and analysing their definitions when needed. The _flow_ is there, but it's not visual. We are always imagining what would the compiler or interpreter do with our code. On the other hand, we must be able to quickly navigate in the source tree. That's why text editors like Vim, that understand the contents of the programs (buffers, text objects, definitions), are so powerful. The same happens with Xcode, for example. The Interface Builder is a graphical tool that defines the code components as visually manageable objects. Maybe the proposed paradigm could extend this ideas of representing the code in flows, as graphs of objects. I'm really curious to try it. Mainly because it seems to feet really well with handheld devices.

pcunite 13y ago

These guys should talk to the creators of VNOS 1.1 and ask how that went. Visual programming does have its place. My mom was able to make a website using Weebly without any help. However, as things get complex I personally can visualize (in my mind) the interconnects. In my opinion, we need easier to use libraries, not visual tools. But I genuinely want to be corrected.

deleted 13y ago

[deleted]

shortcj 13y ago

This read like a long winded video sales pitch on the unbeatable game plan for seducing women.

it_learnses 13y ago

"The paradigm was so disruptive that it was suppressed by computer scientists for decades." Stopped reading after that.

keeran 13y ago

There are multiple communities of 'creative coders' who already work with this paradigm using tools like vvvv[0], MaxMSP[1], PureData[2] & others. Closer(ish) to home we have QuartzComposer[3] available for OSX. I like all of these tools but can't help feeling I'd be more effective if I could just get at the underlying code half the time (in cases where that's not an option). [0] http://vvvv.org/ http://vvvv.org/ [1] http://cycling74.com/ http://cycling74.com/ [2] http://puredata.info/ http://puredata.info/ [3] http://developer.apple.com/graphicsimaging/quartz/quartzcomposer.html http://developer.apple.com/graphicsimaging/quartz/quartzcomp...

deleted 13y ago
[deleted]
lstamour 13y ago
Seconding Max/MSP, you can write C code in it thanks to "gen".
heuermh 13y ago
> I'd be more effective if I could just get at the underlying code For something similar to MaxMSP/PureData but with writing code in a text editor, you may wish to take a look at ChucK [1]. Instead of graphical edges or links between between UGens ChucK provides the '=>' operator. E.g. adc => Chorus c => LPF lpf => Echo e => Delay d => dac; [1] http://chuck.cs.princeton.edu http://chuck.cs.princeton.edu
- melloclello 13y ago
  I took a class in ChucK once. It is moderately terrible and half-finished but I managed to write a patch once while blackout drunk, so I'd say it's pretty intuitive.
aufreak3 13y ago
> I like all of these tools but can't help feeling I'd be more effective if I could just get at the underlying code half the time (in cases where that's not an option). I've seen novices do cool stuff with Max without thinking that. Perhaps this feeling comes more from familiarity with textual coding than something fundamental about textual coding. On the other hand, we keep saying "a picture is worth a thousand words", but here we are writing blog posts in gobs! That suggests that text is what we train for, and is what we might be most effective with in the future too.

jimfuller 13y ago

ya, my eyes gloss over when I read 'rediscoveries' like this ... I think the finer point is more about workflow then programming. That is state based workflow is fundamentally harder for developers to grok, also maintaining state is a kind of fallacy when scaling out and/or up. Flow based workflows, while probably can't cover easily the breadth of exotic corner cases that FSM can achieve, are much easier to understand and state is byproduct of the system, not to mention that a full record of mutation occurs with a copy of data at every step (similar to MVCC - Multi-value-consistency-control) This is similar to how http://www.w3.org/TR/xproc/ http://www.w3.org/TR/xproc/ XProc works, which has a serious implementation in http://xmlcalabash.com/ http://xmlcalabash.com/ and slowly gaining adoption in XML circles. Its nice to see yet another XML technology, which of course is based on older established concepts in computing, being emulated/replicated in the JS universe ... shame about the visual programming antipattern/lunacy.

falcolas 13y ago

Based solely on my previous experience with a "flow-based" programming system, which endeavored to make programming financial decision logic a matter of connecting boxes, it's the 80/20 problem. It makes 80% of the work dirt simple and able to be completed. The remaining 20% still needs to be implemented in code, and takes most of the time. It got so bad that they actually ended up implementing "code" objects, just so the code could be edited and incorporated into directly into the tool instead of around it.

meemoo 13y ago
I'm a JS hacker working on the NoFlo UI, and one of our primary design goals is to make it easy to code new modules on the fly (in the same environment, with tests) when that makes more sense.

oakwhiz 13y ago

The advantage of this method of programming is that complex parallel processing tasks can be reduced to a directed acyclic graph from information sources, through filters, and into information sinks. You can then take this graph and either interpret it by creating threads for each of the nodes in the graph and shuffling information between them using FIFO queues, or try to compile the graph to a better performing, but "flattened" native program. The main problems with this approach are how to properly represent timing, latency, and node state, since you can send one piece of information through the graph at a time, or you can try to improve efficiency by pipelining the graph - using the graph like a shift register, and making sure every node has work to do at all times. Of course it's not a cure-all solution and doesn't map well to every problem domain, but it is generally easier to reason about for complex parallel processing tasks, and can be deployed over multiple machines with minimal "glue" code.

zhemao 13y ago
So, hardware description languages, which are essentially flow-based programming languages, solve this using clocks and sensitivity lists. An FBPL for web programming would probably use events instead of edge/level triggers and timers instead of clocks.

lukifer 13y ago

You know what costs us sanity? It's not code. It's people. Scope creep. Last-minute specifications. Pointy-hairs who don't understand technical debt, and need everything done yesterday. Design by committee. Too many cooks in too many kitchens. Legacy processes, legacy data, legacy user habits. Unrealistic expectations, unrealistic timelines, and unrealistic budgets. Marginally competent coworkers. Office politics. Compared to those, callback hell and and classitis are a dream come true. And while all the problems above are reducable with the right processes, you'll never shrink them all to zero. "Algorithms may have mathematical underpinnings, but computer programming is a behavioral science." - @brixen

jwatte 13y ago

I tried a flow based tool for the Mac in late '80s. It really didn't make things easier - and it had terrible support for things that text languages take for granted, like foreign library import. Computer scientists have not "hidden away" the idea. It simply hasn't provided close to the value that text based languages have in real life. The new tool is as much snake oil as any tool before it that uses the same language in its advertising.

chris_mahan 13y ago

being a good programmer is not about knowing how to handle the minute details; it is about know how to handle complexity.

spellboots 13y ago

"Let's get non-musicians making music! We'll just give them a nice MIDI keyboard instead of that hard-to-use violin" Sure, this will skip over some tricky bits of getting started with programming, just as a MIDI keyboard can make it easier to make violin sounds without having to learn all that pesky fingering. However, as with playing the violin, those bits aren't really the hard part to learn; the easiest to use keyboard in the world won't make you a composer and flow based programming is not going to make you a programmer.

meemoo 13y ago
MIDI tools enabled more people to make new kinds of music. Not everybody will be a programmer, but hopefully a slightly larger percentage of the population will be able to discover the power of algorithmic thinking, and hopefully they will work on some interesting problems.

6ren 13y ago

The fundamental problem with visual programming methods as opposed to linguistic methods is that abstraction doesn't work as well. I agree that we'll get glue programmers, who just stick modules together. We already do: they are those programmers who use APIs. i.e. all of us to some extent. So far, it's still programming.

dclowd9901 13y ago

At this point, I see the use of Flow-based programming to be analogous to something like Titanium (the JS framework that lets you build iOS apps). It's interesting in concept, and will most assuredly open the doors to allow a larger swath of people to participate, but for anything of consequence, you've still gotta buckle down and write some code. The reason is almost entirely about performance optimization, which is something you just can't get from compiling from GUI.

wwweston 13y ago

> What we need is not more programmers. What we need is to enable non-programmers to participate in the creation process, not just the ideation process Why don't we need more programmers participating in the ideation process, not just the creation process?

stormbrew 13y ago

> The paradigm was so disruptive that it was suppressed by computer scientists for decades. Did someone not get their "suppress the disruptive programming technique" kickback cheque in the mail this month?

richardjordan 13y ago

People waiting for "the breakthrough" that suddenly makes programming easy and available to all are kidding themselves. As others have said there's inherent complexity to programming and solving problems that can't be wished away with paradigm changing strategerized facilitations utilized by flow-optimized programmerators or whatever the secret code knowledge suppressed by scientists was prescribing. On the broader point - and the comments by folks saying we should have made things better by now: we have. I started out learning to program with zero help, no tools, few manuals on an Apple II in some flavor of BASIC or machine code by trial and error. Decades later having not coded a lick in more than 15 years I picked up Ruby On Rails quickly and easily (enough to get out MVP) and with the help of Stack Overflow and Google have spent the last three years getting better and better at it. I can do dramatically more today than I could have imagined in my days alone with an Apple II and a poor english translation of a poor chinese translation of a pirated Apple manual. It's all better. The worst programmers have super powers available to them (they don't always use them admittedly) compared to prior days. Things are getting better but they're earning it, not because "THEY" don't want you to know "SECRET KNOWLEDGE".

melloclello 13y ago
> People waiting for "the breakthrough" that suddenly makes programming easy and available to all are kidding themselves. As others have said there's inherent complexity to programming and solving problems that can't be wished away with paradigm changing strategerized facilitations utilized by flow-optimized programmerators or whatever the secret code knowledge suppressed by scientists was prescribing. I totally agree with you and would go so far as to say there's probably a kind of upper bound to the rate at which a human can translate an original idea in their head into an unambiguous logical construct of the kind which can be executed by a computer. However, I would also say that the narrowest bottleneck in this process is the rate at which someone can comprehend the current state and structure of a given program, and I would say that anything that tries to improve on the status quo of text files in a directory is pretty cool.
- Roboprog 13y ago
  I miss call trees. I used to have tools to generate call tree diagrams for procedural languages, back in the 80s. With virtual/dynamic methods, IDEs don't bother to try to make such a feature anymore.

drivebyacct2 13y ago

Polymer Project.

hcarvalhoalves 13y ago

> Today’s web programmers grapple with problems that people in the early days never had to deal with. They’re building complex UIs, juggling a bunch of APIs, and running multiple processes at the same time. All of these tasks require mastering the flow of data between application components in real-time, something which even the most advanced developers struggle with. No. The problem is not moving data around. The biggest problem on web development is the drive from the industry to shoe-horne applications on top of platforms that weren't meant for it.

now_what 13y ago

FWIW the visual graph style IDE is used to write scripts for 3d design programs Rhino 3d (Grasshopper plugin [0]) and Revit (Dynamo). It's pretty popular among architects and the like. [0] http://en.wikipedia.org/wiki/Grasshopper_3d http://en.wikipedia.org/wiki/Grasshopper_3d

drcode 13y ago

FYI, the Clojure Pedestal Client-side framework has a similar update model to this.

MarcScott 13y ago

I was using Xpresso in Cinema 4D years ago, which looks remarkably similar. http://www.maxon.net/products/cinema-4d-prime/customizing.html http://www.maxon.net/products/cinema-4d-prime/customizing.ht... This is nothing new and neither has it been "suppressed".

Raphmedia 13y ago
I agree. A lot of software use that kind of visualization. It's nothing new.

marshray 13y ago

Scratch, Lego Mindstorms, DirectShow audio graph editor, any number of lab and process control systems.

abawany 13y ago

This reminded me in some way of FASTech CELLworks (sic): software used in manufacturing automation to define process flows. It had some interesting features such as message passing via a mailbox abstraction, concurrent execution, and a memory-based data store.

dbond 13y ago

We're building something very similar, aiming to have demos and beta by new year. (Bad) signup page here if interested: http://www.lexasapp.com/ http://www.lexasapp.com/

romansanchez 13y ago

Has any body heard of TIBCO? They've been doing this (flow-based programming) for years(>10).

shepbook 13y ago

There are two points I don't get with such a system. 1) The logic/functions needed in each "black box" still need to be written. How is that handled? Is that where the line of "programmer vs non-programmer" is crossed? 2) Programming is, more or less, the skill of breaking down a problem into the parts needed for its solution and providing the solution. The value of a programmer isn't the ability to type in code, so much as the ability to logically break down the problem and solve it. How would such a programming paradigm actually solve that problem?

meemoo 13y ago
The NoFlo team is coders designing this for our own work first. We are free to put as much code in a black box as we like. We'll be running tests with ourselves and our early community to make some best practices, but there is no rule that keeps you from writing most of your logic in one block. Our theory is that the graph-based view will encourage encapsulation in different ways that make sense for the different domains that adopt the tool. One of the big design differences (vs Quartz Composer, Lab View, Pure Data) is that it will be easy to dive in and edit component code and make new components when needed.

zhemao 13y ago

Okay, yeah, so I agree with most of the criticisms here. Flow-based languages didn't catch on because no-one came up with a really compelling general-purpose implementation that scales up to large, complex projects. It's not because it was actively suppressed by the programmer Illuminati to keep our salaries high. I think the main reason FBP hasn't gone mainstream is because most implementations are built around a graphical programming environments (e.g. LabView, Pure Data, Scratch) in order to cater to non-programmers. But programmers love their text files, and for plenty of good reasons. Code represented as text can be edited by any general-purpose text editor, work well with version control systems, and can (usually) be easily split into separate files. It's probably instructive, then, to look at a domain in which textual flow-based programming languages have become mainstream. I'm speaking, of course, about Hardware Description Languages, which are used by digital circuit designers to simulate and prototype complex circuits, like CPUs. The two dominant HDLs are VHDL and Verilog. Both of these languages use a flow-based model, in which logical blocks called entities are connected to each other through input and output ports. The reason these languages work is because the domain is a good fit for FBP and because they are designed for the domain experts (circuit designers) to use, and not for some non-technical manager to be able to understand. Taking a similar approach to web programming may work. Logical blocks connected through ports. Except instead of ones and zeroes going through these ports, you have I/O events and data. I haven't looked at NoFlo, but it might be interesting. Note that I'm not saying that VHDL and Verilog are good languages, even for their intended domain. They have plenty of warts, and a FBP language for software that basically copies one or the other would be terrible (especially VHDL; oh god, please nobody make VHDL for web programming). But the entity-port model is essentially sound, and I would be quite interested in a general-purpose language that adapts that model in a well thought-out way.

keithnoizu 13y ago

I doubt flow based programming is going to let the nonprogrammer program but I do think there is some potential in the age of the tablet to get stuff done visually with flow based development and touch friendly tools provided you can easily switch back and forth to source. E.g. something sort of like windows worfklow on speed.

leke 13y ago

When I saw this I immediately thought of [Drakon](https://en.wikipedia.org/wiki/DRAKON https://en.wikipedia.org/wiki/DRAKON).

rbanffy 13y ago

It's a real tragedy people who don't know history can't learn from it.

Too 13y ago

I looked at all the pictures in the article and couldn't imagine how to draw something as fundamental as an if/else-statement. Think about that for a moment. http://www.maxon.net/uploads/pics/xpresso_17.jpg http://www.maxon.net/uploads/pics/xpresso_17.jpg shows an alternative that i imagine could be used to lazily evaluate only certain connected boxes but still...just look at that mess. Please don't remind me how if-statments in labview are represented. It gives me nightmares.