4 ms·
Dataflow does something need barriers and synchronization if you have to deal with parallel data sources. If you have sources of data coming in independently of
by scott_s 15y ago
Dataflow does something need barriers and synchronization if you have to deal with parallel data sources. If you have sources of data coming in independently of each other, you sometimes have to contend with the fact that some of the paths might be faster than the other, and the right data might not be able to be paired without some form of synchronization.
Even if your dataflow application does not need synchronization, then you need to be able to reason about the fact that you have inherent asynchrony when you go to look at your results. That is, you may see this item and that item paired together - is that a valid result? The kind of reasoning required to figure that out is similar to what's required in, say, multithreaded programming.
- jerf 15y agoIMHO, the real breakthrough with non-threading-based concurrency like message passing or STM or any of the other more recent primitives isn't that they make concurrency issues "go away"; it is that they reduce the complexity of implementing concurrent programs from exponential in the number of instructions to polynomial in the number of instructions. You'll probably never get to entirely stop thinking about race conditions, but they're much easier to deal with in Erlang, for instance, where there's only a very limited number of ways to create a race condition, as opposed to how there's only a very limited number of ways to fail to create a race condition in imperative-mutable threaded programming. It isn't made trivial, if you try to push it isn't necessarily even easy, but it is made feasible.
- Retric 15y agoYou don't need to be using Earlang to greatly simplify parallel coding though message passing. A lot of fairly clean C code started by using the man thread spawns worker thread who pushes results to a message queue. And then got messy because they stopped using messages for some side channel communication.
- jerf 15y agoWhat Erlang brings you is precisely that... "And then got messy because they stopped using messages for some side channel communication." ... can't happen in pure-Erlang code. Of course there's nothing you can do in Erlang or Haskell that you can't do in pure C, but the problem is that if you are working in C you can't ever quite be sure that something somewhere accidentally mutated a value, when the language makes it so easy. (And let's not talk about C++.) Some of the Haskell leaders call it "wearing the hair shirt". This is extended agreement, by the way, not a disagreement. If you are stuck in C, there are far worse things you can do that try to impose your own message-passing paradigm, just as there are an awful lot of object-oriented C programs in the world.
- Retric 15y ago"an awful lot of object-oriented C programs in the world" IMO, the good parts of object-oriented programming was applying the constraints of mulithreaded messaging passing to the single threaded world. Really clean multithreaded code tends to be extremely decoupled and treat each message is something to be decoded not blindly followed. Yet somehow that fell to the point where using lots of getter/setter are considered acceptable. PS: That could actually be a good metric how many objects do you use that don't have a single getter or setter.
- scott_s 15y agoI agree. My problem is when people say that you don't have to consider concurrency and synchronization at all. And I also agree that such abstractions make the reasoning significantly easier - much as reasoning about the flow of a sequential program is significantly easier with structure programming than with code that has gotos all over the place. In fact, I made a similar argument in the conclusion of my dissertation (first full paragraph on page 96): http://people.cs.vt.edu/~scschnei/papers/scott_dissertation.pdf http://people.cs.vt.edu/~scschnei/papers/scott_dissertation....
- kenjackson 15y agoIt's interesting in that in the supercomputing world the message passing model has been popular for decades, but not by choice. It was the only way to get good performance. But the holy grail has always been shared memory, not message passing. But perf for shared memory applications has continued to be horrible. But anyone who has experience writing a message passing and a shared memory version almost always concedes the shared memory version is easier. Large scale parallel message passing apps are extremely difficult to get right. Most people just haven't done it. With that said, some of the difficulties in the past where tied to the fact that message passing was done with a weak type system, no contracts (I sent you message, but how do I know you're ever going to respond to it?), and weak support for gather/scatter. AFAICT, not having done much at all with Erlang, it deals nicely with the type system issue, but contracts are still a problem. Gather/scatter is partially assisted in the same way that PM/FM handled it in the past (you get to write code to pull messages out of your mailbox). My prediction is that if message passing does take off in a big way, we'll see a pretty strong backlash to shared memory with functionality, such as type ownership and data representation synthesis. Unfortunatley, most of this research is ignored in favor of the more popular functional work (which in itself is good, just not currently balanced in the language community by other types of thinking).
- scott_s 15y agoYour last paragraph is talking about the languages community only, correct? In my experience, functional programming is still exotic in the HPC world.
- kenjackson 15y agoYes. It's becoming less exotic in the HPC world, but still greatly lags in popularity compared the languages community world.
- yvdriess 15y agoDataflow means a language or machine who's execution is driven by the availability of data. In a dataflow computational model you do not have the concept of memory. In a Von Neumann architecture (viz. your current CPU) execution is determined by the previous instruction. Data is stored in a global, randomly accessible memory space. Synchronizing two data streams in dataflow is just a matter of having an instruction with two inputs. The very semantics of your machine tells you the node's instruction will only trigger when both its inputs are available. Critical systems with the problems you described are often written in dataflow languages for exactly these reasons. Citing wikipedia: Lustre is a formally defined, declarative, and synchronous dataflow programming language for programming reactive systems. [...] It is now used for critical control software in aircraft, helicopters, and nuclear power plants.
- scott_s 15y agoDataflow hardware is not the only realization of the dataflow programming model.