4 ms·

How does the parallel "execution strategy" work? Do you specify a maximum concurrency level? Can you distribute the computation onto other machines?

• w01fe 13y ago
  Graph is currently just in-process (no cross-machine distribution), although it's definitely a possibility down the line. The currently released parallel strategy is also just a pedagogical example, not really meant to be used. Here's some WIP on a more practical parallel compilation: https://gist.github.com/w01fe/4710008 https://gist.github.com/w01fe/4710008 Once the kinks get worked out this will go into the OSS project. Presumably concurrency level will be controlled by a parameter, and/or passing an appropriate ExecutorService.
  • jared314 13y ago
    It would be very interesting if you could easily convert a graph into a Storm[0] topology. An example of the Storm Clojure DSL is available in the storm-starter project[1]. [0] http://storm-project.net/ http://storm-project.net/ [1] https://github.com/nathanmarz/storm-starter/blob/master/src/clj/storm/starter/clj/word_count.clj https://github.com/nathanmarz/storm-starter/blob/master/src/...
• batbomb 13y ago
  I'm not sure how it's implemented in Graph, but the best thing you can do is to abstract out an execution to a general object. From there, you can create all sorts of adapters and the "graph" engine just farms out the execution to the appropriate adapter. This could be anything from a job daemon on a batch farm, or, say, a typed ThreadPoolExecutor on a machine (which can either execute native clojure/java, or any sort of other script language.
  • thelittlelisper 13y ago
    This would be brilliant. Does any other framework support this? (outside Clojure)