4 ms·

Just one caveat to the use of core.async as a processing pipeline (as in section 6: Callbacks), make sure you're taking extra care to handle exceptions. Since

by dharbin 12y ago

Just one caveat to the use of core.async as a processing pipeline (as in section 6: Callbacks), make sure you're taking extra care to handle exceptions. Since the exception occurs in a separate thread, it will happen silently and will cause your pipeline to stall. Some recommended reading for how to handle such a scenario: http://martintrojer.github.io/clojure/2014/03/09/working-with-coreasync-exceptions-in-go-blocks/ http://martintrojer.github.io/clojure/2014/03/09/working-wit... https://github.com/zachallaun/async-pipeline https://github.com/zachallaun/async-pipeline https://github.com/ztellman/manifold https://github.com/ztellman/manifold

bguthrie 12y ago
Exception handling is currently my biggest complaint with core.async, and I've gained a fair amount of sympathy for the idea of encapsulating asynchronous operations inside an object that can accommodate failure states, like a future (though Clojure futures don't).
lkrubner 12y ago
There are Clojure systems with good support for handling exceptions. Zach Tellman's has an event driven channel/pipeline system in Clojure called Lamina. It offers good support for monitoring and recovering from exceptions: https://github.com/ztellman/lamina/wiki/Probes-and-Instrumentation https://github.com/ztellman/lamina/wiki/Probes-and-Instrumen... Any system that is concurrent needs first-class support for handling exceptions that occur in a thread pool. I appreciate that Lamina puts an emphasis on this.
jimbokun 12y ago
This seems a good reason to consider the Erlang model, of processes that fail early and exit on any error, with monitor or supervisor process to restart it. Are there any Clojure libraries that do something like this? I think Akka supports this kind of design.
- swannodette 12y ago
  This is the essence of the non-problem. core.async by not choosing a default and acting as substrate allows developers to choose between a supervisor-like model in which go processes can just die and be restarted or an exception oriented one that propagates errors through channels. This means it's less "easy" since you have to think about this up front. But it also means you are not pigeon-holed into working around canned defaults when constructing the appropriate solution for the problem at hand.
  - felixgallo 12y ago
    I get this viewpoint, and it makes surface sense to me. That said, essentially all of the non-hello-world CSP code, in go or clojure, that I've seen ends up trying to implement call/cast/monitor at a minimum, and frequently gen_server/gen_supervisor/gen_fsm. Seems like a problem.
    - swannodette 12y ago
      I've seen plenty of Clojure core.async code that doesn't look like this at all. Also this discussion isn't considering the other reason for core.async's existence - ClojureScript - the patterns are completely different.
- ubolonton_ 12y ago
  I think Pulsar supports this model: http://docs.paralleluniverse.co/pulsar/#error-handling http://docs.paralleluniverse.co/pulsar/#error-handling. Incidentally, in the javascript port of core.async, we are also trying to come up with a good error handling story [1]. One approach we are considering is (sort of) adapting the Erlang model, by treating goroutines' return channels as their identities. I'm researching error handling in CSP literature and prior implementations as well. [1]: https://github.com/ubolonton/js-csp/issues/14 https://github.com/ubolonton/js-csp/issues/14

by dharbin 12y ago

bguthrie 12y ago

Exception handling is currently my biggest complaint with core.async, and I've gained a fair amount of sympathy for the idea of encapsulating asynchronous operations inside an object that can accommodate failure states, like a future (though Clojure futures don't).

lkrubner 12y ago

There are Clojure systems with good support for handling exceptions. Zach Tellman's has an event driven channel/pipeline system in Clojure called Lamina. It offers good support for monitoring and recovering from exceptions: https://github.com/ztellman/lamina/wiki/Probes-and-Instrumentation https://github.com/ztellman/lamina/wiki/Probes-and-Instrumen... Any system that is concurrent needs first-class support for handling exceptions that occur in a thread pool. I appreciate that Lamina puts an emphasis on this.

jimbokun 12y ago

This seems a good reason to consider the Erlang model, of processes that fail early and exit on any error, with monitor or supervisor process to restart it. Are there any Clojure libraries that do something like this? I think Akka supports this kind of design.

swannodette 12y ago
This is the essence of the non-problem. core.async by not choosing a default and acting as substrate allows developers to choose between a supervisor-like model in which go processes can just die and be restarted or an exception oriented one that propagates errors through channels. This means it's less "easy" since you have to think about this up front. But it also means you are not pigeon-holed into working around canned defaults when constructing the appropriate solution for the problem at hand.
- felixgallo 12y ago
  I get this viewpoint, and it makes surface sense to me. That said, essentially all of the non-hello-world CSP code, in go or clojure, that I've seen ends up trying to implement call/cast/monitor at a minimum, and frequently gen_server/gen_supervisor/gen_fsm. Seems like a problem.
  - swannodette 12y ago
    I've seen plenty of Clojure core.async code that doesn't look like this at all. Also this discussion isn't considering the other reason for core.async's existence - ClojureScript - the patterns are completely different.
ubolonton_ 12y ago
I think Pulsar supports this model: http://docs.paralleluniverse.co/pulsar/#error-handling http://docs.paralleluniverse.co/pulsar/#error-handling. Incidentally, in the javascript port of core.async, we are also trying to come up with a good error handling story [1]. One approach we are considering is (sort of) adapting the Erlang model, by treating goroutines' return channels as their identities. I'm researching error handling in CSP literature and prior implementations as well. [1]: https://github.com/ubolonton/js-csp/issues/14 https://github.com/ubolonton/js-csp/issues/14