5 ms·

Master Concurrent Processes with core.async

131 points by swannodette 12y ago

pilif 12y ago
I absolutely love @nonrecursive's writing style. This is one of those books that are a pleasure to read and as such it's very easy to learn (and keep) the information given. Once this is out in dead-tree form I will definitely get a copy or three (there are geeks around me who will enjoy it too). This isn't just a very good introduction into Clojure. This is a real piece of art.
- nonrecursive 12y ago
  This made my day. Thank you :)
dharbin 12y ago
Just one caveat to the use of core.async as a processing pipeline (as in section 6: Callbacks), make sure you're taking extra care to handle exceptions. Since the exception occurs in a separate thread, it will happen silently and will cause your pipeline to stall. Some recommended reading for how to handle such a scenario: http://martintrojer.github.io/clojure/2014/03/09/working-with-coreasync-exceptions-in-go-blocks/ http://martintrojer.github.io/clojure/2014/03/09/working-wit... https://github.com/zachallaun/async-pipeline https://github.com/zachallaun/async-pipeline https://github.com/ztellman/manifold https://github.com/ztellman/manifold
- bguthrie 12y ago
  Exception handling is currently my biggest complaint with core.async, and I've gained a fair amount of sympathy for the idea of encapsulating asynchronous operations inside an object that can accommodate failure states, like a future (though Clojure futures don't).
- lkrubner 12y ago
  There are Clojure systems with good support for handling exceptions. Zach Tellman's has an event driven channel/pipeline system in Clojure called Lamina. It offers good support for monitoring and recovering from exceptions: https://github.com/ztellman/lamina/wiki/Probes-and-Instrumentation https://github.com/ztellman/lamina/wiki/Probes-and-Instrumen... Any system that is concurrent needs first-class support for handling exceptions that occur in a thread pool. I appreciate that Lamina puts an emphasis on this.
- jimbokun 12y ago
  This seems a good reason to consider the Erlang model, of processes that fail early and exit on any error, with monitor or supervisor process to restart it. Are there any Clojure libraries that do something like this? I think Akka supports this kind of design.
  - swannodette 12y ago
    This is the essence of the non-problem. core.async by not choosing a default and acting as substrate allows developers to choose between a supervisor-like model in which go processes can just die and be restarted or an exception oriented one that propagates errors through channels. This means it's less "easy" since you have to think about this up front. But it also means you are not pigeon-holed into working around canned defaults when constructing the appropriate solution for the problem at hand.
    - felixgallo 12y ago
      I get this viewpoint, and it makes surface sense to me. That said, essentially all of the non-hello-world CSP code, in go or clojure, that I've seen ends up trying to implement call/cast/monitor at a minimum, and frequently gen_server/gen_supervisor/gen_fsm. Seems like a problem.
      - swannodette 12y ago
        I've seen plenty of Clojure core.async code that doesn't look like this at all. Also this discussion isn't considering the other reason for core.async's existence - ClojureScript - the patterns are completely different.
  - ubolonton_ 12y ago
    I think Pulsar supports this model: http://docs.paralleluniverse.co/pulsar/#error-handling http://docs.paralleluniverse.co/pulsar/#error-handling. Incidentally, in the javascript port of core.async, we are also trying to come up with a good error handling story [1]. One approach we are considering is (sort of) adapting the Erlang model, by treating goroutines' return channels as their identities. I'm researching error handling in CSP literature and prior implementations as well. [1]: https://github.com/ubolonton/js-csp/issues/14 https://github.com/ubolonton/js-csp/issues/14
alexatkeplar 12y ago
Good timing - I'm halfway through the core.async chapter in Seven Concurrency Models in Seven Weeks, which is well worth a read too...
- krat0sprakhar 12y ago
  Had no clue about there's actually a book on this topic! No more googling around differences between actors/goroutines/threads. Thanks a ton.
  - Matthias247 12y ago
    I'm currently also at reading it. It's definitly worth it, gives a good overview on different concurrency solutions. However if you want to have a really deep dive into actors/core.async/threading-primitives then more specialised books might be more helpful.
hardwaresofton 12y ago
If you're trying to learn clojure, read this book. Maybe even first. Agree with what others have said, Clojure for the Brave and True is a fantastic book, was excited when I got the leanpub email this morning alerting me to a new chapter.
jdimov 12y ago
Or just use Elixir.
StavrosK 12y ago
I tried to follow this (as a Clojure newb) and got stuck on the second code segment. Apparently nothing can find a "chan" function...
- nonrecursive 12y ago
  You'll need to make sure you require/refer the core.async namespace according to the preceding code block. If you have the code there, make sure you re-evaluate the file. Here's a resource on require and refer: http://www.braveclojure.com/organization/ http://www.braveclojure.com/organization/ If you still have trouble feel free to leave a comment on the site or tweet me - I'll be able to respond more quickly that way :)

131 points by swannodette 12y ago

pilif 12y ago

I absolutely love @nonrecursive's writing style. This is one of those books that are a pleasure to read and as such it's very easy to learn (and keep) the information given. Once this is out in dead-tree form I will definitely get a copy or three (there are geeks around me who will enjoy it too). This isn't just a very good introduction into Clojure. This is a real piece of art.

nonrecursive 12y ago
This made my day. Thank you :)

dharbin 12y ago

Just one caveat to the use of core.async as a processing pipeline (as in section 6: Callbacks), make sure you're taking extra care to handle exceptions. Since the exception occurs in a separate thread, it will happen silently and will cause your pipeline to stall. Some recommended reading for how to handle such a scenario: http://martintrojer.github.io/clojure/2014/03/09/working-with-coreasync-exceptions-in-go-blocks/ http://martintrojer.github.io/clojure/2014/03/09/working-wit... https://github.com/zachallaun/async-pipeline https://github.com/zachallaun/async-pipeline https://github.com/ztellman/manifold https://github.com/ztellman/manifold

bguthrie 12y ago
Exception handling is currently my biggest complaint with core.async, and I've gained a fair amount of sympathy for the idea of encapsulating asynchronous operations inside an object that can accommodate failure states, like a future (though Clojure futures don't).
lkrubner 12y ago
There are Clojure systems with good support for handling exceptions. Zach Tellman's has an event driven channel/pipeline system in Clojure called Lamina. It offers good support for monitoring and recovering from exceptions: https://github.com/ztellman/lamina/wiki/Probes-and-Instrumentation https://github.com/ztellman/lamina/wiki/Probes-and-Instrumen... Any system that is concurrent needs first-class support for handling exceptions that occur in a thread pool. I appreciate that Lamina puts an emphasis on this.
jimbokun 12y ago
This seems a good reason to consider the Erlang model, of processes that fail early and exit on any error, with monitor or supervisor process to restart it. Are there any Clojure libraries that do something like this? I think Akka supports this kind of design.
- swannodette 12y ago
  This is the essence of the non-problem. core.async by not choosing a default and acting as substrate allows developers to choose between a supervisor-like model in which go processes can just die and be restarted or an exception oriented one that propagates errors through channels. This means it's less "easy" since you have to think about this up front. But it also means you are not pigeon-holed into working around canned defaults when constructing the appropriate solution for the problem at hand.
  - felixgallo 12y ago
    I get this viewpoint, and it makes surface sense to me. That said, essentially all of the non-hello-world CSP code, in go or clojure, that I've seen ends up trying to implement call/cast/monitor at a minimum, and frequently gen_server/gen_supervisor/gen_fsm. Seems like a problem.
    - swannodette 12y ago
      I've seen plenty of Clojure core.async code that doesn't look like this at all. Also this discussion isn't considering the other reason for core.async's existence - ClojureScript - the patterns are completely different.
- ubolonton_ 12y ago
  I think Pulsar supports this model: http://docs.paralleluniverse.co/pulsar/#error-handling http://docs.paralleluniverse.co/pulsar/#error-handling. Incidentally, in the javascript port of core.async, we are also trying to come up with a good error handling story [1]. One approach we are considering is (sort of) adapting the Erlang model, by treating goroutines' return channels as their identities. I'm researching error handling in CSP literature and prior implementations as well. [1]: https://github.com/ubolonton/js-csp/issues/14 https://github.com/ubolonton/js-csp/issues/14

alexatkeplar 12y ago

Good timing - I'm halfway through the core.async chapter in Seven Concurrency Models in Seven Weeks, which is well worth a read too...

krat0sprakhar 12y ago
Had no clue about there's actually a book on this topic! No more googling around differences between actors/goroutines/threads. Thanks a ton.
- Matthias247 12y ago
  I'm currently also at reading it. It's definitly worth it, gives a good overview on different concurrency solutions. However if you want to have a really deep dive into actors/core.async/threading-primitives then more specialised books might be more helpful.

hardwaresofton 12y ago

If you're trying to learn clojure, read this book. Maybe even first. Agree with what others have said, Clojure for the Brave and True is a fantastic book, was excited when I got the leanpub email this morning alerting me to a new chapter.

jdimov 12y ago

Or just use Elixir.

StavrosK 12y ago

I tried to follow this (as a Clojure newb) and got stuck on the second code segment. Apparently nothing can find a "chan" function...

nonrecursive 12y ago
You'll need to make sure you require/refer the core.async namespace according to the preceding code block. If you have the code there, make sure you re-evaluate the file. Here's a resource on require and refer: http://www.braveclojure.com/organization/ http://www.braveclojure.com/organization/ If you still have trouble feel free to leave a comment on the site or tweet me - I'll be able to respond more quickly that way :)