3 ms·

I am confused. If I remember correctly, go's runtime uses a thread pool as well. It's not like go somehow invented a runtime that does not rely on OS threads. S

I am confused. If I remember correctly, go's runtime uses a thread pool as well. It's not like go somehow invented a runtime that does not rely on OS threads. So in the end of the day, what difference does it make? If you ask me, java.util.concurrent is a much better and battle tested concurrent infrastructure than the simplistic implementation in go, because java.util.concurrent is actually written by the world's foremost concurrency expert, Doug Lee. I will take that over someone else's hand rolling concurrent runtime any day. God knows how many bugs there are in your home grown concurrency code.

armitron 6y ago
It makes a huge difference. You can't use blocking calls in core.async, since they'll block the entire thread. There is no switching when you do a blocking call, the thread your code runs on will be out of commission until the blocking call returns. So this means that blocking APIs are out of the question and this translates to you not being able to write linear, straightforward code. You have to use non-blocking APIs and callbacks, and rely on core.async channels to force switching between blocks (core.async doesn't have lightweight processes). This is a total nightmare to debug and forces you to bend your programming logic to support this non-optimal programming model. Constrast with Go goroutines or Erlang processes where you write linear, synchronous code that executes as part of lightweight processes. The built-in scheduler will automatically schedule these processes on real OS threads, and switch between them when they block (either through channels or blocking calls). OS thread utilization is far more effective, debugging is a piece of cake and the code that you write is easy to read and understand. I'm bypassing your "world's foremost concurrency expert" comments. I suggest you use the concurrency offered by Go and Erlang by spending a few hours to do tasks you've done with core.async. You will learn something and you will see how Clojure falls far short of the optimal.
- didibus 6y ago
  Hum, what you're saying seems to contradict a lot of what I've read. We're talking about IO here. So something has to block unless we're using non-blocking IO. I think what you mean is that Go will detect the block, and will yield your execution, and manage the IO in a non-blocking way automatically for you? Under the hood, go still relies on non-blocking IO, epoll and all. And for disk and other serial IO I think it actually allocates real blocking OS threads as well. I agree, the automatic management is convenient, but I fail to see the difference from core.async and the direct use of async IO or blocking OS threads when that's not available like for disk IO.
  - lenkite 6y ago
    Go scheduler is sophisticated and it indeed intercepts all blocking system calls. Essentially Go runtime will run a different goroutine, if current goroutine is blocked on: blocking syscall (for example opening a file), network input, channel operations, primitives in the sync package. http://www.cs.columbia.edu/~aho/cs6998/reports/12-12-11_DeshpandeSponslerWeiss_GO.pdf http://www.cs.columbia.edu/~aho/cs6998/reports/12-12-11_Desh... I would always prefer the runtime to do this job instead of having to scan code under a microscope across functions, libraries and module boundaries to see what blocks and what doesn't. http://danboykis.com/posts/things-i-wish-i-knew-about-core-async/ http://danboykis.com/posts/things-i-wish-i-knew-about-core-a... If I wanted complete control, I would use C++
    - didibus 6y ago
      I don't disagree about the convenience of having the Go runtime handle this automatically for you. I do wish core.async could as well. That said, all your prior comments seem misleading, at least they confused me because it sounded like you were implying something that didn't feel accurate. Core.async has lightweight processes. The core.async Go macro can yield to other Go blocks and it will be really cheap to do so. You can have hundreds of thousands of them concurrently parked and cooperatively switching between each other, etc. In effect, it means that you can achieve the same scale of concurrency in Clojure as you can in Go. Yes, it will be trickier in Clojure, because you have to manage the async IO on your own, and you have to delegate an OS thread on your own when blocking IO is needed. In ClojureScript it won't be as much an issue since all IO is async, but still not as nice as auto-yielding by the runtime under the hood as Go does.
      - armitron 6y ago
        core.async doesn't have lightweight processes, for any sane definition of these terms. A code transformation spat out by a macro, subject to all the serious drawbacks I've already mentioned, is not a lightweight process. But looking at the number of downvotes I've received, stating the obvious no less, I think I'll stop here. I will however take the completely nonsensical, misleading and downright erroneous responses illustrated in this thread as signaling me to steer well clear of Clojure and its community. Let's not forget that this thread started with someone claiming core.async is better than Go's goroutines.

armitron 6y ago

It makes a huge difference. You can't use blocking calls in core.async, since they'll block the entire thread. There is no switching when you do a blocking call, the thread your code runs on will be out of commission until the blocking call returns. So this means that blocking APIs are out of the question and this translates to you not being able to write linear, straightforward code. You have to use non-blocking APIs and callbacks, and rely on core.async channels to force switching between blocks (core.async doesn't have lightweight processes). This is a total nightmare to debug and forces you to bend your programming logic to support this non-optimal programming model. Constrast with Go goroutines or Erlang processes where you write linear, synchronous code that executes as part of lightweight processes. The built-in scheduler will automatically schedule these processes on real OS threads, and switch between them when they block (either through channels or blocking calls). OS thread utilization is far more effective, debugging is a piece of cake and the code that you write is easy to read and understand. I'm bypassing your "world's foremost concurrency expert" comments. I suggest you use the concurrency offered by Go and Erlang by spending a few hours to do tasks you've done with core.async. You will learn something and you will see how Clojure falls far short of the optimal.

didibus 6y ago
Hum, what you're saying seems to contradict a lot of what I've read. We're talking about IO here. So something has to block unless we're using non-blocking IO. I think what you mean is that Go will detect the block, and will yield your execution, and manage the IO in a non-blocking way automatically for you? Under the hood, go still relies on non-blocking IO, epoll and all. And for disk and other serial IO I think it actually allocates real blocking OS threads as well. I agree, the automatic management is convenient, but I fail to see the difference from core.async and the direct use of async IO or blocking OS threads when that's not available like for disk IO.
- lenkite 6y ago
  Go scheduler is sophisticated and it indeed intercepts all blocking system calls. Essentially Go runtime will run a different goroutine, if current goroutine is blocked on: blocking syscall (for example opening a file), network input, channel operations, primitives in the sync package. http://www.cs.columbia.edu/~aho/cs6998/reports/12-12-11_DeshpandeSponslerWeiss_GO.pdf http://www.cs.columbia.edu/~aho/cs6998/reports/12-12-11_Desh... I would always prefer the runtime to do this job instead of having to scan code under a microscope across functions, libraries and module boundaries to see what blocks and what doesn't. http://danboykis.com/posts/things-i-wish-i-knew-about-core-async/ http://danboykis.com/posts/things-i-wish-i-knew-about-core-a... If I wanted complete control, I would use C++
  - didibus 6y ago
    I don't disagree about the convenience of having the Go runtime handle this automatically for you. I do wish core.async could as well. That said, all your prior comments seem misleading, at least they confused me because it sounded like you were implying something that didn't feel accurate. Core.async has lightweight processes. The core.async Go macro can yield to other Go blocks and it will be really cheap to do so. You can have hundreds of thousands of them concurrently parked and cooperatively switching between each other, etc. In effect, it means that you can achieve the same scale of concurrency in Clojure as you can in Go. Yes, it will be trickier in Clojure, because you have to manage the async IO on your own, and you have to delegate an OS thread on your own when blocking IO is needed. In ClojureScript it won't be as much an issue since all IO is async, but still not as nice as auto-yielding by the runtime under the hood as Go does.
    - armitron 6y ago
      core.async doesn't have lightweight processes, for any sane definition of these terms. A code transformation spat out by a macro, subject to all the serious drawbacks I've already mentioned, is not a lightweight process. But looking at the number of downvotes I've received, stating the obvious no less, I think I'll stop here. I will however take the completely nonsensical, misleading and downright erroneous responses illustrated in this thread as signaling me to steer well clear of Clojure and its community. Let's not forget that this thread started with someone claiming core.async is better than Go's goroutines.