6 ms·

core.async in Clojure is not equivalent to Go's asynchronous support. Even worse, core.async is the simplest & least-performant way to implement asynchronicity:

core.async in Clojure is not equivalent to Go's asynchronous support. Even worse, core.async is the simplest & least-performant way to implement asynchronicity: It's just using an internal thread pool, meaning that go blocks in Clojure are not at all like goroutines in Go since they are state-machines produced by a macro that can block the entire thread when doing blocking calls. So, one has to manually "park" them by using core.async channels. Summarizing: core.async is macro sugar on top of java.util.concurrent. Go has actual lightweight processes that are scheduled on top of OS threads. Big difference in both design and runtime performance. Erlang too has m:n threading and offers a simpler and safer programming model (each Erlang process has its own heap).

puredanger 6y ago
Re "core.async is macro sugar on top of java.util.concurrent" - yes, but also no. Java does not (yet) have a way to spin light-weight processes that can park without consuming a thread OR a way to select/alt over multiple parked ops, so being able to do that in Clojure is actually novel (and enabled by having macros that can rewrite code as a state machine). There have been a few projects that did this in the past on Java using runtime bytecode modification but none of them are widely used afaik. This will change once Project Loom lands in Java and provides m:n fibers.
- fooster 6y ago
  Yes, yes in the future at some undefined point java will be better. Meanwhile go works very well today. And you don't need to be saddled with the JVM and the very bloated java ecosystem.
  - ashes-of-sol 6y ago
    Could you expand on the problems you see with the JVM and ecosystem? I was skeptical, but it hasn't been a problem for us so far, and having java libraries and good java interop has helped us more than it has hurt.
  - deleted 6y ago
    [deleted]
  - nojito 6y ago
    >very bloated java ecosystem Why waste time reinventing what java has already solved?
    - armitron 6y ago
      Life is too short to waste on Java? Yes, some of us really find Java that bad. Plenty of other languages have solved the same problems in much better ways (as demonstrated by this very discussion re: asynchronicity). I'll pick Go over Java, Erlang over Go and wouldn't pick Java (incl languages running on top of) for anything today.
      - nojito 6y ago
        Why not? The java ecosystem has solved so many problems already. Don’t you want to just focus on the business problems?
        armitron 6y ago
        I don't solve business problems, I solve my own problems and try to have fun along the way.
        lostcolony 6y ago
        The business problem still can include things like "concurrency", which is just -painful- in Java, especially if I/O bound. I much prefer languages that make easy stuff harder, but hard stuff easier, than languages like Java that make easy stuff easier, and hard stuff harder.
        waheoo 6y ago
        Java doesnt solve business problems. It solves enterprise ones. Or at least claims to.
        pjmlp 6y ago
        Enterprises do business last time I checked.
  - pron 6y ago
    Sure, sometimes a simple, crude runtime will do, but if you need really good performance and low-overhead deep observability, you might want to be saddled with something that's more state-of-the-art.
  - didibus 6y ago
    Regarding Java, sure. But in the context of Clojure, it's here right now as well, no need to wait for anything. I also don't really understand the "bloat" comment. What do you refer too? Are you referring to the fact the JVM supports too many features?
huahaiy 6y ago
I am confused. If I remember correctly, go's runtime uses a thread pool as well. It's not like go somehow invented a runtime that does not rely on OS threads. So in the end of the day, what difference does it make? If you ask me, java.util.concurrent is a much better and battle tested concurrent infrastructure than the simplistic implementation in go, because java.util.concurrent is actually written by the world's foremost concurrency expert, Doug Lee. I will take that over someone else's hand rolling concurrent runtime any day. God knows how many bugs there are in your home grown concurrency code.
- armitron 6y ago
  It makes a huge difference. You can't use blocking calls in core.async, since they'll block the entire thread. There is no switching when you do a blocking call, the thread your code runs on will be out of commission until the blocking call returns. So this means that blocking APIs are out of the question and this translates to you not being able to write linear, straightforward code. You have to use non-blocking APIs and callbacks, and rely on core.async channels to force switching between blocks (core.async doesn't have lightweight processes). This is a total nightmare to debug and forces you to bend your programming logic to support this non-optimal programming model. Constrast with Go goroutines or Erlang processes where you write linear, synchronous code that executes as part of lightweight processes. The built-in scheduler will automatically schedule these processes on real OS threads, and switch between them when they block (either through channels or blocking calls). OS thread utilization is far more effective, debugging is a piece of cake and the code that you write is easy to read and understand. I'm bypassing your "world's foremost concurrency expert" comments. I suggest you use the concurrency offered by Go and Erlang by spending a few hours to do tasks you've done with core.async. You will learn something and you will see how Clojure falls far short of the optimal.
  - didibus 6y ago
    Hum, what you're saying seems to contradict a lot of what I've read. We're talking about IO here. So something has to block unless we're using non-blocking IO. I think what you mean is that Go will detect the block, and will yield your execution, and manage the IO in a non-blocking way automatically for you? Under the hood, go still relies on non-blocking IO, epoll and all. And for disk and other serial IO I think it actually allocates real blocking OS threads as well. I agree, the automatic management is convenient, but I fail to see the difference from core.async and the direct use of async IO or blocking OS threads when that's not available like for disk IO.
bjoli 6y ago
Go's concurrency primitives are a less expressive version of concurrentML, which imo has been the nicest way of writing concurrent (but for a long time not parallel) applications the last 30 years. Guile scheme has a parallel version of it that is implemented as a library: https://github.com/wingo/fibers/wiki/Manual https://github.com/wingo/fibers/wiki/Manual Comparing concurrent systems is hard. I would suggest implementing selecting with negative acknowledgement (nack). If I can do that nicely I know I will probably like the system.

puredanger 6y ago

Re "core.async is macro sugar on top of java.util.concurrent" - yes, but also no. Java does not (yet) have a way to spin light-weight processes that can park without consuming a thread OR a way to select/alt over multiple parked ops, so being able to do that in Clojure is actually novel (and enabled by having macros that can rewrite code as a state machine). There have been a few projects that did this in the past on Java using runtime bytecode modification but none of them are widely used afaik. This will change once Project Loom lands in Java and provides m:n fibers.

fooster 6y ago
Yes, yes in the future at some undefined point java will be better. Meanwhile go works very well today. And you don't need to be saddled with the JVM and the very bloated java ecosystem.
- ashes-of-sol 6y ago
  Could you expand on the problems you see with the JVM and ecosystem? I was skeptical, but it hasn't been a problem for us so far, and having java libraries and good java interop has helped us more than it has hurt.
- deleted 6y ago
  [deleted]
- nojito 6y ago
  >very bloated java ecosystem Why waste time reinventing what java has already solved?
  - armitron 6y ago
    Life is too short to waste on Java? Yes, some of us really find Java that bad. Plenty of other languages have solved the same problems in much better ways (as demonstrated by this very discussion re: asynchronicity). I'll pick Go over Java, Erlang over Go and wouldn't pick Java (incl languages running on top of) for anything today.
    - nojito 6y ago
      Why not? The java ecosystem has solved so many problems already. Don’t you want to just focus on the business problems?
      - armitron 6y ago
        I don't solve business problems, I solve my own problems and try to have fun along the way.
      - lostcolony 6y ago
        The business problem still can include things like "concurrency", which is just -painful- in Java, especially if I/O bound. I much prefer languages that make easy stuff harder, but hard stuff easier, than languages like Java that make easy stuff easier, and hard stuff harder.
      - waheoo 6y ago
        Java doesnt solve business problems. It solves enterprise ones. Or at least claims to.
        pjmlp 6y ago
        Enterprises do business last time I checked.
- pron 6y ago
  Sure, sometimes a simple, crude runtime will do, but if you need really good performance and low-overhead deep observability, you might want to be saddled with something that's more state-of-the-art.
- didibus 6y ago
  Regarding Java, sure. But in the context of Clojure, it's here right now as well, no need to wait for anything. I also don't really understand the "bloat" comment. What do you refer too? Are you referring to the fact the JVM supports too many features?

huahaiy 6y ago

I am confused. If I remember correctly, go's runtime uses a thread pool as well. It's not like go somehow invented a runtime that does not rely on OS threads. So in the end of the day, what difference does it make? If you ask me, java.util.concurrent is a much better and battle tested concurrent infrastructure than the simplistic implementation in go, because java.util.concurrent is actually written by the world's foremost concurrency expert, Doug Lee. I will take that over someone else's hand rolling concurrent runtime any day. God knows how many bugs there are in your home grown concurrency code.

armitron 6y ago
It makes a huge difference. You can't use blocking calls in core.async, since they'll block the entire thread. There is no switching when you do a blocking call, the thread your code runs on will be out of commission until the blocking call returns. So this means that blocking APIs are out of the question and this translates to you not being able to write linear, straightforward code. You have to use non-blocking APIs and callbacks, and rely on core.async channels to force switching between blocks (core.async doesn't have lightweight processes). This is a total nightmare to debug and forces you to bend your programming logic to support this non-optimal programming model. Constrast with Go goroutines or Erlang processes where you write linear, synchronous code that executes as part of lightweight processes. The built-in scheduler will automatically schedule these processes on real OS threads, and switch between them when they block (either through channels or blocking calls). OS thread utilization is far more effective, debugging is a piece of cake and the code that you write is easy to read and understand. I'm bypassing your "world's foremost concurrency expert" comments. I suggest you use the concurrency offered by Go and Erlang by spending a few hours to do tasks you've done with core.async. You will learn something and you will see how Clojure falls far short of the optimal.
- didibus 6y ago
  Hum, what you're saying seems to contradict a lot of what I've read. We're talking about IO here. So something has to block unless we're using non-blocking IO. I think what you mean is that Go will detect the block, and will yield your execution, and manage the IO in a non-blocking way automatically for you? Under the hood, go still relies on non-blocking IO, epoll and all. And for disk and other serial IO I think it actually allocates real blocking OS threads as well. I agree, the automatic management is convenient, but I fail to see the difference from core.async and the direct use of async IO or blocking OS threads when that's not available like for disk IO.

bjoli 6y ago

Go's concurrency primitives are a less expressive version of concurrentML, which imo has been the nicest way of writing concurrent (but for a long time not parallel) applications the last 30 years. Guile scheme has a parallel version of it that is implemented as a library: https://github.com/wingo/fibers/wiki/Manual https://github.com/wingo/fibers/wiki/Manual Comparing concurrent systems is hard. I would suggest implementing selecting with negative acknowledgement (nack). If I can do that nicely I know I will probably like the system.