4 ms·
I can't help but wonder if the article of the author has seen Quasar: http://docs.paralleluniverse.co/quasar/ http://docs.paralleluniverse.co/quasar/ It seems
by heavenlyhash 12y ago
I can't help but wonder if the article of the author has seen Quasar: http://docs.paralleluniverse.co/quasar/ http://docs.paralleluniverse.co/quasar/ It seems to be a concrete refutation of his claims of impossibility. Quasar successfully brings green threads to the jvm. It includes both channels as are now popularized by golang, as well as higher level patterns like actors. Despite the young nature of the framework, benchmarks show it comparing reasonably well with both golang and erlang. Quasar also provides libraries that step all the way up into the OTP realms of supervision trees (though I haven't myself used this, yet).
The article mentions bytecode weaving, but dismisses it with very wavery justifications. Bytecode manipulation tools are a successful part of the jvm ecosystem. Frankly, they're part of why I consider the jvm ecosystem so successful: bytecode manipulation has allowed things like:
- third party tree-shakers/minifiers and obfuscators (i.e. proguard)
- cross compilers (i.e. robovm)
- concurrency libraries that DO have real green threads and continuations (i.e. Kilim, Quasar, and others)
- code coverage and complexity analysis tooling (i.e. jacoco)
- scala
- clojure
- groovy
- kotlin
- [... more languages ...]
There are two critical points about the above:
- All of these tools were built without direct cooperation with the compiler and core tool chain. That means experimentation and growth were possible from the community.
- Everyone's tools play nice with each other! You can use Quasar as a library in Clojure and then feed that bytecode into Proguard for minification, and then add code coverage instrumentation, and then feed it into Robovm!
Given the wild success of bytecode and bytecode manipulators, I have no idea how the article can so whimsically poo-poo the entire field.
(Yes, I'm well aware Erlang has a VM that allows alternate languages as well. And yes, Elixer is pretty. OT, no, I won't be making investments of my time into Elixer, because I like strong compile-time type systems, and Elixer doesn't have that.)
It is true that even in the presence of a full greenthreading tool like Quasar, code can call legacy APIs that still block a full thread, but this is not sufficient cause to dismiss the possibilities. To quote back part of the article, blocking will always be an issue in any cooperatively multitasked environment: "There’s no real way to limit what that code can do, unless it is explicitly disallowed from [...] looping." And yet I wouldn't claim Erlang fails to give me concurrency just because it still allows loops! Part of the compromise of cooperative multitasking is the very premise that in exchange for the higher performance possible from cooperative code, yes, poorly written code can suck up arbitrary amounts of CPU before yielding. If this were a practical concern, it would also be entirely possible for a bytecode instrumenting library to inject cooperative rescheduling points even into loops; and yet I have no real desire to see this feature.
Furthermore, I strongly object to the claim ForkJoin is "notorious for its overhead". All thread synchronization is notorious for its overhead. That's completely known to any programmer with experience in this area, and in no way unique to ForkJoin.
For an excellent, in-depth coverage of what exactly ForkJoin is and the problems it solves for you, see https://www.youtube.com/watch?v=sq0MX3fHkro https://www.youtube.com/watch?v=sq0MX3fHkro . I highly recommend watching the entire thing despite its length, and even if you are not a JVM programmer -- even if you've been doing concurrent programmer for years, you will almost certainly walk away knowing significantly more about concurrent scheduling from the (relatively) high levels of memory fencing all the way down to CPU architecture choices and their impacts.
I'm not going to claim there are no issues with something like Quasar. In particular, I find that it is harder to operate in an ecosystem where very few existing libraries understand what your application is trying to do with green threads. Mostly, this doesn't phase me if my application is calling out to other libraries, because I control the scheduling one step above them (just like I would in a plainer actor framework without green threads like Akka). The problem is more with "hollywood" style frameworks -- the "don't call me, I'll call you" type -- so far it feels like these are very hard to use when your application is using green threading, but the calling framework has no clue about it. Some sort of interfacing code is required and usually has thread handoffs of its own, which can be moderately unpleasant, and limits your scalability at that juncture. But this is a present-tense bummer, and can be solved by patching (or outright replacing) these hollywood frameworks, or simply avoiding frameworks of that kind altogether.
But in short, I still think it's a bit unreasonable to dismiss the existence of ponies.
- meddlepal 12y agoHe links to quasar in the article...
- heavenlyhash 12y agoAnd yet I don't understand why the article is FUD'ing about it! > hopefully without altering its meaning ...What? I'm using Quasar right now. It works as advertised. Bytecode manipulation tools are not some scary unexplored arena of JVM tooling.
- jeremyjh 12y agoErlang loops will not block the OS thread because the BEAM VM implements preemptive multi-tasking at the (green) process level.
- felixgallo 12y agoNot quite. BEAM implements a cooperative multitasking environment using reductions that are checked at function calls. In practice you will rarely write code that runs forever without calling a function or returning, but it's possible, especially if you use NIFs. This will generally cause scheduler collapse and result in all sorts of weird problems, so in the most recent version of the BEAM, there's 'dirty scheduler' support so that you can work around that problem if you have native code that runs for a long time (> 1 ms). A good primer on all of this is http://jlouisramblings.blogspot.com/2013/01/how-erlang-does-scheduling.html http://jlouisramblings.blogspot.com/2013/01/how-erlang-does-....
- arielweisberg 12y agoFor me the caveat that calling a library or existing code can block the scheduler is a non-starter. I love what everyone is doing and I don't think it means they are useless, but for certain tasks and existing projects it means they are impractical to incorporate. I think it's fair to say that what we are being offered is not ponies. It's a useful tool, but also a leaky abstraction and not what I want to be working with in the long run. Also the statements about Erlang and loops is odd. My understanding is loops in Erlang are preemptible and that the VM bends over backwards to provide consistent scheduling even if it means losing some performance.