5 ms·

Regarding the JVM as a strict superset, there are a few features I'm not sure fit that model. Isolated garbage collection. Limiting GC to a single process/thre

Regarding the JVM as a strict superset, there are a few features I'm not sure fit that model. Isolated garbage collection. Limiting GC to a single process/thread. It's my understanding that this is only possible with individual heaps, prohibiting shared memory. I see this as a polarizing tradeoff (with pros and cons). I don't think the JVM does this, and therefore isn't a superset. Preemptive scheduling and soft real time. It's my understanding that Quasar bridges this gap by looking for natural points in execution to insert a pause/continue (oversimplified I'm sure). What are the guarantees? Erlang provides fairly strong guarantees about fair scheduling, making it very suitable for soft real-time systems. Would Quasar, for instance, pause a tight for loop if execution was running long? Or does it only pause on blocking operations like IO?

pron 11y ago
> It's my understanding that this is only possible with individual heaps, prohibiting shared memory. Yes, but the only advantage is a simpler GC algorithm. HotSpot's GCs are so advanced (let alone HotSpot descendents like Zing) that they give you similar behavior even without isolation, and they include collection of shared data structures, too. > Would Quasar, for instance, pause a tight for loop if execution was running long? Or does it only pause on blocking operations like IO? Quasar does full preemptive scheduling but not time-sliced based preemption. This is not because it can't -- as a matter of fact, early versions of Quasar did time-sliced based preemption, but it turned out to provide no benefit whatsoever. In fact, Erlang's behavior is a limitation. The reason is that Erlang has only one type of thread -- the process, or the user-mode thread -- and therefore has to handle any type of thread, including those that are computation heavy. Thing is, work-stealing is a great scheduling mechanism for transaction-serving threads (that block very often), but not so good for computational threads. Quasar lets you choose: fibers for transactions stuff, and plain threads for long-running computations, both are abstracted into what we call a "strand".
- phamilton 11y ago
  I'm skeptical that Hotspots GC can provide the same behavior, but I'm not familiar enough to challenge that claim. My skepticism arises from most JVM benchmarks around latency (for example, a webserver) having a significant standard deviation, which I had always attributed to an overly broad GC. I have a chip on my shoulder about limited concurrency abstractions. I've been burned too many times by Akka and thread pools and Scala Parallel collections etc.. I've always appreciated the fact that while I won't get the most performance out of BEAM, there's very little I can do wrong that will break my application. I'm going to have to spend some time with Quasar. If your claims hold up, then Clojure plus Quasar seems like fantastic platform.
  - pron 11y ago
    The Clojure bindings for Quasar are called Pulsar[1], and I believe that the guarantees made by Clojure are exactly what you're looking for. [1]: https://github.com/puniverse/pulsar https://github.com/puniverse/pulsar
- dschiptsov 11y ago
  You cannot give "similar behavior" of lock-free, share-nothing data-structures with locking-concurrent ones.
  - pron 11y ago
    The JDK has some of the best implementations of lock-free data structures anywhere, and they are actually all implemented in Java (the JVM exposes direct access to memory fences and CAS). You really never have to drop down to C unless you want to interface with some OS-specific code. The JVM is simply much more general-purpose. Just like your machine and OS themselves can support Erlang, so can the JVM.
    - phamilton 11y ago
      So is this a sort of Greenspun's tenth rule situation? Any sufficiently advanced distributed systems contains a buggy implementation of Erlang? More seriously, is that what you are suggesting? By rewriting (or heavily reconfiguring) the GC, adding a scheduler, etc you end up with all the benefits of Erlang while maintaining the multi-paradigm platform of the JVM? Sounds amazing. Give it to me with out of the box sane configuration and I'm sold. It sounds like it's not quite the case, especially around distributed systems. But perhaps that's just the intended design because clustering machines is sort of out of style in the age of RESTful microservices.
      - pron 11y ago
        > Any sufficiently advanced distributed systems contains a buggy implementation of Erlang? Why won't you say that about machine code? The JVM simply operates at a lower level than Erlang, hence it can be used to implement Erlang, and, it turns out that doing so might gives better results than current Erlang VMs implemented in C, because there are man-decades (or centuries) of shared functionality that's already in the JVM. > By rewriting (or heavily reconfiguring) the GC, adding a scheduler, etc You don't need to rewrite or heavily configure the GC (besides, which one? There are plenty of JVM GCs) -- just to use it as Erlang does, or one that is pauseless no matter how you use it (two implementations, one commercial and one open-source under development at Red Hat). Also, you don't need to add a scheduler. One of the world's best implementations of a work-stealing scheduler, also with about a decade of effort behind it by Doug Lea, is packaged with the JDK. See here[1] to get a feel for the effort behind it. > It sounds like it's not quite the case, especially around distributed systems. Erlang doesn't add anything not already found in high-quality, battle-tested libraries like JGroups[2] when it comes to clustering (actually, JGroups is more powerful than Erlang's clustering). The JVM ecosystem is so huge that it's very hard to come up with something (other than syntax) that doesn't already have a solid implementation out there. I'd really like to retrofit Erjang to use Quasar rather than Kilim (Quasar is just better maintained, more battle-tested and probably faster). I really believe that running Erlang on the JVM is a win for everyone. The Erlang community simply doesn't have a tenth of the resources that are put into OpenJDK[3]. Why not enjoy the benefits? [1]: https://www.youtube.com/watch?v=sq0MX3fHkro https://www.youtube.com/watch?v=sq0MX3fHkro [2]: http://www.jgroups.org/ http://www.jgroups.org/ [3]: like a groundbreaking optimizing compiler that will start being used in Java 9, and can be especially appropriate for compiling Erlang: https://wiki.openjdk.java.net/display/Graal/Publications%20and%20Presentations https://wiki.openjdk.java.net/display/Graal/Publications%20a...

pron 11y ago

> It's my understanding that this is only possible with individual heaps, prohibiting shared memory. Yes, but the only advantage is a simpler GC algorithm. HotSpot's GCs are so advanced (let alone HotSpot descendents like Zing) that they give you similar behavior even without isolation, and they include collection of shared data structures, too. > Would Quasar, for instance, pause a tight for loop if execution was running long? Or does it only pause on blocking operations like IO? Quasar does full preemptive scheduling but not time-sliced based preemption. This is not because it can't -- as a matter of fact, early versions of Quasar did time-sliced based preemption, but it turned out to provide no benefit whatsoever. In fact, Erlang's behavior is a limitation. The reason is that Erlang has only one type of thread -- the process, or the user-mode thread -- and therefore has to handle any type of thread, including those that are computation heavy. Thing is, work-stealing is a great scheduling mechanism for transaction-serving threads (that block very often), but not so good for computational threads. Quasar lets you choose: fibers for transactions stuff, and plain threads for long-running computations, both are abstracted into what we call a "strand".

phamilton 11y ago
I'm skeptical that Hotspots GC can provide the same behavior, but I'm not familiar enough to challenge that claim. My skepticism arises from most JVM benchmarks around latency (for example, a webserver) having a significant standard deviation, which I had always attributed to an overly broad GC. I have a chip on my shoulder about limited concurrency abstractions. I've been burned too many times by Akka and thread pools and Scala Parallel collections etc.. I've always appreciated the fact that while I won't get the most performance out of BEAM, there's very little I can do wrong that will break my application. I'm going to have to spend some time with Quasar. If your claims hold up, then Clojure plus Quasar seems like fantastic platform.
- pron 11y ago
  The Clojure bindings for Quasar are called Pulsar[1], and I believe that the guarantees made by Clojure are exactly what you're looking for. [1]: https://github.com/puniverse/pulsar https://github.com/puniverse/pulsar
dschiptsov 11y ago
You cannot give "similar behavior" of lock-free, share-nothing data-structures with locking-concurrent ones.
- pron 11y ago
  The JDK has some of the best implementations of lock-free data structures anywhere, and they are actually all implemented in Java (the JVM exposes direct access to memory fences and CAS). You really never have to drop down to C unless you want to interface with some OS-specific code. The JVM is simply much more general-purpose. Just like your machine and OS themselves can support Erlang, so can the JVM.
  - phamilton 11y ago
    So is this a sort of Greenspun's tenth rule situation? Any sufficiently advanced distributed systems contains a buggy implementation of Erlang? More seriously, is that what you are suggesting? By rewriting (or heavily reconfiguring) the GC, adding a scheduler, etc you end up with all the benefits of Erlang while maintaining the multi-paradigm platform of the JVM? Sounds amazing. Give it to me with out of the box sane configuration and I'm sold. It sounds like it's not quite the case, especially around distributed systems. But perhaps that's just the intended design because clustering machines is sort of out of style in the age of RESTful microservices.
    - pron 11y ago
      > Any sufficiently advanced distributed systems contains a buggy implementation of Erlang? Why won't you say that about machine code? The JVM simply operates at a lower level than Erlang, hence it can be used to implement Erlang, and, it turns out that doing so might gives better results than current Erlang VMs implemented in C, because there are man-decades (or centuries) of shared functionality that's already in the JVM. > By rewriting (or heavily reconfiguring) the GC, adding a scheduler, etc You don't need to rewrite or heavily configure the GC (besides, which one? There are plenty of JVM GCs) -- just to use it as Erlang does, or one that is pauseless no matter how you use it (two implementations, one commercial and one open-source under development at Red Hat). Also, you don't need to add a scheduler. One of the world's best implementations of a work-stealing scheduler, also with about a decade of effort behind it by Doug Lea, is packaged with the JDK. See here[1] to get a feel for the effort behind it. > It sounds like it's not quite the case, especially around distributed systems. Erlang doesn't add anything not already found in high-quality, battle-tested libraries like JGroups[2] when it comes to clustering (actually, JGroups is more powerful than Erlang's clustering). The JVM ecosystem is so huge that it's very hard to come up with something (other than syntax) that doesn't already have a solid implementation out there. I'd really like to retrofit Erjang to use Quasar rather than Kilim (Quasar is just better maintained, more battle-tested and probably faster). I really believe that running Erlang on the JVM is a win for everyone. The Erlang community simply doesn't have a tenth of the resources that are put into OpenJDK[3]. Why not enjoy the benefits? [1]: https://www.youtube.com/watch?v=sq0MX3fHkro https://www.youtube.com/watch?v=sq0MX3fHkro [2]: http://www.jgroups.org/ http://www.jgroups.org/ [3]: like a groundbreaking optimizing compiler that will start being used in Java 9, and can be especially appropriate for compiling Erlang: https://wiki.openjdk.java.net/display/Graal/Publications%20and%20Presentations https://wiki.openjdk.java.net/display/Graal/Publications%20a...