4 ms·

Erlang Doesn’t Fit The JVM

44 points by newsit 18y ago

rjprins 18y ago
If these are real limitations of the JVM, what does that say about Clojure?
- pavelludiq 18y ago
  I barely know Clojure, but from what i know it has a workaround for the lack of TCO: http://clojure.org/special_forms#toc10 http://clojure.org/special_forms#toc10
  - johnm 18y ago
    This isn't a complete solution but does handle a lot of cases that non-FP developers will write. Alas, even though there is TCO in MLVM, it doesn't look like that will get through the JCP in time to get into Java v7. :-(
stcredzero 18y ago
At it’s heart the JVM is designed to run OO languages very efficiently. This is why languages like Ruby and Python are (mostly) easily ported to the platform... However, the JVM is less suited to running non-OO languages. Languages like Erlang, Haskell, and Scheme provide features, like tail recursion, closures, and continuations, which are not prominent in the mainstream OO world the JVM targets. They depart far enough from the OO model to make the JVM a poor platform choice. That's strange, because all of the Smalltalk VMs are capable of closures and many of them can do continuations, yet Smalltalk is often cited as one of the purest OO environments. (I'm also fairly sure that tail recursion can be done as a compiler trick.) That said, there are real limitations of the JVM. Most Smalltalk VMs can be targeted by a Java compiler to do a dandy job as a JVM. It's much harder to target Smalltalk for a JVM, unless you limit the capabilities of the environment. (Particularly runtime compilation.)
- ori_b 18y ago
  I'm also fairly sure that tail recursion can be done as a compiler trick. Only for a single function, at least on the JVM. For mutually recursive functions (ie, a() tailcalls b() which tailcalls a()) you either need the VM to do the optimization, or some explicit control over the stack pointer, or implementing your own function call model within one function. For the first, you would need some sort of new tailcall instruction, or the VM would have to be guaranteed to do tail call optimizations. (You can't really give control of the stack explicitly while keeping things portable and efficient) For the second, it's workable... if you don't care about interoperating with other languages on the JVM.
- 10ren 18y ago
  I get the impression that Smalltalk is very late-bound, whereas Java is sort of half-late-bound[1]. Late binding enables you to do pretty much anything. [1] For example: in Java, methods with the same name but different argument types ("overloaded") are statically bound; whereas in Smalltalk, they are dynamically bound. Java dynamically binds methods with the same name and same argument types that are defined in a subclass/implementation ("overridden"). Sounds like a performance tradeoff to me. (Is Smalltalk performance C-like or Python-like?)
  - nraynaud 18y ago
    smalltalk is generaly slow, and it has the same calling system as ruby. in smalltalk, methods with the same name don't exist. every method on a object has an unique selector. The selector plays exactly the same role as the signature in java there is no difference on this aspect.
    - jimbokun 18y ago
      "in smalltalk, methods with the same name don't exist. every method on a object has an unique selector." Also true of Objective C (as many of you might already know).
      - 10ren 18y ago
noss 18y ago
Isn't LLVM a more interesting universal machine to target anyway?
- gps408 18y ago
  Isn't part of the reason for targeting the JVM the ton of libraries you get for free?
  - ori_b 18y ago
    How many of them fit idiomatically into the language you choose to use on the JVM?
    - jwinter 18y ago
      Clojure does a good job of introducing new idioms specifically for Java support. From their docs: (doto (new java.util.HashMap) (.put "a" 1) (.put "b" 2)) (.. System (getProperties) (get "os.name"))
      - mahmud 18y ago
        iteration and getting system setting stuff are so generic you might as well implement it yourself or write a thin wrapper (in this case iterating over entries in a hash table and getting the OS name) IMO, a good chunk of the java libraries are there to overcome java deficiencies, and don't produce anything grand on their own for other languages.
      - ori_b 18y ago
        So, in other words, it doesn't fit into preexisting languages idiomatically. You want to introduce new idioms into Closure to make it integrate better with the JVM. That's fine, but it also means that if you want to do a direct port of, say, Ruby or Python or Erlang, you'll have several rather large warts compared to new languages designed to run on the JVM and use the Java class libraries.
- nraynaud 18y ago
glymor 18y ago
Erlang's VM is unmatched scaling above 16+ cores (for example process migration http://unlimitednovelty.com/2009/01/cutting-edge-of-vm-design.html http://unlimitednovelty.com/2009/01/cutting-edge-of-vm-desig... ) this is something that the java vm hasn't even begun to look at.
- framiere 18y ago
  That is untrue. Check http://blogs.azulsystems.com/cliff/ http://blogs.azulsystems.com/cliff/ for really impressive work on jvm that works on 800+ cores ... oh yes, and you have constant time garbage collection.
  - glymor 18y ago
    Azul's system is proprietary and only runs on special hardware. The openjdk that's in common use has had essentially no work done on green threads. And from your link "we have a lite microkernel style OS; we can easily handle 100K runnable threads" it's not the VM that's handling it.
    - framiere 18y ago
      You are correct, I was pointing to cliff click's blog as he his giving back to the community. Look at http://blogs.azulsystems.com/cliff/2007/03/a_nonblocking_h.html http://blogs.azulsystems.com/cliff/2007/03/a_nonblocking_h.h... : an implementation of a non blocking hashmap without lock that scales beautifully on a bunch of cores without the need of their hardware (http://sourceforge.net/projects/high-scale-lib http://sourceforge.net/projects/high-scale-lib) As per actor on the JVM open source projects are blooming around such as http://code.google.com/p/actorom/ http://code.google.com/p/actorom/. Alex Miller did a nice review here of the solutions out there: http://www.javaworld.com/javaworld/jw-03-2009/jw-03-actor-concurrency2.html http://www.javaworld.com/javaworld/jw-03-2009/jw-03-actor-co...
      - glymor 18y ago
        I remember his hashmap when he published it, state machines are a great way to do concurrency. The Erlang way of handling concurrency is to spawn millions of "processes" (threads) obviously these can't be hardware threads so Erlang is hitting problems OS schedulers would handle (or not be able to handle) in the JVM's case. I would argue that the JVM is eventually going to have to handle the threading itself if it's to support any actor based programming. OS schedulers have other uses they need to be fast for and hardware threads will always have higher overhead. Python's Stackless is perhaps an example of the amount of work it would require.
adammarkey 18y ago
I feel your pain with java serialization. The fact that you cant serialize exceptions to notify systems connected to a JVM that something blew up is just insane. All objects should be able to be serialized (in binary) from one system to another as long as JVM versions and class libraries match... end of story.
- axod 18y ago
  They can... you just have to do it yourself that's all.
zitterbewegung 18y ago
SISC targets the JVM with tail call optimization with some trickery. http://sisc-scheme.org/ http://sisc-scheme.org/
nraynaud 18y ago
first class closures are not a JVM problem, but a language problem. Moreover, closures (lexical closures) exists in java itself: an annonymous inner class captures (well, on demand, for perf reasons) all the lexical terms visible at the point of use. the tail call is a bytecode security problem. Looks like this problem has been solved and will be merged.
- akeefer 18y ago
  Anonymous inner classes aren't really closures in the more traditional sense, since they require that the captured variables be final; that makes them more restricted than closures generally are in other programming languages. That might not be a problem for Erlang given its write-once variables, but it is a problem for other languages. We've been working on generating JVM bytecode for our in-house language, which does support a less-restricted version of closures, and we work around that by wrapping closed variables in one-element arrays, though you have to do for both the initial declaration and on every reference to that variable, even outside the closure, which gets really annoying when the closed variable is something passed onto the stack as a function argument, and probably in other cases I'm not thinking about right now. It's definitely workable, it's just messy and incurs more overhead than is ideal.
davidw 18y ago
He doesn't mention the Erlang scheduler and processes. Shouldn't be that hard to hack into the JVM, but as I've commented elsewhere, that's only half the battle...
- johnbender 18y ago
  Yah process scheduling is trivially easy...
johnm 18y ago
If you want a VM that's totally geared towards "dynamic languages" you should definitely check out Parrot ( http://www.parrot.org/ http://www.parrot.org/ ).
- jerf 18y ago
  Erlang isn't a dynamic language in the sense that Parrot is optimized for, either. Erlang-on-Parrot wouldn't be able to use any other libraries developed for the Parrot ecosystem as they would depend on mutable languages any better than it already can (it would still have to go through the Erlang ports), eliminating a major reason to go to a given VM, and the Parrot VM optimizes a lot of cases that Erlang doesn't care about while missing out on many it does. Erlang's data structures are not manifestly typed, but in most other ways it is a static language.
hendler 18y ago
Ruby language on the Erlang VM - http://on-ruby.blogspot.com/2009/03/reia-new-dynamic-language-on-erlang-vm.html http://on-ruby.blogspot.com/2009/03/reia-new-dynamic-languag...
johnbender 18y ago
Why would you even bother doing this in the first place. Erlang's VM is one the things that makes it interesting, Not the syntax. The only reason I can see is the OTP libraries, but then again, why not just use the Erlang VM as you get so much great stuff for Actor based concurrency out of it.

44 points by newsit 18y ago

rjprins 18y ago

If these are real limitations of the JVM, what does that say about Clojure?

pavelludiq 18y ago
I barely know Clojure, but from what i know it has a workaround for the lack of TCO: http://clojure.org/special_forms#toc10 http://clojure.org/special_forms#toc10
- johnm 18y ago
  This isn't a complete solution but does handle a lot of cases that non-FP developers will write. Alas, even though there is TCO in MLVM, it doesn't look like that will get through the JCP in time to get into Java v7. :-(

stcredzero 18y ago

At it’s heart the JVM is designed to run OO languages very efficiently. This is why languages like Ruby and Python are (mostly) easily ported to the platform... However, the JVM is less suited to running non-OO languages. Languages like Erlang, Haskell, and Scheme provide features, like tail recursion, closures, and continuations, which are not prominent in the mainstream OO world the JVM targets. They depart far enough from the OO model to make the JVM a poor platform choice. That's strange, because all of the Smalltalk VMs are capable of closures and many of them can do continuations, yet Smalltalk is often cited as one of the purest OO environments. (I'm also fairly sure that tail recursion can be done as a compiler trick.) That said, there are real limitations of the JVM. Most Smalltalk VMs can be targeted by a Java compiler to do a dandy job as a JVM. It's much harder to target Smalltalk for a JVM, unless you limit the capabilities of the environment. (Particularly runtime compilation.)

ori_b 18y ago
I'm also fairly sure that tail recursion can be done as a compiler trick. Only for a single function, at least on the JVM. For mutually recursive functions (ie, a() tailcalls b() which tailcalls a()) you either need the VM to do the optimization, or some explicit control over the stack pointer, or implementing your own function call model within one function. For the first, you would need some sort of new tailcall instruction, or the VM would have to be guaranteed to do tail call optimizations. (You can't really give control of the stack explicitly while keeping things portable and efficient) For the second, it's workable... if you don't care about interoperating with other languages on the JVM.
10ren 18y ago
I get the impression that Smalltalk is very late-bound, whereas Java is sort of half-late-bound[1]. Late binding enables you to do pretty much anything. [1] For example: in Java, methods with the same name but different argument types ("overloaded") are statically bound; whereas in Smalltalk, they are dynamically bound. Java dynamically binds methods with the same name and same argument types that are defined in a subclass/implementation ("overridden"). Sounds like a performance tradeoff to me. (Is Smalltalk performance C-like or Python-like?)
- nraynaud 18y ago
  smalltalk is generaly slow, and it has the same calling system as ruby. in smalltalk, methods with the same name don't exist. every method on a object has an unique selector. The selector plays exactly the same role as the signature in java there is no difference on this aspect.
  - jimbokun 18y ago
    "in smalltalk, methods with the same name don't exist. every method on a object has an unique selector." Also true of Objective C (as many of you might already know).
    - 10ren 18y ago

noss 18y ago

Isn't LLVM a more interesting universal machine to target anyway?

gps408 18y ago
Isn't part of the reason for targeting the JVM the ton of libraries you get for free?
- ori_b 18y ago
  How many of them fit idiomatically into the language you choose to use on the JVM?
  - jwinter 18y ago
    Clojure does a good job of introducing new idioms specifically for Java support. From their docs: (doto (new java.util.HashMap) (.put "a" 1) (.put "b" 2)) (.. System (getProperties) (get "os.name"))
    - mahmud 18y ago
      iteration and getting system setting stuff are so generic you might as well implement it yourself or write a thin wrapper (in this case iterating over entries in a hash table and getting the OS name) IMO, a good chunk of the java libraries are there to overcome java deficiencies, and don't produce anything grand on their own for other languages.
    - ori_b 18y ago
      So, in other words, it doesn't fit into preexisting languages idiomatically. You want to introduce new idioms into Closure to make it integrate better with the JVM. That's fine, but it also means that if you want to do a direct port of, say, Ruby or Python or Erlang, you'll have several rather large warts compared to new languages designed to run on the JVM and use the Java class libraries.
nraynaud 18y ago

glymor 18y ago

Erlang's VM is unmatched scaling above 16+ cores (for example process migration http://unlimitednovelty.com/2009/01/cutting-edge-of-vm-design.html http://unlimitednovelty.com/2009/01/cutting-edge-of-vm-desig... ) this is something that the java vm hasn't even begun to look at.

framiere 18y ago
That is untrue. Check http://blogs.azulsystems.com/cliff/ http://blogs.azulsystems.com/cliff/ for really impressive work on jvm that works on 800+ cores ... oh yes, and you have constant time garbage collection.
- glymor 18y ago
  Azul's system is proprietary and only runs on special hardware. The openjdk that's in common use has had essentially no work done on green threads. And from your link "we have a lite microkernel style OS; we can easily handle 100K runnable threads" it's not the VM that's handling it.
  - framiere 18y ago
    You are correct, I was pointing to cliff click's blog as he his giving back to the community. Look at http://blogs.azulsystems.com/cliff/2007/03/a_nonblocking_h.html http://blogs.azulsystems.com/cliff/2007/03/a_nonblocking_h.h... : an implementation of a non blocking hashmap without lock that scales beautifully on a bunch of cores without the need of their hardware (http://sourceforge.net/projects/high-scale-lib http://sourceforge.net/projects/high-scale-lib) As per actor on the JVM open source projects are blooming around such as http://code.google.com/p/actorom/ http://code.google.com/p/actorom/. Alex Miller did a nice review here of the solutions out there: http://www.javaworld.com/javaworld/jw-03-2009/jw-03-actor-concurrency2.html http://www.javaworld.com/javaworld/jw-03-2009/jw-03-actor-co...
    - glymor 18y ago
      I remember his hashmap when he published it, state machines are a great way to do concurrency. The Erlang way of handling concurrency is to spawn millions of "processes" (threads) obviously these can't be hardware threads so Erlang is hitting problems OS schedulers would handle (or not be able to handle) in the JVM's case. I would argue that the JVM is eventually going to have to handle the threading itself if it's to support any actor based programming. OS schedulers have other uses they need to be fast for and hardware threads will always have higher overhead. Python's Stackless is perhaps an example of the amount of work it would require.

adammarkey 18y ago

I feel your pain with java serialization. The fact that you cant serialize exceptions to notify systems connected to a JVM that something blew up is just insane. All objects should be able to be serialized (in binary) from one system to another as long as JVM versions and class libraries match... end of story.

axod 18y ago
They can... you just have to do it yourself that's all.

zitterbewegung 18y ago

SISC targets the JVM with tail call optimization with some trickery. http://sisc-scheme.org/ http://sisc-scheme.org/

nraynaud 18y ago

first class closures are not a JVM problem, but a language problem. Moreover, closures (lexical closures) exists in java itself: an annonymous inner class captures (well, on demand, for perf reasons) all the lexical terms visible at the point of use. the tail call is a bytecode security problem. Looks like this problem has been solved and will be merged.

akeefer 18y ago
Anonymous inner classes aren't really closures in the more traditional sense, since they require that the captured variables be final; that makes them more restricted than closures generally are in other programming languages. That might not be a problem for Erlang given its write-once variables, but it is a problem for other languages. We've been working on generating JVM bytecode for our in-house language, which does support a less-restricted version of closures, and we work around that by wrapping closed variables in one-element arrays, though you have to do for both the initial declaration and on every reference to that variable, even outside the closure, which gets really annoying when the closed variable is something passed onto the stack as a function argument, and probably in other cases I'm not thinking about right now. It's definitely workable, it's just messy and incurs more overhead than is ideal.

davidw 18y ago

He doesn't mention the Erlang scheduler and processes. Shouldn't be that hard to hack into the JVM, but as I've commented elsewhere, that's only half the battle...

johnbender 18y ago
Yah process scheduling is trivially easy...

johnm 18y ago

If you want a VM that's totally geared towards "dynamic languages" you should definitely check out Parrot ( http://www.parrot.org/ http://www.parrot.org/ ).

jerf 18y ago
Erlang isn't a dynamic language in the sense that Parrot is optimized for, either. Erlang-on-Parrot wouldn't be able to use any other libraries developed for the Parrot ecosystem as they would depend on mutable languages any better than it already can (it would still have to go through the Erlang ports), eliminating a major reason to go to a given VM, and the Parrot VM optimizes a lot of cases that Erlang doesn't care about while missing out on many it does. Erlang's data structures are not manifestly typed, but in most other ways it is a static language.

hendler 18y ago

Ruby language on the Erlang VM - http://on-ruby.blogspot.com/2009/03/reia-new-dynamic-language-on-erlang-vm.html http://on-ruby.blogspot.com/2009/03/reia-new-dynamic-languag...

johnbender 18y ago

Why would you even bother doing this in the first place. Erlang's VM is one the things that makes it interesting, Not the syntax. The only reason I can see is the OTP libraries, but then again, why not just use the Erlang VM as you get so much great stuff for Actor based concurrency out of it.