7 ms·

Scala on LLVM

• swah 15y ago
  Were there compromises in Scala's design because it targetted the JVM?
  • mike_esspe 15y ago
    Java's type erasure is bad for matching List[SomeType], because information about SomeType is erased. This is my greatest nuisance of JVM with Scala.
  • smcj 15y ago
    The compiler can only optimize simple calls in tail position, so recursive algorithms can overflow the stack, because the JVM lacks the support for any optimization here. No good IO/File API (because it needs support from the JVM for certain operations, which will only be added in Java 7). No good Unicode support (it uses Java's java.lang.String). Type erasure (although the consequences are not as bad as Java's). Java's reflection API doesn't agree with Scala code sometimes.
    • jbooth 15y ago
      java.lang.String is UTF-8 by default unless you call toBytes() with a different encoder.. how is this not "unicode" enough? (Honest question). Type erasure is a huge pain though.
      • wolfgangK 15y ago
        I would have sworn that it was utf-16 (which is still Unicode enough btw) .
        • jbooth 15y ago
          Java chars are 2 bytes, but the default String encoding when serializing to/from byte[] arrays and streams is UTF-8 (on unix at least, may vary on windows).
          • jbooth 15y ago
            Uh, not sure why this comment was controversial?
      • stephenjudkins 15y ago
        java.lang.String uses UTF-16 internally. It's a wrapper around an array of 16-bit chars. Thus, it's possible for one to create a String which is NOT valid unicode by abusing surrogate pairs. Thus, the JVM has no native datatype that represents a "valid unicode string". This is unfortunate, because if java.lang.String did enforce this it would let us make some helpful assumptions.
        • jbooth 15y ago
          Well, every single way to read/write that string to bytes is unicode by default, unless you go out of your way to plug in a different encoder. What are you trying to do, export a pointer and write the raw bytes to some destination while assuming it's correct unicode? If you're doing something that low level, it's always possible to corrupt your data and have invalid unicode, just set an invalid byte/rune somewhere in that byte string. Direct memory access always throws guarantees out the window. It is annoying that getBytes() has to allocate and fill a byte array because of the mismatch between char/byte, but you can work around it when necessary and that's not really related to "not being unicode enough", if anything it's "too unicode" with the insistence on the char type for internal structure.
          • stephenjudkins 15y ago
            No, one of the constructors for String takes only char[] as a parameter. You can pass in an arbitrary array of chars, even invalid UTF-16. You're correct that well-written code should never do this. However, there is no guarantee that some library you're using doesn't. You can never assume that 'new String(oldString.getBytes("UTF-8"), "UTF-8").equals(oldString)', which has some unfortunate side-effects if you're doing anything involving serialization and equality. I agree that Java's String API is generally quite well-designed, but the ability to access the raw UTF-16 is a very big leak in the abstraction.
            • Confusion 15y ago
              If that ability was lacking, other people would be complaining about it. Abstractions should not prevent you from accessing the bits underneath: they should make it unnecessary. Which they never completely succeed in, because there are always fringe use cases you didn't foresee.
  • bobbyi 15y ago
    Scala had a CLR version from pretty much the beginning, so I don't think it was designed with the intention of exclusively targeting the JVM.
    • skrebbel 15y ago
      The current CLR version is definitely not at production quality like the JVM version, and afaik it never was. At a recent talk in the Netherlands, Martin Odersky also basically admitted that there's still a fairly long way until the CLR version is really at production quality, and even then it will probably keep lagging behind the JVM version. So surely you're right about intention, but that's also about all it is.
  • vukk 15y ago
    No lightweight threads, one has to resort to an evented approach (and lose the fault tolerance etc. afaik).
    • gtani 15y ago
      The actors- hotspot tie is pretty big http://stackoverflow.com/questions/2987408/actors-in-scala-net http://stackoverflow.com/questions/2987408/actors-in-scala-n...
  • riffraff 15y ago
    I am aware of one that is related to compatibility with java more than the JVM itself: no multiple dispatch (at least, this is what martin odersky said to me when I asked him why they are not in scala)
• rickmode 15y ago
  I'll totally jump back on to the Scala bandwagon once it breaks free of the JVM. I'd love for Clojure to make the same jump to LLVM however it relies far more on the host platform's libraries (the Java/C# libraries).
  • jherdman 15y ago
    I'm actually a little confused by this. Isn't one of the major benefits of Scala, Clojure, JRuby, et al, that it is hosted on the JVM?
    • wbhart 15y ago
      Yes, it is. And I suspect the Scala people aren't advocating that it move away from the JVM. However, I can definitely say, should a Scala on LLVM become available with trivial FFI with native code and close to compiled C performance even for the Scala interpreter (this is definitely possible), I would heartily switch to Scala from C. I absolutely love the idea of Scala on the LLVM back end. That opens up a whole new world to me which currently doesn't exist. I've recently been trying to design my own language on the LLVM Jit. So far, many of the design decisions I am making are very similar to, or the same as Scala, even though I am familiar with many different languages. What's missing from Scala for me personally, is precisely what is proposed in this paper.
      • runT1ME 15y ago
        Functional and hybrid functional languages benefit greatly from a highly tuned GC. I don't see LLVM supporting something on par with the JVM's collector anytime soon so I think you'll lose a lot of the speed you see with Scala on the JVM unfortunately.
        • aphexairlines 15y ago
          The flip side is that the Scala developers can provide a GC tuned to Scala's use case, rather than the JVM's focus on Java's use case.
    • 15y ago
• ErikCorry 15y ago
  I am very sceptical about an approach that concentrates on code generation and punts GC to later. My experience is that you can always improve the code generation later, but the early decisions you make about GC will be with your forever. Things like data representation, whether objects can move, the interface to native code, the concurrency model etc. are very dependent on the GC and once you have built a big system that doesn't have a moving, precise GC you have probably boxed yourself in in terms of making a good design for your memory management later.
  • wbhart 15y ago
    LLVM has almost no support for GC and it is about the biggest problem that just hits you in the face when you use it. What I've been doing for my own experiments is very naive (and quite possibly broken). I've been implementing an interpreter with all GC done on the "compile side" of the Jit. What this means is that to allocate at runtime it reaches back around and calls functions which are pre-compiled into the interpreter codebase. The interpreter itself uses linked lists extensively for its symbol table and to hold references to memory that it needs at compile time, and this is just hooked into by Jit compiled code at runtime, i.e. there's automatically an object associated with each allocation done at runtime because there is an object associated with it at compile time which is stored in the symbol table. Once a given object is out of scope, assuming no other object (which is still in the symbol table) holds a reference to it, the GC can then see the object can be cleaned up. [And just writing this I know this sounds very much like nonsense. After all, once something is out of scope at compile time, what is to stop the GC cleaning it up before runtime? Indeed one has to be careful about this. In my case, the source code itself is connected with the symbol table. The source code is itself represented as a linked list and the symbols in it are the symbols in the symbol table. Until the source is relinquished, the part of the symbol table that is currently in scope is not relinquished. And of course the source is not relinquished until after it has been "evaluated", i.e. Jit compiled and executed.] So far I haven't managed to trick it, and it is very efficient. But then again I'm using Boehm-Demers-Weiser which is possibly so conservative that nothing is actually getting cleaned up in the examples I am running. I simply don't know. I do know that this is a technical kludge and (I think) I can see it isn't going to work for a compiler, only for an interpreter.
• skrebbel 15y ago
  If this works out, it basically also means Scala->JavaScript through emscripten (https://github.com/kripken/emscripten https://github.com/kripken/emscripten). Slow, probably, and probably with nearly impossible interfacing with the DOM and JS libraries, but still - pretty cool
• smcj 15y ago
  I'm really interested in that idea. That would create the possibility to clean up the remaining Java cruft like "every object is a monitor", clone vs. Cloneable, ...