5 ms·

Scala on LLVM

148 points by ddispaltro 11y ago

nkvoll 11y ago
While interesting in and of it self, it seems to have seen no changes since mid-2013, looking at its GitHub repository (https://github.com/greedy/scala https://github.com/greedy/scala) > This branch is 211 commits ahead, 10086 commits behind scala:2.12.x. Is there any new context / development that makes this extra interesting right now?
oelang 11y ago
This project has been discontinued. But fear not, a new scala-native is on the way, Denys Shabalin a phd student at EPFL is working on completely new implementation of scala-native. His previous work, scala offheap, a way to use off-heap memory on the JVM with type- and memory-safety is very impressive.
- thinkpad20 11y ago
  Would native-compiled scala allow for some of the features that have been prevented by the need for Java semantic compatibility, like guaranteed tail call optimization?
  - densh 11y ago
    Yes, especially talking about better tail call optimisation.
  - duaneb 11y ago
    > like guaranteed tail call optimization? Isn't this what the @tailrec annotation is for? If it compiles it's tail recursive. This is separate than actually JIT'ing the call bytecode into a loop, I realize, but for a pragmatic engineer there's already a solution.
    - densh 11y ago
      @tailrec works only for simple cases of tail recursion. For example it doesn't support mutual tail recursion at the moment.
- JLangley 11y ago
  Really looking forward to hearing this talk! I wonder if there will be any potential for C++ interop. I would __LOVE__ to be able to code Scala that can be called from and in turn call C++ interfaces.
  - btown 11y ago
    I mean, there's nothing stopping you from writing JNI wrappers; they work with anything that uses the JVM. http://hohonuuli.blogspot.com/2013/08/a-simple-java-native-interface-jni.html http://hohonuuli.blogspot.com/2013/08/a-simple-java-native-i...
geodel 11y ago
The Github and mailing list shows no activity for >3 years. Either the idea did not work out or all they had was just idea and nobody came forward to pickup and develop working software. In any case Scala seems to have large following among those who love JVM and Java ecosystem libraries.
densh 11y ago
A small clarification: this an older project that is not directly related to my upcoming ScalaDays talk [1]. More updates to come in next few months, stay tuned. [1] http://event.scaladays.org/scaladays-nyc-2016#!#schedulePopupExtras-7571 http://event.scaladays.org/scaladays-nyc-2016#!#schedulePopu...
- Cyph0n 11y ago
  Wow, sounds really interesting. I can't wait for the talk!
- walkingolof 11y ago
  This sounds very interesting, allot of very interesting develoment is happening in Scala/JVM land, Dotty, Akka, Lagom etc For those of you that havnt heard about Dotty, the next generation Scala compiler, I can recommend this link: http://www.scala-lang.org/blog/2015/10/23/dotty-compiler-bootstraps.html http://www.scala-lang.org/blog/2015/10/23/dotty-compiler-boo...
0xFFC 11y ago
This may sound ridiculous, but I am literally sick of Java/JVM land. I would beg for native Scala backend (and it would be my first choice for doing pretty much every thing).I don't know people does share this opinion or not ,but I literally hate JVM.Slow Start up.Tremendously slow/unresponsive java app(I am talking about big apps, like android studio). Ridiculously bad font handling in Swing(there is no way to disable hinting). Syntactically and semantically Scala is one of the best language's I have ever worked with (although I spend little time with it).It's middle approach to OOP and FP is best approach for enterprise projects.It does not try to shove FP in your throat, but provide best platform to do FP when it is much more efficient than simple OOP. However when it comes to JVM (especially in Linux desktop). My answer is big fat NO. Of course, I am talking about desktop experience.JVM in server side is great and robust, I don't have anything against it, other than praise.
- geodel 11y ago
  Well in my experience for heaps greater than 10-12GB server side Java apps can have long and unpredictable GC pauses. As for desktop apps I do not think many will share your opinion ,mainly because in large enterprises such as where I work, Java app not freezing every single time it is invoked would be considered a great experience.
  - eklavya 11y ago
    It's very likely that Scala native will also be garbage collected. I don't think you will see improvements in gc pauses. JVM GCs have been tuned for years, so it will be an achievement to beat them. Aside: Shenendoah was supposed to be an improvement over existing GCs in pause times, but it's not yet ready if it's alive and it's for very big heaps and when you want low latency over performance/throughput.
- the_af 11y ago
  My opinion is exactly the opposite. The best thing about Scala is that it brings some niceties from FP-land while not looking entirely alien to Java developers, and you get to keep the JVM/Java ecosystem. Were it not for the Java-land compatibility & ecosystem requirement (and all the baggage associated with it, such as not alienating Java OOP programmers), I wouldn't use Scala. I'd probably ditch the OOP part of the FP+OOP hybrid that causes problems in Scala, and use something like Haskell instead. I honestly have no interest in a native/JVM-less Scala. There are better languages for that out there.
natrius 11y ago
This is a cool direction. Other than previous Scala knowledge, what are some advantages of using some sort of scala-native over Rust? I've used Scala and I always imagined that I'd use Rust if I wanted something similar without the ecosystem of JVM libraries.
- CheKhovHemingw3 11y ago
  Not to forget about Julia http://julialang.org/ http://julialang.org/ and its high-performance JIT compiler. A comparison with some of Julia's features: (1) Multiple dispatch: providing ability to define function behavior across many combinations of argument types Scala has the Cake Pattern (2) Built-in package manager SBT, anyone? (3) Automatic generation of efficient, specialized code for different argument types Dotty's Linker is on the horizon, https://d-d.me/talks/scaladays2015/#/ https://d-d.me/talks/scaladays2015/#/
- whateveracct 11y ago
  > Other than previous Scala knowledge, what are some advantages of using some sort of scala-native over Rust? For one, it isn't as low-level as Rust. That can be good or bad for your use-case. Also, Scala has a bit more expressive type system than Rust so you get that. Scala has scalaz/cats and shapeless which aren't really possible in Rust.
- lmm 11y ago
  Garbage collection (presumably - not sure what implementation they'll use, this is something that really matters for performance and where the JVM gives a big advantage). Higher-kinded types (supposedly coming in Rust eventually, but Scala has them today) and for/yield syntax. Easier closures that support some functional idioms better (made possible by having garbage collection). And even without Java libraries the library ecosystem for Scala is likely bigger than that for Rust.
- PeCaN 11y ago
  I think D is closer to Scala than Rust is. D's metaprogramming is quite a bit more advanced than Rust and it has opt-in garbage collection. The type system isn't advanced like Rust, let alone Scala, but it mostly gets the job done. It's like C and Scala had a remarkably pleasant child.
CheKhovHemingw3 11y ago
The other announcement[1] from EPFL last week was the establishment of Scala Center, which is already staffed: http://search.epfl.ch/ubrowse.action?acro=SCALA-GE http://search.epfl.ch/ubrowse.action?acro=SCALA-GE The Center's mission: "Organize open source projects around Scala". About scala-native, no details are given on whether it targets scalac or Dotty (which another talk [2] describes as a couple years away from production-ready). The situation could change if, say, Akka or Play announced a Roadmap for Dotty. [1] Martin Odersky: "Scala in 2016". http://files.meetup.com/18712511/Scala2016.pdf http://files.meetup.com/18712511/Scala2016.pdf [2] ScalaSphere: Guillaume Martres "Hacking on Dotty: a live demo". https://www.youtube.com/watch?v=0OOYGeZLHs4 https://www.youtube.com/watch?v=0OOYGeZLHs4
- gmartres 11y ago
  If you're interested in Dotty, other talks with more information are listed at https://github.com/lampepfl/dotty/wiki/Getting-Started#talks-on-dotty https://github.com/lampepfl/dotty/wiki/Getting-Started#talks...
deleted 11y ago
[deleted]
deleted 11y ago
[deleted]
deleted 11y ago
[deleted]
__jal 11y ago
I've clearly been paying too much attention to politics. My first scan of headlines here left me asking, in which case before the Supreme Court was LLVM an issue? Although I must admit I would love to read the late Scalia's opinion of LLVM.
ninjakeyboard 11y ago
Apart from startup times, would this really be faster than the JVM? I don't know how the LLVM runtime stacks up - I'm guessing that there are a few targets eg traits that might be more efficient to implement in native code as Scala would be trying to work around what exists in the JVM and Java Bytecode but I have a feeling that it would be very specific use cases like embedded systems that would be considering LLVM as a target where as the JVM would still be preferred in most circumstances? I'm asking from a fairly naive place - I'm genuinely interested and curious to hear some use cases where LLVM would be almost strictly better.
- pjmlp 11y ago
  Actually you could already do it today. RoboVM uses LLVM and quite a few People were using Scala with it. As "the JVM" there are plenty to choose from. Embedded vendors do have implementations that run with a few hundred KBs or 1-2 MB depending on the use case. I would say it is just yet another implementation. Personally I find healthy that languages have multiple implementations, specially since it clears the confusion between language and implementation.
ape4 11y ago
How about compile speed. I found Scala compiled much slower than Java.
- hibikir 11y ago
  Java's type system is barely doing anything for you in comparison with what Scala does. Higher kinded types, implicit parameters, implicit conversions and type inference are not free. The type level tools that libraries like Shapeless use let you do type checks that are unimaginable in the java compiler. Guess what: The more work a compiler is doing, the slower it's going to run. Comparing Java and Scala compilation is like comparing the gas mileage of an 18 wheeler truck and a small city car. Guess what? The 18 wheeler won't do 30 mpg.
  - dastbe 11y ago
    The scala type system is incredibly strong (and undecidable) but the implementation itself is not conducive to compilation speed. Two of the primary motivations for Dotty are 1) reducing the complexity of the language and 2) starting clean slate on a more reasonable compiler. Im of the opinion that most groups don't want the complexity that scala brings, and we're going to see that in the community growing around spark. The databricks style guide clearly avoids a lot the complexity of the language, and the people who are coming to scala for the tech and not the language are going to follow their lead.
  - ape4 11y ago
    I agree the Scala compiler has more to do. (Although Java now does type inference.) But I'm like any user: greedy. I want it all and I want it fast ;)

148 points by ddispaltro 11y ago

While interesting in and of it self, it seems to have seen no changes since mid-2013, looking at its GitHub repository (https://github.com/greedy/scala https://github.com/greedy/scala) > This branch is 211 commits ahead, 10086 commits behind scala:2.12.x. Is there any new context / development that makes this extra interesting right now?

oelang 11y ago

This project has been discontinued. But fear not, a new scala-native is on the way, Denys Shabalin a phd student at EPFL is working on completely new implementation of scala-native. His previous work, scala offheap, a way to use off-heap memory on the JVM with type- and memory-safety is very impressive.

thinkpad20 11y ago
Would native-compiled scala allow for some of the features that have been prevented by the need for Java semantic compatibility, like guaranteed tail call optimization?
- densh 11y ago
  Yes, especially talking about better tail call optimisation.
- duaneb 11y ago
  > like guaranteed tail call optimization? Isn't this what the @tailrec annotation is for? If it compiles it's tail recursive. This is separate than actually JIT'ing the call bytecode into a loop, I realize, but for a pragmatic engineer there's already a solution.
  - densh 11y ago
    @tailrec works only for simple cases of tail recursion. For example it doesn't support mutual tail recursion at the moment.
JLangley 11y ago
Really looking forward to hearing this talk! I wonder if there will be any potential for C++ interop. I would __LOVE__ to be able to code Scala that can be called from and in turn call C++ interfaces.
- btown 11y ago
  I mean, there's nothing stopping you from writing JNI wrappers; they work with anything that uses the JVM. http://hohonuuli.blogspot.com/2013/08/a-simple-java-native-interface-jni.html http://hohonuuli.blogspot.com/2013/08/a-simple-java-native-i...

geodel 11y ago

The Github and mailing list shows no activity for >3 years. Either the idea did not work out or all they had was just idea and nobody came forward to pickup and develop working software. In any case Scala seems to have large following among those who love JVM and Java ecosystem libraries.

densh 11y ago

A small clarification: this an older project that is not directly related to my upcoming ScalaDays talk [1]. More updates to come in next few months, stay tuned. [1] http://event.scaladays.org/scaladays-nyc-2016#!#schedulePopupExtras-7571 http://event.scaladays.org/scaladays-nyc-2016#!#schedulePopu...

Cyph0n 11y ago
Wow, sounds really interesting. I can't wait for the talk!
walkingolof 11y ago
This sounds very interesting, allot of very interesting develoment is happening in Scala/JVM land, Dotty, Akka, Lagom etc For those of you that havnt heard about Dotty, the next generation Scala compiler, I can recommend this link: http://www.scala-lang.org/blog/2015/10/23/dotty-compiler-bootstraps.html http://www.scala-lang.org/blog/2015/10/23/dotty-compiler-boo...

0xFFC 11y ago

This may sound ridiculous, but I am literally sick of Java/JVM land. I would beg for native Scala backend (and it would be my first choice for doing pretty much every thing).I don't know people does share this opinion or not ,but I literally hate JVM.Slow Start up.Tremendously slow/unresponsive java app(I am talking about big apps, like android studio). Ridiculously bad font handling in Swing(there is no way to disable hinting). Syntactically and semantically Scala is one of the best language's I have ever worked with (although I spend little time with it).It's middle approach to OOP and FP is best approach for enterprise projects.It does not try to shove FP in your throat, but provide best platform to do FP when it is much more efficient than simple OOP. However when it comes to JVM (especially in Linux desktop). My answer is big fat NO. Of course, I am talking about desktop experience.JVM in server side is great and robust, I don't have anything against it, other than praise.

geodel 11y ago
Well in my experience for heaps greater than 10-12GB server side Java apps can have long and unpredictable GC pauses. As for desktop apps I do not think many will share your opinion ,mainly because in large enterprises such as where I work, Java app not freezing every single time it is invoked would be considered a great experience.
- eklavya 11y ago
  It's very likely that Scala native will also be garbage collected. I don't think you will see improvements in gc pauses. JVM GCs have been tuned for years, so it will be an achievement to beat them. Aside: Shenendoah was supposed to be an improvement over existing GCs in pause times, but it's not yet ready if it's alive and it's for very big heaps and when you want low latency over performance/throughput.
the_af 11y ago
My opinion is exactly the opposite. The best thing about Scala is that it brings some niceties from FP-land while not looking entirely alien to Java developers, and you get to keep the JVM/Java ecosystem. Were it not for the Java-land compatibility & ecosystem requirement (and all the baggage associated with it, such as not alienating Java OOP programmers), I wouldn't use Scala. I'd probably ditch the OOP part of the FP+OOP hybrid that causes problems in Scala, and use something like Haskell instead. I honestly have no interest in a native/JVM-less Scala. There are better languages for that out there.

natrius 11y ago

This is a cool direction. Other than previous Scala knowledge, what are some advantages of using some sort of scala-native over Rust? I've used Scala and I always imagined that I'd use Rust if I wanted something similar without the ecosystem of JVM libraries.

CheKhovHemingw3 11y ago
Not to forget about Julia http://julialang.org/ http://julialang.org/ and its high-performance JIT compiler. A comparison with some of Julia's features: (1) Multiple dispatch: providing ability to define function behavior across many combinations of argument types Scala has the Cake Pattern (2) Built-in package manager SBT, anyone? (3) Automatic generation of efficient, specialized code for different argument types Dotty's Linker is on the horizon, https://d-d.me/talks/scaladays2015/#/ https://d-d.me/talks/scaladays2015/#/
whateveracct 11y ago
> Other than previous Scala knowledge, what are some advantages of using some sort of scala-native over Rust? For one, it isn't as low-level as Rust. That can be good or bad for your use-case. Also, Scala has a bit more expressive type system than Rust so you get that. Scala has scalaz/cats and shapeless which aren't really possible in Rust.
lmm 11y ago
Garbage collection (presumably - not sure what implementation they'll use, this is something that really matters for performance and where the JVM gives a big advantage). Higher-kinded types (supposedly coming in Rust eventually, but Scala has them today) and for/yield syntax. Easier closures that support some functional idioms better (made possible by having garbage collection). And even without Java libraries the library ecosystem for Scala is likely bigger than that for Rust.
PeCaN 11y ago
I think D is closer to Scala than Rust is. D's metaprogramming is quite a bit more advanced than Rust and it has opt-in garbage collection. The type system isn't advanced like Rust, let alone Scala, but it mostly gets the job done. It's like C and Scala had a remarkably pleasant child.

CheKhovHemingw3 11y ago

The other announcement[1] from EPFL last week was the establishment of Scala Center, which is already staffed: http://search.epfl.ch/ubrowse.action?acro=SCALA-GE http://search.epfl.ch/ubrowse.action?acro=SCALA-GE The Center's mission: "Organize open source projects around Scala". About scala-native, no details are given on whether it targets scalac or Dotty (which another talk [2] describes as a couple years away from production-ready). The situation could change if, say, Akka or Play announced a Roadmap for Dotty. [1] Martin Odersky: "Scala in 2016". http://files.meetup.com/18712511/Scala2016.pdf http://files.meetup.com/18712511/Scala2016.pdf [2] ScalaSphere: Guillaume Martres "Hacking on Dotty: a live demo". https://www.youtube.com/watch?v=0OOYGeZLHs4 https://www.youtube.com/watch?v=0OOYGeZLHs4

gmartres 11y ago
If you're interested in Dotty, other talks with more information are listed at https://github.com/lampepfl/dotty/wiki/Getting-Started#talks-on-dotty https://github.com/lampepfl/dotty/wiki/Getting-Started#talks...

deleted 11y ago

[deleted]

deleted 11y ago

[deleted]

deleted 11y ago

[deleted]

__jal 11y ago

I've clearly been paying too much attention to politics. My first scan of headlines here left me asking, in which case before the Supreme Court was LLVM an issue? Although I must admit I would love to read the late Scalia's opinion of LLVM.

ninjakeyboard 11y ago

Apart from startup times, would this really be faster than the JVM? I don't know how the LLVM runtime stacks up - I'm guessing that there are a few targets eg traits that might be more efficient to implement in native code as Scala would be trying to work around what exists in the JVM and Java Bytecode but I have a feeling that it would be very specific use cases like embedded systems that would be considering LLVM as a target where as the JVM would still be preferred in most circumstances? I'm asking from a fairly naive place - I'm genuinely interested and curious to hear some use cases where LLVM would be almost strictly better.

pjmlp 11y ago
Actually you could already do it today. RoboVM uses LLVM and quite a few People were using Scala with it. As "the JVM" there are plenty to choose from. Embedded vendors do have implementations that run with a few hundred KBs or 1-2 MB depending on the use case. I would say it is just yet another implementation. Personally I find healthy that languages have multiple implementations, specially since it clears the confusion between language and implementation.

ape4 11y ago

How about compile speed. I found Scala compiled much slower than Java.

hibikir 11y ago
Java's type system is barely doing anything for you in comparison with what Scala does. Higher kinded types, implicit parameters, implicit conversions and type inference are not free. The type level tools that libraries like Shapeless use let you do type checks that are unimaginable in the java compiler. Guess what: The more work a compiler is doing, the slower it's going to run. Comparing Java and Scala compilation is like comparing the gas mileage of an 18 wheeler truck and a small city car. Guess what? The 18 wheeler won't do 30 mpg.
- dastbe 11y ago
  The scala type system is incredibly strong (and undecidable) but the implementation itself is not conducive to compilation speed. Two of the primary motivations for Dotty are 1) reducing the complexity of the language and 2) starting clean slate on a more reasonable compiler. Im of the opinion that most groups don't want the complexity that scala brings, and we're going to see that in the community growing around spark. The databricks style guide clearly avoids a lot the complexity of the language, and the people who are coming to scala for the tech and not the language are going to follow their lead.
- ape4 11y ago
  I agree the Scala compiler has more to do. (Although Java now does type inference.) But I'm like any user: greedy. I want it all and I want it fast ;)