18 ms·

Scala Native v0.1

642 points by zepolud 10y ago

deleted 10y ago
[deleted]
gbersac 10y ago
That's a great news ! We are exclusively using scala at work for back end and I wonder if it could be interesting to switch new projects to scala native. Did you test scala native against well known and massive open source scala project ? Did the performance improved or regress ? Did you wrote a brand new scala compiler for native code ?
- sreque 10y ago
  I'm not involved with this project in any way, but I would expect performance to be overall worse with Scala Native. The advantages of Scala native are likely: 1) Much faster startup times 2) smaller memory footprint for small programs. 3) Potential for easier installation since no dependency on the JDK (assuming binaries are statically linked) So basically you could use scala native to cover some cases that are better covered by golang or rust right now. For large and long-running server-side processes, the JVM is still king.
- therealmarv 10y ago
  Never touch a running system ;) Scala on JVM is much more tested than the new shiny thing. Also don't expect improved performance... many people think that the JVM is bloated and makes programs slower (this is mostly not true). The downsides of the JVM are more memory consumption/footprint (when you have e.g. small servers or micro instances) and the cold startup time of the JVM itself (which is not relevant on a server in comparison to desktop Java apps). Would be interested to hear if any backend Scala projects like e.g. Play work on Scala Native.
  - bluejekyll 10y ago
    Long time Java lover here. I agree with all your points, but in the context of Java at least (does Scala support this?) there is no simple static binary that can be built and released, which includes the JVM. I think 1.9 will have this option, but this is something I didn't realize I missed until I started work with Rust and Go. It makes deployment so much simpler.
    - hocuspocus 10y ago
      In the age of containers it's really not that much harder to build and deploy a JVM app. Edit: thanks for the downvotes but you could at least tell me what's so crazy about my statement.
      - djsumdog 10y ago
        There are still some licensing issues. For instance, Atlassian has an official docker container to evaluate Confluence, but they don't support it in production since it uses OpenJDK and Confluence is still somewhat broken on OpenJDK. Rather than fix Confluence to work on OpenJDK (I don't want to imagine what type of reflection garbage they've got going on down there that breaks so bad on OpenJDK), their instructions tell you how to make your own Dockerfile using the official Oracle runtime. Actually, in that situation, if it won't run on OracleJDK it's probably not going to work via a native compiler either.
      - whatnotests 10y ago
        One app, sure. At work I'm running 17 different containers- many of which require their own JVM. (That's 3 different JRuby apps, zookeeper, kafka, and ElasticSearch.) Those JVMs get heavy when you're shipping container images compared to small Go or Rust binaries.
      - lmm 10y ago
        To my mind the JVM is where containers make the least sense. If you build an executable jar you can run with "java -jar ..." then that seems just as simple as "docker run ..." and gets you the single-file deployment, and you can control memory allocation via flags if you need to. You don't get virtual networking but IME that doesn't add value in the first place.
  - algesten 10y ago
    > the cold startup time of the JVM itself (which is not relevant on a server in comparison to desktop Java apps). I disagree somewhat with this. We found that when we started writing microservices in languages that are not java, the short startup time changed how we did some error handling. For errors where we say lose connection to the database, or rabbitmq, we much rather have the nodejs-process die and restart, than try to construct reconnect logic. The problem with reconnect-logic is that it is code that (may) be tested very rarely. This in turn means it's easy to get strange long term problems there like very slow memory leak due to some listener being added to a connection object once the connection is initiated. We did a 180 on reconnect-logic in our nodejs-processes and let the exceptions just bubble unhandled and take the entire vm down. With automatic restart script, the process will be back in seconds anyway, and with docker having built in back-off timers for auto restart, we don't necessarily overload the shared resources.
    - mason55 10y ago
      I guess I don't see the difference here if your VM startup time is 3 seconds or 15-30 seconds. If that's the difference between the site remaining stable and the whole thing collapsing then it seems like you're setting yourself up for a big outage one day when the nodejs process isn't able to come back in three seconds for whatever reason.
      - algesten 10y ago
        I think it depends a bit on class of errors. Certainly not everything is suitable for this treatment. Lost connectivity to RabbitMQ or Elasticsearch would mean our site is dead anyhow (you can't do anything). So either of those errors should arguably result in some static 500 pardon-our-appearance page. But say someone messes up the network connection or we get a brief problem. Why wouldn't the nodejs process start quickly?
    - wtetzner 10y ago
      > For errors where we say lose connection to the database, or rabbitmq, we much rather have the nodejs-process die and restart, than try to construct reconnect logic. This sounds like a very Erlang-ish way to handle the problem. Another advantage is that if the server/process is in some weird state that's causing problems, killing and restarting it lets you clear out the broken state, and get back into the state that it's most likely been tested under.
  - manojlds 10y ago
    I'm interested in building CLIs with this.
  - acjohnson55 10y ago
    At the moment, there isn't direct support for multithreading [1], so I'm guessing it would be very difficult to run any of the common web servers or computing frameworks natively. It may be possible for libraries that have pluggable concurrency, for example by creating an `ExecutionContext` that wraps OS threads, but that's waaay beyond my pay grade. [1] http://www.scala-native.org/en/latest/user/lang.html#lang http://www.scala-native.org/en/latest/user/lang.html#lang
    - conjectures 10y ago
      Cheers - this is the #1 item on my excitement checklist.
  - dragonwriter 10y ago
    > the cold startup time of the JVM itself (which is not relevant on a server in comparison to desktop Java apps). With microservices run as containers that are started on demand it matters; with other architectural choices it may matter less.
- LeanderK 10y ago
  I think for Server-Application Scala on the JVM will probably beat Scala-Native. The benefits of Scala-Native over Scala JVM are: - faster startup time - (drastically) lower memory footprint - fine hand-tuning of you application All these things are not super important in server-applications. For example Java trades memory for throughput (higher memory footprint, but also higher throughput. These usually go hand in hand.).
  - therealmarv 10y ago
    But can be important if you only have a 512MB DigitalOcean instance or using small Linux containers/Docker.
    - LeanderK 10y ago
      I used the JVM on the 512MB DO instances and containers and they run fine. I think for containers there are other issues (most likely you are going in a micro service-direction where latency is eventually going to be important, so picking another JVM GC-algorithm might be suitable). There may be applications for which the 512 instances and JVM are not suitable, but you can most likely just upgrade the instance.
    - sreque 10y ago
      The JVM itself doesn't add a huge memory footprint. I remember Charles Nutter of JRuby fame calling it the "20-30 MB memory tax" (see http://blog.headius.com/2008/11/noise-cancelling.html http://blog.headius.com/2008/11/noise-cancelling.html). A lot of the extra memory usage of Java apps comes from sloppy programming and from depending on lots of heavyweight libraries and frameworks.
      - geodel 10y ago
        If you include idiomatic Java programing as part of sloppy programming I also agree with that. https://www.cs.virginia.edu/kim/publicity/pldi09tutorials/memory-efficient-java-tutorial.pdf https://www.cs.virginia.edu/kim/publicity/pldi09tutorials/me...
- q3r3qr3q 10y ago
  Yeah go for it. Nothing could go wrong: https://github.com/scala-native/scala-native/issues/543 https://github.com/scala-native/scala-native/issues/543
LeanderK 10y ago
Is Scala Native GCed?
- merb 10y ago
  yes (https://github.com/scala-native/scala-native/issues/128 https://github.com/scala-native/scala-native/issues/128)
- zerr 10y ago
  Should be, because Scala doesn't have strong vs weak reference distinction.
wst_ 10y ago
Somewhat relevant: https://blog.plan99.net/kotlin-native-310ffac94af2#.ijzik0jxx https://blog.plan99.net/kotlin-native-310ffac94af2#.ijzik0jx... Title says Kotlin, but it is about JVM languages going native, in general. Or should they?
- sreque 10y ago
  They should. The article basically throws in some FUD to convince people that they don't want what they think they want. Except, we do want it! The JVM ecosystem's general aversion to native code and native system integration is what gave C# the opportunity to take over as the closest thing we have to a WORA language. If I want to write code in a higher-level language than C++ that can run on any mobile device or desktop OS, I do it in C#, not Java. The Java ecosystem is playing huge catch up here, and I don't think it's a bad thing that people are exploring alternatives, whether that's Avian, the now defunct RoboVM, or LLVM-based backends.
  - pjmlp 10y ago
    The JVM eco-system is full of options to compile Java into native code. The only thing is that most ignore there is a JVM world outside OpenJDK. And even those that know that world aren't willing to pay for commercial JVMs. Thus propagating the myth that Java doesn't support AOT. Which in a way is also understandable, given Sun's political decision that AOT compilation was tabu. Luckily Oracle isn't Sun and has heard the industry. So if all goes as planned, by Java 10, those that don't want to pay for AOT compilers, will have one in OpenJDK.
lukax 10y ago
The generated binary for simple Hello World is 3.45 MB which is quite a lot for printing one line of text but it can be compressed to 326K using UPX.
- densh 10y ago
  All of our binary size problems are mostly caused by a single well-known issue [1]. We hope to address that in one of the future releases. In the meanwhile, compressing binaries with upx is a temporary solution that can alleviate some of the pain. [1] https://github.com/scala-native/scala-native/issues/180 https://github.com/scala-native/scala-native/issues/180
- cderwin 10y ago
  Rust's is almost the exact same size (3.46 MB). It's probably just that the libraries that are statically linked by default (jemalloc, std, etc.).
  - densh 10y ago
    That's correct. We statically link all of the transitive Scala dependencies.
jcstauffer 10y ago
Great! Any benchmarks on whether the Scala compiler runs faster when compiled to native?
- densh 10y ago
  We're looking into that, but nothing to report yet.
speg 10y ago
I can't get hello_world to work, something about a unresolved dependency: org.scala-native#sbt-Scalia-native;0.1.0: not found I'm not a regular Scala user.
- densh 10y ago
  Please drop by our gitter chatroom [1] and we'll try to debug the issue together. It's likely some sort of environment misconfiguration. [1] https://gitter.im/scala-native/scala-native https://gitter.im/scala-native/scala-native
- manojlds 10y ago
  I am a regular Scala user and I hate that message
mafribe 10y ago
Nice work, I hope this will eventually be a serious alternative to the JVM route. Quick question: does this compile down to DOT before going to LLVM? Or has DOT not yet arrived in Scala Native?
- densh 10y ago
  DOT [1] is a theoretical calculus that is used as a foundation for our latest front-end compiler called dotty [2]. Scala Native is mostly a middle-end technology that bridges front-end compiler and LLVM. We plan to support dotty as a front-end in the future. 0.1 release is based on older Scala 2.11 front-end. [1] http://www.scala-lang.org/blog/2016/02/03/essence-of-scala.html http://www.scala-lang.org/blog/2016/02/03/essence-of-scala.h... [2] https://github.com/lampepfl/dotty https://github.com/lampepfl/dotty
  - bnjmn 10y ago
    Just what I hoped to hear, thanks!
andrewvijay 10y ago
scala no-exp here. How is this different from regular scala running on a jvm? There is no need of jvm for scala native?
cwyers 10y ago
It seems to me like Scala's biggest benefit and biggest downside are two sides of the same coin: easy interop with the JVM and Java code. Scala Native just seems like you're paying all the price of that for none of the benefit.
- dillonb 10y ago
  Well, this is just targeting a different VM, LLVM instead of the JVM. You get easy interop with all LLVM languages with this, including C.
  - krona 10y ago
    Easy interop with C++ or Rust? I'll believe it when I see it.
    - dillonb 10y ago
      It looks like it can! http://www.scala-native.org/en/latest/user/interop.html http://www.scala-native.org/en/latest/user/interop.html >Scala Native provides an interop layer that makes it easy to interact with foreign native code. This includes C and other languages that can expose APIs via C ABI (e.g. C++, D, Rust etc.)
      - krona 10y ago
        > languages that can expose APIs via C ABI That's what I thought.. another C-based FFI. Better than nothing, I suppose.
      - username223 10y ago
        From that page, it looks like Scala-C interop is decent, but that's a far cry from C++/Rust interop. For C++ at least, you more or less need to write a pure-C wrapper API to call from Scala, since it doesn't handle C++ types.
        steveklabnik 10y ago
Negative1 10y ago
Important bit: "The project has reached a point of feature completeness in terms of the coverage of the Scala language. We support the whole language including the more advanced features such as method dispatch via structural types and even macros." It must be frustrating to work on a project like this, see areas where the language can be improved, but only be able to do the work to make it purely compatible. Hopefully some good comes out in the form of some good SIPs.
- acjohnson55 10y ago
  Or inspiring! I guess it depends on your outlook.
gravypod 10y ago
Does anyone have any example binaries compiled with this? What are the sizes that you could expect? They say one of their targets is using this for command-line tools (I'm guessing for startup speed and needing to be small in memory footprint) but it's not of much value if an "echo" or "grep" implementation takes up 15 to 30MB on the drive.
- dualogy 10y ago
  > but it's not of much value if an "echo" or "grep" implementation takes up 15 to 30MB on the drive In this day & age? It might be of value if an echo or grep takes ~20MB and a full-blown HTTP REST server with DB access takes ~25-30MB. Why do I care again exactly whether there is a fixed portion of bootstrap/runtime core code in the binary, as long as I'm not writing echo or grep? (Any helloworld app not written in asm (or lib-less C) will consist largely of "non-helloworld code" in terms of "bytes occupied in the binary", right?) Dunno might be of concern eg with embedded/raspberry and such, but for code to be "moved from a jvm-kinda flair to a go-kinda flair" I'm not seeing the issue .. yet ;)
  - usrusr 10y ago
    When you are writing that full-blown HTTP REST server with DB access you would probably be served better on the JVM. Scala native is just as garbage collected as Scala JVM and garbage collection is where the JVM shines brightly. In my eyes, the main value proposition of Scala native will be "you don't have to learn a new language if you need to write an echo or a grep", with "you don't need to learn a new language if you need to run some of your code on a platform that does not have a reasonably good JVM" a distant second. The former would suffer a lot from oversized binaries, the latter depending on the specifics of the target environment.
    - Recurecur 10y ago
      A major value proposition is also "I want to run my code with predictable latency". This opens Scala to embedded and realtime development. The LLVM AOT optimizations are also likely to be a lot stronger than JVM JIT optimizations - granted it's good to compile on the exact target architecture. Also granted that sometimes dynamic optimization helps a lot. Nice handle BTW. lol
amelius 10y ago
Does this provide a garbage collector?
- cmsimike 10y ago
  I asked a similar question on the reddit thread about this. I'm bummed that I didn't find this answer right away in this documentation. Probably the most interesting question for me
- mark_l_watson 10y ago
  Boom GC is a dependency
- deleted 10y ago
  [deleted]
- ddispaltro 10y ago
  Yeah it uses the Boehm garbage collector.
huula 10y ago
Question: what kind of frameworks can be practically migrated to Scala Native?
- pale-hands 10y ago
  I think the dependencies (like Netty for a web framework) would have to be migrated first. Scala native has a subset of java core libraries (from e.g. io, nio, util) rewritten in Scala, which could be a blocker if there's anything missing.
- lmm 10y ago
  In Scala it's very normal to write libraries or frameworks in "pure" Scala (i.e. not using reflection, annotations, proxies or anything like that), and all of those should be fine to cross-build for Scala Native (assuming their upstream dependencies do first). The state of libraries for Scala.js should be a good indication - anything that's cross-compatible with that is likely to work just as well for Scala Native.
pale-hands 10y ago
Will macros work with Scala Native, as they do with ScalaJs? (I believe that compile-time metaprogramming is the way forward, especially if the target doesn't support reflection or dynamic code loading).
- ddispaltro 10y ago
  "We support the whole language including the more advanced features such as method dispatch via structural types and even macros."
  - pale-hands 10y ago
    D'oh, thanks! Can't believe I missed that.
- coldsauce 10y ago
  If i recall correctly, they are planning to (or have already) remove macros in a newer version of Scala.
  - pale-hands 10y ago
    Yeah, the current macro system will be replaced with scala-meta, which is a different macro system, but it hasn't happened yet. Scala-meta doesn't have def macros yet, so it's not ready.
evdev 10y ago
As a Scala guy on a Scala team, I'd think this would be most immediately useful on smaller fill-the-gaps sub-projects where we have to integrate with native code.
- manojlds 10y ago
  Looks like a good fit for the CLI we have been planning for our API bases on a Scala backend.
auggierose 10y ago
It seems like that would open up a nice way of using Scala on iOS also.
0xFFC 10y ago
so dream comes true! P.S. I think this is related to rust, in a sense before Rust there was no serious competitor to C/C++, but after seeing what Rust doing to C/C++ I think there will be more native language to compete with in low level area.
- usrusr 10y ago
  Between the "scripty" GC agility of Golang and all the impressive low level benefits that Rust is enjoying from folding memory management into the type system, I suspect that Scala native will be a difficult sell. On the other hand, the new targets (js and native) might make Scala more interesting to Java pragmatists who don't care much about going functional but would not mind writing Java in a more streamlined syntax.
  - Recurecur 10y ago
    Golang is pretty deficient as languages go, really. I find Scala to be far more appealing.
brangalinafoeva 10y ago
How do the compilation times compare to targeting the JVM?
- densh 10y ago
  There is roughly 1-2s (after sbt warm-up) penalty to perform compilation to native on iMac (Retina 5K, 27-inch, Late 2015) with 4 GHz Intel Core i7. The linker is highly parallel so you can throw more hardware at it as your project grows bigger. Barebones project includes nearly 1000 classes and 2000 methods in transitive dependencies that are being compiled due to closed-world nature of the toolchain.
mark242 10y ago
This will be huge for getting Scala running on AWS Lambda. The cold-start times for JVM apps is just ridiculous and makes Lambda/API gateway essentially unusable for anything written on the JVM.
- sjrd 10y ago
  Actually, there are people using Scala.js to run Scala on AWS lambda, precisely for that reason. * https://github.com/tptodorov/aws-lambda-scalajs https://github.com/tptodorov/aws-lambda-scalajs * http://underscore.io/blog/posts/2016/03/21/serverless-scale-summit.html http://underscore.io/blog/posts/2016/03/21/serverless-scale-...
  - acjohnson55 10y ago
    ^ From the creator of Scala.js :) I am excited that now there will be an option for getting both quick startup and fastest execution time!
  - danaliv 10y ago
    Sure, but it'd be nice not to have everything in the known universe depend on JavaScript.
    - sjrd 10y ago
      Absolutely! I'm just pointing out that this need is so strong that people have reached for the slow throughput of Scala.js just to get its fast startup time. If we can have both, that's even better!
      - edko 10y ago
        No intention of arguing with Scala.js' creator :-) But, I do not think it has a "slow throughput" at all. In my experience, it's had pretty good performance.
        sjrd 10y ago
zabed143 10y ago
Clipping Service India is an online outsourcing company. We provide service in the field of photo editing and graphic design, specialist in... Clipping Service India is an online offshore graphic design studio providing clipping path, Photoshop masking, image manipulation, image retouching, Color Correction, drop shadow, Reflection shadow, Multiple clipping path, raster to vector, Website image optimization, all types of photo editing service... Clipping Service India is established in order to provide image editing services through internet and this internet makes our life very easy and comfortable to communicate around the world's people within in a click on mouse. http://www.clippingserviceindia.com/index.php http://www.clippingserviceindia.com/index.php
rekado 10y ago
Unfortunately, this requires an existing Scala compiler to build, so it won't be useful as a bootstrap compiler for Scala on the JVM. Does anyone here know of an alternative implementation of Scala that could be used to build the libraries and tools of the reference implementation from source? It is a problem that many compilers cannot be bootstrapped from source without a trusted binary of a previous release.
- ajross 10y ago
  It's a universal truism that all compilers cannot be bootstrapped from source without a trusted binary. It's true that in the world of standardized languages (which mostly means "C" in practice) there are compilers (mostly just gcc and clang) that can bootstrap themselves with a trusted binary of a previous release of some other compiler. Is that such a big deal?
  - rekado 10y ago
    I wrote "a trusted binary of a previous release" not just "a trusted binary". There is obviously a difference between having a small set of trusted binaries to bootstrap and having a bootstrap binary for every language or build system. There are several compiler implementations that enable bootstrapping from alternative implementations, which shifts the problem to a simpler language, which may already have a bootstrap path. See also http://bootstrappable.org/best-practises.html http://bootstrappable.org/best-practises.html
- platz 10y ago
  how big of a problem is it, really? is it a theoretical problem or a practical problem for industry users?
  - rekado 10y ago
    It is a practical problem for people who want to have a correspondence between source and binary. Some users would like to have to rely on as few opaque binaries as possible. There are efforts underway to build a minimal C compiler in a subset of Scheme that can be implemented on bare metal. The goal is to reduce the set of binaries that needs to be trusted or audited manually.
rainhacker 10y ago
> This opens the door for Scala to be used in environments where full-blown virtual machine is usually an overkill Not sure if I get this, don't Java VMs support this use case (J2ME) ?
crudbug 10y ago
Will language support destructors for manual memory management ?
- densh 10y ago
  Please file a feature request on http://github.com/scala-native/scala-native http://github.com/scala-native/scala-native, we can discuss the details over there.
tejasmanohar 10y ago
Does this mean the future of Scala is off the JVM? I ask because the post calls the JVM impl. a "reference implementation".
- sjrd 10y ago
  IMO the future of the JVM is highly multi-platform: JVM, native and JS.
- emodendroket 10y ago
  Does that not mean that it's the canonical one which other implementations should be measured against? That would seem to imply just the opposite.
squar1sm 10y ago
I wonder if Akka will be a part of scala native? It's sort of considered stdlib? When our team spiked on Akka, we liked it and got our near-reality proof of concept to work. It'd be awesome to have such a high level library like Akka compile to a binary.
scotchmi_st 10y ago
I would love for there to be a similarly thorough project with Clojure. It really bothers me there's no good native compiler. Apart from anything else, it means that Clojure lives and dies by the languages it compiles to, and while Java is used everywhere still, it probably isn't the thing the kids are learning these days. Besides, without going into any further rational arguments for why using the JVM (or another VM) isn't always great, something about it feels a bit icky to me. On an aesthetic level.
- pjmlp 10y ago
  > isn't the thing the kids are learning these days Yep, they learn Ruby and then they need to go JRuby when performance becomes relevant. :)
  - jwdunne 10y ago
    Or they tend to just Go. Interestingly, I've read a lot of Go users come from the Ruby and Python communities.
    - pryelluw 10y ago
      Im one of those. Anytime Python / Django doesnt cut it I use Go. Works very well.
      - jwdunne 10y ago
        Have you got a few examples where you had to replace the former for the latter? Really interesting.
        pryelluw 10y ago
        Sure. Last one was a static asset server that had to perform or the users would notice a delay. Built one in Go in about a week and put it behind nginx on a vps. It was faster than required, stand alone, and used very little memory. Its still running to this day.
c-smile 10y ago
I've created once custom barebone JavaVM with binary size of around 100k. It was made as an executable jsmile.exe that was capable to read bytecodes cat'ed to the executable itself : http://www.terrainformatica.com/org/j-smile/index.htm http://www.terrainformatica.com/org/j-smile/index.htm The goal was to create JVM suitable for standalone GUI applications. Project was abandoned when Sun/MS Java wars started in favor of the Sciter (https://sciter.com https://sciter.com). As of NativeScala ... I think that approach (binary with nano JVM + attached class files) may work better and with less effort. Scala needs JVM infrastructure, GC, etc. as far as I understand.
eli_gottlieb 10y ago
This is amazing. Scala is one of my favorite languages to work with, and getting it native-code support will finally help make it fast enough to justify using it everywhere.
crustycoder 10y ago
The lack of thread support is a deal breaker for me though.
mihaela 10y ago
Most things called native are not.
kelvich 10y ago
Nice job, thanks! Seems that right now you had implemented some wrappers for libc and some java classes ported to scala. What plans do you have to further evolve API? Will you focus on reimplementing java.* or create your own set of classes?
kentosi 10y ago
This is extremely exciting. I can't wait to try this out. On the other hand, I wonder why such an effort was never carried out with Java itself? Or maybe it was but just never took off?
- wtetzner 10y ago
  There was RoboVM[1], but it looks like it shut down. GCJ [2] was GCC's Java implementation, which was natively compiled, but I think has been removed from GCC. I'm sure there are other projects. [1] https://robovm.com/ https://robovm.com/ [2] https://en.wikipedia.org/wiki/GNU_Compiler_for_Java https://en.wikipedia.org/wiki/GNU_Compiler_for_Java
  - invalidname 10y ago
    Codename One predated RoboVM, was always larger and is still open source and well supported: https://www.codenameone.com/ https://www.codenameone.com/
- david-given 10y ago
  Would this be a good time to mention that a few years ago I wrote an experimental Java AOT compiler which worked by converting bytecode into C++? http://cowlark.com/cowjac http://cowlark.com/cowjac I used Apache Harmony as the standard library, and binaries were about 2MB, which back in 2012 I considered unacceptably large; benchmarks were adequate only. I never got round to implementing a garbage collector but the hooks were in place for a simple mark/sweep stop-the-world collector. The hard part would be to actually make all threads of the program stop so they could be collected; I had a plan, but no implementation. Interested parties may also be interested in my experimental Java JIT which worked by converting bytecode into Lua and then running it with LuaJIT: http://cowlark.com/luje http://cowlark.com/luje It actually beat OpenJDK Server for some unrepresentative microbenchmarks. It's very unfinished, and generates fairly pathological Lua code which LuaJIT has trouble with, but the biggest problem is that Lua and LuaJIT can't do shared-memory threading, so most Java code won't work on it. (JVMs are surprisingly easy to write. If you're interested in interpreters or code generation, I encourage you to try it.) (Man, I completely forgot I wrote these until now!)
webserg 10y ago
good idea!
enjoiful 10y ago
At first glance of this article's title, I thought it would be a terrible idea to use Scala to write native mobile applications. Imagine using Scala.JS to write a NativeScript/React Native app. shutters
- sjrd 10y ago
  > Imagine using Scala.JS to write a NativeScript/React Native app. People do [1], and are very happy with it. [1] https://github.com/chandu0101/sri https://github.com/chandu0101/sri
NickHoff 10y ago
Will this mean that we can get rid of type erasure when running native code?
- tormeh 10y ago
  Why is type erasure necessary anyway? Surely this can be worked around somehow. You could manually do what you do with array length in C - pass the information along manually. Why can't the compiler do this in a transparent manner for us? Is it speed?
- lmm 10y ago
  It's more likely to make type erasure more extensive. The main effect of type erasure is to stop people doing naughty things in generic methods. When one doesn't have to support JVM reflection, erasure can be pushed further.
twic 10y ago
This is certainly an impressive piece of work. However, i think it's worth paying attention to the limitations, and the use cases they imply; overall, this looks less like "compile your existing Scala app to native code!" and more like "use Scala to interface with existing native libraries!". On the other hand, it's also worth bearing in mind that this is version 0.1.0; over time, some of these limitations will lift. What i don't know is whether Scala Native will develop into a complete version of Scala which compiles to native code, or evolve into a variant of Scala more tightly adapted to a niche of talking to native libraries. Anyway ... (1) No threading [1]: Scala Native doesn’t yet provide libraries for parallel multi-threaded programming and assumes single-threaded execution by default. It’s possible to use C libraries to get access to multi-threading and synchronization primitives but this is not officially supported at the moment. So forget about using Akka for now. (2) NullPointerExceptions are replaced with segfaults (hopefully) [1]: A number of error conditions which are well-defined on JVM are undefined behavior: Dereferencing null. Division by zero. Stack overflows. Those typically crash application with a segfault on the supported architectures. That's not so bad; where Java apps might let nulls flow around and rely on catching NullPointerExceptions to recover from them, Scala apps are much more likely to use Optional consistently. (3) If you do want to talk to a native library, and you need to allocate memory to do it, you're on your own [2]: Unlike standard Scala objects that are managed automatically by the underlying runtime system, one has to manage native pointers manually. Scala Native provides a built-in way to perform stack allocations of unmanaged memory using native.stackalloc function: [...] When using stack allocated memory one has to be careful not to capture this memory beyond the lifetime of the method. Dereferencing stack allocated memory after the method’s execution has completed is undefined behaviour. Scala Native’s library contains a bindings for a subset of the standard libc functionality. This includes the trio of malloc, realloc and free functions Java's traditional JNI is a verbose, slow, pain in the stdout, but it was designed pretty carefully to avoid problems like this. (a) Intermission! Check out how they do type-level numbers [1]: Natural numbers are types that are composed of base naturals Nat._0, ... Nat._9 and an additional Nat.Digit constructor. That's a new one on me! (4) Incomplete JDK libraries [3]: Scala Native supports a subset of the JDK core libraries reimplemented in Scala. Here is the list of currently available classes: [...] This is an ongoing effort, some of the classes listed here might be partially implemented. The list has most of the fundamental stuff - a good chunk of java.io and NIO, the collections, java.lang, atomics. But no java.text, java.net, concurrency, regexp, date and time, JDBC, reflection, XML, etc. They don't mention how much of the Scala libraries they support. I would imagine that they can build anything that's in pure Scala and depends only on JDK classes in that list, so you'll get the core language stuff and the collections. Not sure. [1] http://www.scala-native.org/en/latest/user/lang.html http://www.scala-native.org/en/latest/user/lang.html [2] http://www.scala-native.org/en/latest/user/interop.html http://www.scala-native.org/en/latest/user/interop.html [3] http://www.scala-native.org/en/latest/lib/javalib.html http://www.scala-native.org/en/latest/lib/javalib.html
lacampbell 10y ago
Scala is a language I desperately wanted to like - high level, statically typed pure OO language. But in practice I found it almost unusable. The type signatures were unreadable and I distinctly recall writing a 100 line or so program where the type declarations crashed the compiler. And the tools themselves were huge memory hogs - sbt was a particularly bad offender (though otherwise quite pleasant). I also did not get on well with the community, which seemed to have a lot of people with the attitude - "they won't let me use haskell at work so I'll make do with this shit". They didn't seem to understand or be interested in OO at all, and were very fanatical about driving application logic with types, purity, and the like. Regardless, a native variant would be something well worth investigating if it ever reaches "production ready".
- eklavya 10y ago
  It looks like you dislike the haskell like usage of Scala, I wonder how could you then produce types so complicated/weird that you crashed the compiler in 100 lines (In 4 years I have seen it crash only once in an unreleased version of pickling a long time ago).
  - lacampbell 10y ago
    I never claimed my criticism was fair or consistent :P I think I was more interested in Haskell-like type driven programming at the time though. I honestly can't remember what it was. And it's not so much that I hate type driven programming or purity. I just view them as tools in my toolbox, not as methodologies or worse - religions. I still think objects are an amazing concept if you look at them through the smalltalk lens. Or if you take a moment to step back and realise that an anonymous function is very similar to an object with an "apply" method, or that a constructor for an immutable object is like a functor that partially applies many related functions at once.
    - eklavya 10y ago
      In my opinion you are mostly describing Scala. Mutability and impurity is trivially easy to do in Scala. You may see that as a positive or a negative. Local mutability is definitely easier. Complexity of Scala comes from mixing OO and FP worlds and I think it's an amazing engineering feat. It will undoubtedly look ugly after you do Haskell though.
deleted 10y ago
[deleted]
deleted 10y ago
[deleted]
anta40 10y ago
Any Windows pre-built binary to try? I imagine building this stuff on Windows will be challenging :/

642 points by zepolud 10y ago

deleted 10y ago

[deleted]

That's a great news ! We are exclusively using scala at work for back end and I wonder if it could be interesting to switch new projects to scala native. Did you test scala native against well known and massive open source scala project ? Did the performance improved or regress ? Did you wrote a brand new scala compiler for native code ?

sreque 10y ago
I'm not involved with this project in any way, but I would expect performance to be overall worse with Scala Native. The advantages of Scala native are likely: 1) Much faster startup times 2) smaller memory footprint for small programs. 3) Potential for easier installation since no dependency on the JDK (assuming binaries are statically linked) So basically you could use scala native to cover some cases that are better covered by golang or rust right now. For large and long-running server-side processes, the JVM is still king.
therealmarv 10y ago
Never touch a running system ;) Scala on JVM is much more tested than the new shiny thing. Also don't expect improved performance... many people think that the JVM is bloated and makes programs slower (this is mostly not true). The downsides of the JVM are more memory consumption/footprint (when you have e.g. small servers or micro instances) and the cold startup time of the JVM itself (which is not relevant on a server in comparison to desktop Java apps). Would be interested to hear if any backend Scala projects like e.g. Play work on Scala Native.
- bluejekyll 10y ago
  Long time Java lover here. I agree with all your points, but in the context of Java at least (does Scala support this?) there is no simple static binary that can be built and released, which includes the JVM. I think 1.9 will have this option, but this is something I didn't realize I missed until I started work with Rust and Go. It makes deployment so much simpler.
  - hocuspocus 10y ago
    In the age of containers it's really not that much harder to build and deploy a JVM app. Edit: thanks for the downvotes but you could at least tell me what's so crazy about my statement.
    - djsumdog 10y ago
      There are still some licensing issues. For instance, Atlassian has an official docker container to evaluate Confluence, but they don't support it in production since it uses OpenJDK and Confluence is still somewhat broken on OpenJDK. Rather than fix Confluence to work on OpenJDK (I don't want to imagine what type of reflection garbage they've got going on down there that breaks so bad on OpenJDK), their instructions tell you how to make your own Dockerfile using the official Oracle runtime. Actually, in that situation, if it won't run on OracleJDK it's probably not going to work via a native compiler either.
    - whatnotests 10y ago
      One app, sure. At work I'm running 17 different containers- many of which require their own JVM. (That's 3 different JRuby apps, zookeeper, kafka, and ElasticSearch.) Those JVMs get heavy when you're shipping container images compared to small Go or Rust binaries.
    - lmm 10y ago
      To my mind the JVM is where containers make the least sense. If you build an executable jar you can run with "java -jar ..." then that seems just as simple as "docker run ..." and gets you the single-file deployment, and you can control memory allocation via flags if you need to. You don't get virtual networking but IME that doesn't add value in the first place.
- algesten 10y ago
  > the cold startup time of the JVM itself (which is not relevant on a server in comparison to desktop Java apps). I disagree somewhat with this. We found that when we started writing microservices in languages that are not java, the short startup time changed how we did some error handling. For errors where we say lose connection to the database, or rabbitmq, we much rather have the nodejs-process die and restart, than try to construct reconnect logic. The problem with reconnect-logic is that it is code that (may) be tested very rarely. This in turn means it's easy to get strange long term problems there like very slow memory leak due to some listener being added to a connection object once the connection is initiated. We did a 180 on reconnect-logic in our nodejs-processes and let the exceptions just bubble unhandled and take the entire vm down. With automatic restart script, the process will be back in seconds anyway, and with docker having built in back-off timers for auto restart, we don't necessarily overload the shared resources.
  - mason55 10y ago
    I guess I don't see the difference here if your VM startup time is 3 seconds or 15-30 seconds. If that's the difference between the site remaining stable and the whole thing collapsing then it seems like you're setting yourself up for a big outage one day when the nodejs process isn't able to come back in three seconds for whatever reason.
    - algesten 10y ago
      I think it depends a bit on class of errors. Certainly not everything is suitable for this treatment. Lost connectivity to RabbitMQ or Elasticsearch would mean our site is dead anyhow (you can't do anything). So either of those errors should arguably result in some static 500 pardon-our-appearance page. But say someone messes up the network connection or we get a brief problem. Why wouldn't the nodejs process start quickly?
  - wtetzner 10y ago
    > For errors where we say lose connection to the database, or rabbitmq, we much rather have the nodejs-process die and restart, than try to construct reconnect logic. This sounds like a very Erlang-ish way to handle the problem. Another advantage is that if the server/process is in some weird state that's causing problems, killing and restarting it lets you clear out the broken state, and get back into the state that it's most likely been tested under.
- manojlds 10y ago
  I'm interested in building CLIs with this.
- acjohnson55 10y ago
  At the moment, there isn't direct support for multithreading [1], so I'm guessing it would be very difficult to run any of the common web servers or computing frameworks natively. It may be possible for libraries that have pluggable concurrency, for example by creating an `ExecutionContext` that wraps OS threads, but that's waaay beyond my pay grade. [1] http://www.scala-native.org/en/latest/user/lang.html#lang http://www.scala-native.org/en/latest/user/lang.html#lang
  - conjectures 10y ago
    Cheers - this is the #1 item on my excitement checklist.
- dragonwriter 10y ago
  > the cold startup time of the JVM itself (which is not relevant on a server in comparison to desktop Java apps). With microservices run as containers that are started on demand it matters; with other architectural choices it may matter less.
LeanderK 10y ago
I think for Server-Application Scala on the JVM will probably beat Scala-Native. The benefits of Scala-Native over Scala JVM are: - faster startup time - (drastically) lower memory footprint - fine hand-tuning of you application All these things are not super important in server-applications. For example Java trades memory for throughput (higher memory footprint, but also higher throughput. These usually go hand in hand.).
- therealmarv 10y ago
  But can be important if you only have a 512MB DigitalOcean instance or using small Linux containers/Docker.
  - LeanderK 10y ago
    I used the JVM on the 512MB DO instances and containers and they run fine. I think for containers there are other issues (most likely you are going in a micro service-direction where latency is eventually going to be important, so picking another JVM GC-algorithm might be suitable). There may be applications for which the 512 instances and JVM are not suitable, but you can most likely just upgrade the instance.
  - sreque 10y ago
    The JVM itself doesn't add a huge memory footprint. I remember Charles Nutter of JRuby fame calling it the "20-30 MB memory tax" (see http://blog.headius.com/2008/11/noise-cancelling.html http://blog.headius.com/2008/11/noise-cancelling.html). A lot of the extra memory usage of Java apps comes from sloppy programming and from depending on lots of heavyweight libraries and frameworks.
    - geodel 10y ago
      If you include idiomatic Java programing as part of sloppy programming I also agree with that. https://www.cs.virginia.edu/kim/publicity/pldi09tutorials/memory-efficient-java-tutorial.pdf https://www.cs.virginia.edu/kim/publicity/pldi09tutorials/me...
q3r3qr3q 10y ago
Yeah go for it. Nothing could go wrong: https://github.com/scala-native/scala-native/issues/543 https://github.com/scala-native/scala-native/issues/543

LeanderK 10y ago

Is Scala Native GCed?

merb 10y ago
yes (https://github.com/scala-native/scala-native/issues/128 https://github.com/scala-native/scala-native/issues/128)
zerr 10y ago
Should be, because Scala doesn't have strong vs weak reference distinction.

wst_ 10y ago

Somewhat relevant: https://blog.plan99.net/kotlin-native-310ffac94af2#.ijzik0jxx https://blog.plan99.net/kotlin-native-310ffac94af2#.ijzik0jx... Title says Kotlin, but it is about JVM languages going native, in general. Or should they?

sreque 10y ago
They should. The article basically throws in some FUD to convince people that they don't want what they think they want. Except, we do want it! The JVM ecosystem's general aversion to native code and native system integration is what gave C# the opportunity to take over as the closest thing we have to a WORA language. If I want to write code in a higher-level language than C++ that can run on any mobile device or desktop OS, I do it in C#, not Java. The Java ecosystem is playing huge catch up here, and I don't think it's a bad thing that people are exploring alternatives, whether that's Avian, the now defunct RoboVM, or LLVM-based backends.
- pjmlp 10y ago
  The JVM eco-system is full of options to compile Java into native code. The only thing is that most ignore there is a JVM world outside OpenJDK. And even those that know that world aren't willing to pay for commercial JVMs. Thus propagating the myth that Java doesn't support AOT. Which in a way is also understandable, given Sun's political decision that AOT compilation was tabu. Luckily Oracle isn't Sun and has heard the industry. So if all goes as planned, by Java 10, those that don't want to pay for AOT compilers, will have one in OpenJDK.

lukax 10y ago

The generated binary for simple Hello World is 3.45 MB which is quite a lot for printing one line of text but it can be compressed to 326K using UPX.

densh 10y ago
All of our binary size problems are mostly caused by a single well-known issue [1]. We hope to address that in one of the future releases. In the meanwhile, compressing binaries with upx is a temporary solution that can alleviate some of the pain. [1] https://github.com/scala-native/scala-native/issues/180 https://github.com/scala-native/scala-native/issues/180
cderwin 10y ago
Rust's is almost the exact same size (3.46 MB). It's probably just that the libraries that are statically linked by default (jemalloc, std, etc.).
- densh 10y ago
  That's correct. We statically link all of the transitive Scala dependencies.

jcstauffer 10y ago

Great! Any benchmarks on whether the Scala compiler runs faster when compiled to native?

densh 10y ago
We're looking into that, but nothing to report yet.

speg 10y ago

I can't get hello_world to work, something about a unresolved dependency: org.scala-native#sbt-Scalia-native;0.1.0: not found I'm not a regular Scala user.

densh 10y ago
Please drop by our gitter chatroom [1] and we'll try to debug the issue together. It's likely some sort of environment misconfiguration. [1] https://gitter.im/scala-native/scala-native https://gitter.im/scala-native/scala-native
manojlds 10y ago
I am a regular Scala user and I hate that message

mafribe 10y ago

Nice work, I hope this will eventually be a serious alternative to the JVM route. Quick question: does this compile down to DOT before going to LLVM? Or has DOT not yet arrived in Scala Native?

densh 10y ago
DOT [1] is a theoretical calculus that is used as a foundation for our latest front-end compiler called dotty [2]. Scala Native is mostly a middle-end technology that bridges front-end compiler and LLVM. We plan to support dotty as a front-end in the future. 0.1 release is based on older Scala 2.11 front-end. [1] http://www.scala-lang.org/blog/2016/02/03/essence-of-scala.html http://www.scala-lang.org/blog/2016/02/03/essence-of-scala.h... [2] https://github.com/lampepfl/dotty https://github.com/lampepfl/dotty
- bnjmn 10y ago
  Just what I hoped to hear, thanks!

andrewvijay 10y ago

scala no-exp here. How is this different from regular scala running on a jvm? There is no need of jvm for scala native?

cwyers 10y ago

It seems to me like Scala's biggest benefit and biggest downside are two sides of the same coin: easy interop with the JVM and Java code. Scala Native just seems like you're paying all the price of that for none of the benefit.

dillonb 10y ago
Well, this is just targeting a different VM, LLVM instead of the JVM. You get easy interop with all LLVM languages with this, including C.
- krona 10y ago
  Easy interop with C++ or Rust? I'll believe it when I see it.
  - dillonb 10y ago
    It looks like it can! http://www.scala-native.org/en/latest/user/interop.html http://www.scala-native.org/en/latest/user/interop.html >Scala Native provides an interop layer that makes it easy to interact with foreign native code. This includes C and other languages that can expose APIs via C ABI (e.g. C++, D, Rust etc.)
    - krona 10y ago
      > languages that can expose APIs via C ABI That's what I thought.. another C-based FFI. Better than nothing, I suppose.
    - username223 10y ago
      From that page, it looks like Scala-C interop is decent, but that's a far cry from C++/Rust interop. For C++ at least, you more or less need to write a pure-C wrapper API to call from Scala, since it doesn't handle C++ types.
      - steveklabnik 10y ago

Negative1 10y ago

Important bit: "The project has reached a point of feature completeness in terms of the coverage of the Scala language. We support the whole language including the more advanced features such as method dispatch via structural types and even macros." It must be frustrating to work on a project like this, see areas where the language can be improved, but only be able to do the work to make it purely compatible. Hopefully some good comes out in the form of some good SIPs.

acjohnson55 10y ago
Or inspiring! I guess it depends on your outlook.

gravypod 10y ago

Does anyone have any example binaries compiled with this? What are the sizes that you could expect? They say one of their targets is using this for command-line tools (I'm guessing for startup speed and needing to be small in memory footprint) but it's not of much value if an "echo" or "grep" implementation takes up 15 to 30MB on the drive.

dualogy 10y ago
> but it's not of much value if an "echo" or "grep" implementation takes up 15 to 30MB on the drive In this day & age? It might be of value if an echo or grep takes ~20MB and a full-blown HTTP REST server with DB access takes ~25-30MB. Why do I care again exactly whether there is a fixed portion of bootstrap/runtime core code in the binary, as long as I'm not writing echo or grep? (Any helloworld app not written in asm (or lib-less C) will consist largely of "non-helloworld code" in terms of "bytes occupied in the binary", right?) Dunno might be of concern eg with embedded/raspberry and such, but for code to be "moved from a jvm-kinda flair to a go-kinda flair" I'm not seeing the issue .. yet ;)
- usrusr 10y ago
  When you are writing that full-blown HTTP REST server with DB access you would probably be served better on the JVM. Scala native is just as garbage collected as Scala JVM and garbage collection is where the JVM shines brightly. In my eyes, the main value proposition of Scala native will be "you don't have to learn a new language if you need to write an echo or a grep", with "you don't need to learn a new language if you need to run some of your code on a platform that does not have a reasonably good JVM" a distant second. The former would suffer a lot from oversized binaries, the latter depending on the specifics of the target environment.
  - Recurecur 10y ago
    A major value proposition is also "I want to run my code with predictable latency". This opens Scala to embedded and realtime development. The LLVM AOT optimizations are also likely to be a lot stronger than JVM JIT optimizations - granted it's good to compile on the exact target architecture. Also granted that sometimes dynamic optimization helps a lot. Nice handle BTW. lol

amelius 10y ago

Does this provide a garbage collector?

cmsimike 10y ago
I asked a similar question on the reddit thread about this. I'm bummed that I didn't find this answer right away in this documentation. Probably the most interesting question for me
mark_l_watson 10y ago
Boom GC is a dependency
deleted 10y ago
[deleted]
ddispaltro 10y ago
Yeah it uses the Boehm garbage collector.

huula 10y ago

Question: what kind of frameworks can be practically migrated to Scala Native?

pale-hands 10y ago
I think the dependencies (like Netty for a web framework) would have to be migrated first. Scala native has a subset of java core libraries (from e.g. io, nio, util) rewritten in Scala, which could be a blocker if there's anything missing.
lmm 10y ago
In Scala it's very normal to write libraries or frameworks in "pure" Scala (i.e. not using reflection, annotations, proxies or anything like that), and all of those should be fine to cross-build for Scala Native (assuming their upstream dependencies do first). The state of libraries for Scala.js should be a good indication - anything that's cross-compatible with that is likely to work just as well for Scala Native.

pale-hands 10y ago

Will macros work with Scala Native, as they do with ScalaJs? (I believe that compile-time metaprogramming is the way forward, especially if the target doesn't support reflection or dynamic code loading).

ddispaltro 10y ago
"We support the whole language including the more advanced features such as method dispatch via structural types and even macros."
- pale-hands 10y ago
  D'oh, thanks! Can't believe I missed that.
coldsauce 10y ago
If i recall correctly, they are planning to (or have already) remove macros in a newer version of Scala.
- pale-hands 10y ago
  Yeah, the current macro system will be replaced with scala-meta, which is a different macro system, but it hasn't happened yet. Scala-meta doesn't have def macros yet, so it's not ready.

evdev 10y ago

As a Scala guy on a Scala team, I'd think this would be most immediately useful on smaller fill-the-gaps sub-projects where we have to integrate with native code.

manojlds 10y ago
Looks like a good fit for the CLI we have been planning for our API bases on a Scala backend.

auggierose 10y ago

It seems like that would open up a nice way of using Scala on iOS also.

0xFFC 10y ago

so dream comes true! P.S. I think this is related to rust, in a sense before Rust there was no serious competitor to C/C++, but after seeing what Rust doing to C/C++ I think there will be more native language to compete with in low level area.

usrusr 10y ago
Between the "scripty" GC agility of Golang and all the impressive low level benefits that Rust is enjoying from folding memory management into the type system, I suspect that Scala native will be a difficult sell. On the other hand, the new targets (js and native) might make Scala more interesting to Java pragmatists who don't care much about going functional but would not mind writing Java in a more streamlined syntax.
- Recurecur 10y ago
  Golang is pretty deficient as languages go, really. I find Scala to be far more appealing.

brangalinafoeva 10y ago

How do the compilation times compare to targeting the JVM?

densh 10y ago
There is roughly 1-2s (after sbt warm-up) penalty to perform compilation to native on iMac (Retina 5K, 27-inch, Late 2015) with 4 GHz Intel Core i7. The linker is highly parallel so you can throw more hardware at it as your project grows bigger. Barebones project includes nearly 1000 classes and 2000 methods in transitive dependencies that are being compiled due to closed-world nature of the toolchain.

mark242 10y ago

This will be huge for getting Scala running on AWS Lambda. The cold-start times for JVM apps is just ridiculous and makes Lambda/API gateway essentially unusable for anything written on the JVM.

sjrd 10y ago
Actually, there are people using Scala.js to run Scala on AWS lambda, precisely for that reason. * https://github.com/tptodorov/aws-lambda-scalajs https://github.com/tptodorov/aws-lambda-scalajs * http://underscore.io/blog/posts/2016/03/21/serverless-scale-summit.html http://underscore.io/blog/posts/2016/03/21/serverless-scale-...
- acjohnson55 10y ago
  ^ From the creator of Scala.js :) I am excited that now there will be an option for getting both quick startup and fastest execution time!
- danaliv 10y ago
  Sure, but it'd be nice not to have everything in the known universe depend on JavaScript.
  - sjrd 10y ago
    Absolutely! I'm just pointing out that this need is so strong that people have reached for the slow throughput of Scala.js just to get its fast startup time. If we can have both, that's even better!
    - edko 10y ago
      No intention of arguing with Scala.js' creator :-) But, I do not think it has a "slow throughput" at all. In my experience, it's had pretty good performance.
      - sjrd 10y ago

zabed143 10y ago

Clipping Service India is an online outsourcing company. We provide service in the field of photo editing and graphic design, specialist in... Clipping Service India is an online offshore graphic design studio providing clipping path, Photoshop masking, image manipulation, image retouching, Color Correction, drop shadow, Reflection shadow, Multiple clipping path, raster to vector, Website image optimization, all types of photo editing service... Clipping Service India is established in order to provide image editing services through internet and this internet makes our life very easy and comfortable to communicate around the world's people within in a click on mouse. http://www.clippingserviceindia.com/index.php http://www.clippingserviceindia.com/index.php

rekado 10y ago

Unfortunately, this requires an existing Scala compiler to build, so it won't be useful as a bootstrap compiler for Scala on the JVM. Does anyone here know of an alternative implementation of Scala that could be used to build the libraries and tools of the reference implementation from source? It is a problem that many compilers cannot be bootstrapped from source without a trusted binary of a previous release.

ajross 10y ago
It's a universal truism that all compilers cannot be bootstrapped from source without a trusted binary. It's true that in the world of standardized languages (which mostly means "C" in practice) there are compilers (mostly just gcc and clang) that can bootstrap themselves with a trusted binary of a previous release of some other compiler. Is that such a big deal?
- rekado 10y ago
  I wrote "a trusted binary of a previous release" not just "a trusted binary". There is obviously a difference between having a small set of trusted binaries to bootstrap and having a bootstrap binary for every language or build system. There are several compiler implementations that enable bootstrapping from alternative implementations, which shifts the problem to a simpler language, which may already have a bootstrap path. See also http://bootstrappable.org/best-practises.html http://bootstrappable.org/best-practises.html
platz 10y ago
how big of a problem is it, really? is it a theoretical problem or a practical problem for industry users?
- rekado 10y ago
  It is a practical problem for people who want to have a correspondence between source and binary. Some users would like to have to rely on as few opaque binaries as possible. There are efforts underway to build a minimal C compiler in a subset of Scheme that can be implemented on bare metal. The goal is to reduce the set of binaries that needs to be trusted or audited manually.

rainhacker 10y ago

> This opens the door for Scala to be used in environments where full-blown virtual machine is usually an overkill Not sure if I get this, don't Java VMs support this use case (J2ME) ?

crudbug 10y ago

Will language support destructors for manual memory management ?

densh 10y ago
Please file a feature request on http://github.com/scala-native/scala-native http://github.com/scala-native/scala-native, we can discuss the details over there.

tejasmanohar 10y ago

Does this mean the future of Scala is off the JVM? I ask because the post calls the JVM impl. a "reference implementation".

sjrd 10y ago
IMO the future of the JVM is highly multi-platform: JVM, native and JS.
emodendroket 10y ago
Does that not mean that it's the canonical one which other implementations should be measured against? That would seem to imply just the opposite.

squar1sm 10y ago

I wonder if Akka will be a part of scala native? It's sort of considered stdlib? When our team spiked on Akka, we liked it and got our near-reality proof of concept to work. It'd be awesome to have such a high level library like Akka compile to a binary.

scotchmi_st 10y ago

I would love for there to be a similarly thorough project with Clojure. It really bothers me there's no good native compiler. Apart from anything else, it means that Clojure lives and dies by the languages it compiles to, and while Java is used everywhere still, it probably isn't the thing the kids are learning these days. Besides, without going into any further rational arguments for why using the JVM (or another VM) isn't always great, something about it feels a bit icky to me. On an aesthetic level.

pjmlp 10y ago
> isn't the thing the kids are learning these days Yep, they learn Ruby and then they need to go JRuby when performance becomes relevant. :)
- jwdunne 10y ago
  Or they tend to just Go. Interestingly, I've read a lot of Go users come from the Ruby and Python communities.
  - pryelluw 10y ago
    Im one of those. Anytime Python / Django doesnt cut it I use Go. Works very well.
    - jwdunne 10y ago
      Have you got a few examples where you had to replace the former for the latter? Really interesting.
      - pryelluw 10y ago
        Sure. Last one was a static asset server that had to perform or the users would notice a delay. Built one in Go in about a week and put it behind nginx on a vps. It was faster than required, stand alone, and used very little memory. Its still running to this day.

c-smile 10y ago

I've created once custom barebone JavaVM with binary size of around 100k. It was made as an executable jsmile.exe that was capable to read bytecodes cat'ed to the executable itself : http://www.terrainformatica.com/org/j-smile/index.htm http://www.terrainformatica.com/org/j-smile/index.htm The goal was to create JVM suitable for standalone GUI applications. Project was abandoned when Sun/MS Java wars started in favor of the Sciter (https://sciter.com https://sciter.com). As of NativeScala ... I think that approach (binary with nano JVM + attached class files) may work better and with less effort. Scala needs JVM infrastructure, GC, etc. as far as I understand.

eli_gottlieb 10y ago

This is amazing. Scala is one of my favorite languages to work with, and getting it native-code support will finally help make it fast enough to justify using it everywhere.

crustycoder 10y ago

The lack of thread support is a deal breaker for me though.

mihaela 10y ago

Most things called native are not.

kelvich 10y ago

Nice job, thanks! Seems that right now you had implemented some wrappers for libc and some java classes ported to scala. What plans do you have to further evolve API? Will you focus on reimplementing java.* or create your own set of classes?

kentosi 10y ago

This is extremely exciting. I can't wait to try this out. On the other hand, I wonder why such an effort was never carried out with Java itself? Or maybe it was but just never took off?

wtetzner 10y ago
There was RoboVM[1], but it looks like it shut down. GCJ [2] was GCC's Java implementation, which was natively compiled, but I think has been removed from GCC. I'm sure there are other projects. [1] https://robovm.com/ https://robovm.com/ [2] https://en.wikipedia.org/wiki/GNU_Compiler_for_Java https://en.wikipedia.org/wiki/GNU_Compiler_for_Java
- invalidname 10y ago
  Codename One predated RoboVM, was always larger and is still open source and well supported: https://www.codenameone.com/ https://www.codenameone.com/
david-given 10y ago
Would this be a good time to mention that a few years ago I wrote an experimental Java AOT compiler which worked by converting bytecode into C++? http://cowlark.com/cowjac http://cowlark.com/cowjac I used Apache Harmony as the standard library, and binaries were about 2MB, which back in 2012 I considered unacceptably large; benchmarks were adequate only. I never got round to implementing a garbage collector but the hooks were in place for a simple mark/sweep stop-the-world collector. The hard part would be to actually make all threads of the program stop so they could be collected; I had a plan, but no implementation. Interested parties may also be interested in my experimental Java JIT which worked by converting bytecode into Lua and then running it with LuaJIT: http://cowlark.com/luje http://cowlark.com/luje It actually beat OpenJDK Server for some unrepresentative microbenchmarks. It's very unfinished, and generates fairly pathological Lua code which LuaJIT has trouble with, but the biggest problem is that Lua and LuaJIT can't do shared-memory threading, so most Java code won't work on it. (JVMs are surprisingly easy to write. If you're interested in interpreters or code generation, I encourage you to try it.) (Man, I completely forgot I wrote these until now!)

webserg 10y ago

good idea!

enjoiful 10y ago

At first glance of this article's title, I thought it would be a terrible idea to use Scala to write native mobile applications. Imagine using Scala.JS to write a NativeScript/React Native app. shutters

sjrd 10y ago
> Imagine using Scala.JS to write a NativeScript/React Native app. People do [1], and are very happy with it. [1] https://github.com/chandu0101/sri https://github.com/chandu0101/sri

NickHoff 10y ago

Will this mean that we can get rid of type erasure when running native code?

tormeh 10y ago
Why is type erasure necessary anyway? Surely this can be worked around somehow. You could manually do what you do with array length in C - pass the information along manually. Why can't the compiler do this in a transparent manner for us? Is it speed?
lmm 10y ago
It's more likely to make type erasure more extensive. The main effect of type erasure is to stop people doing naughty things in generic methods. When one doesn't have to support JVM reflection, erasure can be pushed further.

twic 10y ago

This is certainly an impressive piece of work. However, i think it's worth paying attention to the limitations, and the use cases they imply; overall, this looks less like "compile your existing Scala app to native code!" and more like "use Scala to interface with existing native libraries!". On the other hand, it's also worth bearing in mind that this is version 0.1.0; over time, some of these limitations will lift. What i don't know is whether Scala Native will develop into a complete version of Scala which compiles to native code, or evolve into a variant of Scala more tightly adapted to a niche of talking to native libraries. Anyway ... (1) No threading [1]: Scala Native doesn’t yet provide libraries for parallel multi-threaded programming and assumes single-threaded execution by default. It’s possible to use C libraries to get access to multi-threading and synchronization primitives but this is not officially supported at the moment. So forget about using Akka for now. (2) NullPointerExceptions are replaced with segfaults (hopefully) [1]: A number of error conditions which are well-defined on JVM are undefined behavior: Dereferencing null. Division by zero. Stack overflows. Those typically crash application with a segfault on the supported architectures. That's not so bad; where Java apps might let nulls flow around and rely on catching NullPointerExceptions to recover from them, Scala apps are much more likely to use Optional consistently. (3) If you do want to talk to a native library, and you need to allocate memory to do it, you're on your own [2]: Unlike standard Scala objects that are managed automatically by the underlying runtime system, one has to manage native pointers manually. Scala Native provides a built-in way to perform stack allocations of unmanaged memory using native.stackalloc function: [...] When using stack allocated memory one has to be careful not to capture this memory beyond the lifetime of the method. Dereferencing stack allocated memory after the method’s execution has completed is undefined behaviour. Scala Native’s library contains a bindings for a subset of the standard libc functionality. This includes the trio of malloc, realloc and free functions Java's traditional JNI is a verbose, slow, pain in the stdout, but it was designed pretty carefully to avoid problems like this. (a) Intermission! Check out how they do type-level numbers [1]: Natural numbers are types that are composed of base naturals Nat._0, ... Nat._9 and an additional Nat.Digit constructor. That's a new one on me! (4) Incomplete JDK libraries [3]: Scala Native supports a subset of the JDK core libraries reimplemented in Scala. Here is the list of currently available classes: [...] This is an ongoing effort, some of the classes listed here might be partially implemented. The list has most of the fundamental stuff - a good chunk of java.io and NIO, the collections, java.lang, atomics. But no java.text, java.net, concurrency, regexp, date and time, JDBC, reflection, XML, etc. They don't mention how much of the Scala libraries they support. I would imagine that they can build anything that's in pure Scala and depends only on JDK classes in that list, so you'll get the core language stuff and the collections. Not sure. [1] http://www.scala-native.org/en/latest/user/lang.html http://www.scala-native.org/en/latest/user/lang.html [2] http://www.scala-native.org/en/latest/user/interop.html http://www.scala-native.org/en/latest/user/interop.html [3] http://www.scala-native.org/en/latest/lib/javalib.html http://www.scala-native.org/en/latest/lib/javalib.html

lacampbell 10y ago

Scala is a language I desperately wanted to like - high level, statically typed pure OO language. But in practice I found it almost unusable. The type signatures were unreadable and I distinctly recall writing a 100 line or so program where the type declarations crashed the compiler. And the tools themselves were huge memory hogs - sbt was a particularly bad offender (though otherwise quite pleasant). I also did not get on well with the community, which seemed to have a lot of people with the attitude - "they won't let me use haskell at work so I'll make do with this shit". They didn't seem to understand or be interested in OO at all, and were very fanatical about driving application logic with types, purity, and the like. Regardless, a native variant would be something well worth investigating if it ever reaches "production ready".

eklavya 10y ago
It looks like you dislike the haskell like usage of Scala, I wonder how could you then produce types so complicated/weird that you crashed the compiler in 100 lines (In 4 years I have seen it crash only once in an unreleased version of pickling a long time ago).
- lacampbell 10y ago
  I never claimed my criticism was fair or consistent :P I think I was more interested in Haskell-like type driven programming at the time though. I honestly can't remember what it was. And it's not so much that I hate type driven programming or purity. I just view them as tools in my toolbox, not as methodologies or worse - religions. I still think objects are an amazing concept if you look at them through the smalltalk lens. Or if you take a moment to step back and realise that an anonymous function is very similar to an object with an "apply" method, or that a constructor for an immutable object is like a functor that partially applies many related functions at once.
  - eklavya 10y ago
    In my opinion you are mostly describing Scala. Mutability and impurity is trivially easy to do in Scala. You may see that as a positive or a negative. Local mutability is definitely easier. Complexity of Scala comes from mixing OO and FP worlds and I think it's an amazing engineering feat. It will undoubtedly look ugly after you do Haskell though.

deleted 10y ago

[deleted]

deleted 10y ago

[deleted]

anta40 10y ago

Any Windows pre-built binary to try? I imagine building this stuff on Windows will be challenging :/