5 ms·

FastR: An implementation of the R language in Java [pdf]

85 points by susi22 13y ago

th0br0 13y ago
FWIW, there's also Renjin that does this: https://github.com/bedatadriven/renjin https://github.com/bedatadriven/renjin
Demiurge 13y ago
Are there any tools to convert R to JS or Python, or... any other common language that doesn't require a 60mb runtime distribution?
- CJefferson 13y ago
  I have done some work converting R to C++ by hand. The problem, as this presentation discusses, is that a 95% automated conversion is easy, but that last 5% often involves diving deep into the weirdness of the R interpreter.
  - Demiurge 13y ago
    I am using RCPP for interfacing, didn't see anything automatic for conversion though, it's just an interface.
- chubot 13y ago
  That's a really bizarre reason not to use R. Python's distribution is about 30MB if I recall. It doesn't really make much sense to convert R to JS or Python, since the semantics are so different. If you don't want to use R, Pandas in Python provides very powerful data frames (which are likely faster for many cases). However, it depends on NumPy, matplotlib, and a few other libraries, which probably total more than 60 MB.
  - neeee 13y ago
    Python 3's installed size is 90MB here, 2 is 60MB.
    - Demiurge 13y ago
      Good thing is that python is pretty ubiquitous, as is virtual env :)
  - Demiurge 13y ago
    I already use python, so having it in same codebase would be perfect. However, I have some code in R that is pretty important, so I have to include that as well with an application (model).
- long 13y ago
  A guy named Jony Hudson has gotten R to kind of compile to JS using emscripten. See http://r.789695.n4.nabble.com/R-in-the-browser-td4667985.html http://r.789695.n4.nabble.com/R-in-the-browser-td4667985.htm... But the JS blob ends up being like ~15mb!
brendano 13y ago
The programming language analysis is pretty interesting, but you have to ask, what's the point of a brand-new Java implementation? R isn't just a programming language, but it's a software framework/ecosystem. They mentioned this in the slides, but it's problematic because R crucially relies on C and Fortran interaction (which I thought the JVM can't do efficiently, since it doesn't like giving C/Fortran raw memory access to its internals). Decades of work has gone into highly optimized Fortran linear algebra libraries, for example -- which R and all the other high-level numerical languages (NumPy/SciPy, Matlab, Julia) use. And many of the CRAN packages (the availability of which are a major reason anyone uses R in the first place) are partly or mostly C/Fortran code. There are many other R implementation efforts going on right now -- Radford Neal lists a few (as well as his own) here: http://radfordneal.wordpress.com/2013/07/24/deferred-evaluation-in-renjin-riposte-and-pqr/ http://radfordneal.wordpress.com/2013/07/24/deferred-evaluat... The presentation focuses on the R programming language, which they nicely show has all sorts of misfeatures that impede rapid execution. If you're going to not try to have compatibility with R and CRAN, you might as well start from scratch with design and performance in mind, as in Julia: http://julialang.org/ http://julialang.org/
- bedatadriven 13y ago
  I think compatibility with GNU R and CRAN is a pretty reasonable goal - even if it's not the focus of this particular Oracle research project. We've been working hard and now systematically to get Renjin (also R on the JVM) to run CRAN packages: http://packages.renjin.org http://packages.renjin.org. Renjin also compiles C and Fortran code to JVM bytecode, though there is still some work to do there as well. Regarding the hand-tuned matrix math libraries, there's nothing to stop you from using them with Renjin - you can drop in MKL or Atlas as desired, or fall back to pure-Java versions in a pinch.
- pjmlp 13y ago
  Julia is quite cool, actually. As for Java, Oracle together with AMD, are in the process of making the GPU trasparent to Java developers as part of the Sumatra project. So this is one are where R could benefit of running on Oracle's JVM. It remains to be seen if other Java vendors would adopt such feature.
susi22 13y ago
Video presentation for the slides: http://download.oracle.com/technetwork/java/javase/community/H264_1280x720/20144523.mov http://download.oracle.com/technetwork/java/javase/community...
knodi 13y ago
more java -_- I need less java in my life, not more.
- AsmMAn 13y ago
  Which platforms are you targeting?
ubasu 13y ago
Given the problems with Java's floating-point implementation [1], would this be reliable for statistical analysis? [1] https://news.ycombinator.com/item?id=6585828 https://news.ycombinator.com/item?id=6585828
- aardvark179 13y ago
  The semantics of Java need not restrict what other languages on the JVM do, though it may make the generated byte code a little larger from the inclusion of f2d and d2f instructions.
- pron 13y ago
  1. It is more a philosophical disagreement than a problem, and whatever the semantics you want, the JVM does not limit you. The disagreement revolves around the Java compiler and language semantics, not the JVM. 2. The math in FastR, if I understand the presentation correctly, is performed by FORTRAN libraries anyway. Using battle tested FORTRAN libraries for matrix computations is common practice in C, Java, Julia, Matlab and most other environments. They basically all share the same underlying matrix math code.
  - StefanKarpinski 13y ago
    I do think that Kahan's objections to Java's floating-point support were more than just philosophical, although it's unclear to me at this point how many of them still apply. The biggest issue that seems to still remain is that you can't change rounding modes or check flags, both of which are essential for verifying numerical stability and correctness. Far worse than any floating-point issues on the JVM is that your indices and integers are 32-bit, so you're limited to 2GB arrays before you have to take bizarre measures to access larger amounts of data. Although it is standard in high-level systems to call out to a BLAS library [1], for some inexplicable reason it seems that both R and NumPy use the reference BLAS by default, which is quite slow – around 4x slower than better BLASes. Matlab ships with Intel's proprietary MKL, which includes a very fast BLAS implementation, while Julia ships with OpenBLAS, which is a similarly fast open source BLAS implementation derived from the legendary GotoBLAS [2]. Since all BLAS implementations share a common Fortran ABI, it's easy to swap them out, but it's not quite true that all of these systems are using the exact same Fortran code. [1] https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprogra... [2] https://en.wikipedia.org/wiki/GotoBLAS https://en.wikipedia.org/wiki/GotoBLAS
noahmarc 13y ago
The Relite mention at the end looks very worthwhile. We were just about to begin a large rewrite of analysis code from R to CUDA; Relite has the potential to save the effort of rewriting our existing code.
- tiarkrompf 13y ago
  Relite author here. I'd love to learn more about your use cases, feel free to get in touch!
  - noahmarc 13y ago
    Definitely, thank you for the offer. I'm running through the install now but will send you an email once I've run a couple tests. I see your contact info is listed on your home page.
moondowner 13y ago
Worth noting is that FastR is a JVM implementation of R that uses Truffle and Graal. http://openjdk.java.net/projects/graal/ http://openjdk.java.net/projects/graal/ https://wiki.openjdk.java.net/display/Graal/Publications+and+Presentations https://wiki.openjdk.java.net/display/Graal/Publications+and...

85 points by susi22 13y ago

FWIW, there's also Renjin that does this: https://github.com/bedatadriven/renjin https://github.com/bedatadriven/renjin

Are there any tools to convert R to JS or Python, or... any other common language that doesn't require a 60mb runtime distribution?

CJefferson 13y ago
I have done some work converting R to C++ by hand. The problem, as this presentation discusses, is that a 95% automated conversion is easy, but that last 5% often involves diving deep into the weirdness of the R interpreter.
- Demiurge 13y ago
  I am using RCPP for interfacing, didn't see anything automatic for conversion though, it's just an interface.
chubot 13y ago
That's a really bizarre reason not to use R. Python's distribution is about 30MB if I recall. It doesn't really make much sense to convert R to JS or Python, since the semantics are so different. If you don't want to use R, Pandas in Python provides very powerful data frames (which are likely faster for many cases). However, it depends on NumPy, matplotlib, and a few other libraries, which probably total more than 60 MB.
- neeee 13y ago
  Python 3's installed size is 90MB here, 2 is 60MB.
  - Demiurge 13y ago
    Good thing is that python is pretty ubiquitous, as is virtual env :)
- Demiurge 13y ago
  I already use python, so having it in same codebase would be perfect. However, I have some code in R that is pretty important, so I have to include that as well with an application (model).
long 13y ago
A guy named Jony Hudson has gotten R to kind of compile to JS using emscripten. See http://r.789695.n4.nabble.com/R-in-the-browser-td4667985.html http://r.789695.n4.nabble.com/R-in-the-browser-td4667985.htm... But the JS blob ends up being like ~15mb!

brendano 13y ago

The programming language analysis is pretty interesting, but you have to ask, what's the point of a brand-new Java implementation? R isn't just a programming language, but it's a software framework/ecosystem. They mentioned this in the slides, but it's problematic because R crucially relies on C and Fortran interaction (which I thought the JVM can't do efficiently, since it doesn't like giving C/Fortran raw memory access to its internals). Decades of work has gone into highly optimized Fortran linear algebra libraries, for example -- which R and all the other high-level numerical languages (NumPy/SciPy, Matlab, Julia) use. And many of the CRAN packages (the availability of which are a major reason anyone uses R in the first place) are partly or mostly C/Fortran code. There are many other R implementation efforts going on right now -- Radford Neal lists a few (as well as his own) here: http://radfordneal.wordpress.com/2013/07/24/deferred-evaluation-in-renjin-riposte-and-pqr/ http://radfordneal.wordpress.com/2013/07/24/deferred-evaluat... The presentation focuses on the R programming language, which they nicely show has all sorts of misfeatures that impede rapid execution. If you're going to not try to have compatibility with R and CRAN, you might as well start from scratch with design and performance in mind, as in Julia: http://julialang.org/ http://julialang.org/

bedatadriven 13y ago
I think compatibility with GNU R and CRAN is a pretty reasonable goal - even if it's not the focus of this particular Oracle research project. We've been working hard and now systematically to get Renjin (also R on the JVM) to run CRAN packages: http://packages.renjin.org http://packages.renjin.org. Renjin also compiles C and Fortran code to JVM bytecode, though there is still some work to do there as well. Regarding the hand-tuned matrix math libraries, there's nothing to stop you from using them with Renjin - you can drop in MKL or Atlas as desired, or fall back to pure-Java versions in a pinch.
pjmlp 13y ago
Julia is quite cool, actually. As for Java, Oracle together with AMD, are in the process of making the GPU trasparent to Java developers as part of the Sumatra project. So this is one are where R could benefit of running on Oracle's JVM. It remains to be seen if other Java vendors would adopt such feature.

susi22 13y ago

Video presentation for the slides: http://download.oracle.com/technetwork/java/javase/community/H264_1280x720/20144523.mov http://download.oracle.com/technetwork/java/javase/community...

knodi 13y ago

more java -_- I need less java in my life, not more.

AsmMAn 13y ago
Which platforms are you targeting?

ubasu 13y ago

Given the problems with Java's floating-point implementation [1], would this be reliable for statistical analysis? [1] https://news.ycombinator.com/item?id=6585828 https://news.ycombinator.com/item?id=6585828

aardvark179 13y ago
The semantics of Java need not restrict what other languages on the JVM do, though it may make the generated byte code a little larger from the inclusion of f2d and d2f instructions.
pron 13y ago
1. It is more a philosophical disagreement than a problem, and whatever the semantics you want, the JVM does not limit you. The disagreement revolves around the Java compiler and language semantics, not the JVM. 2. The math in FastR, if I understand the presentation correctly, is performed by FORTRAN libraries anyway. Using battle tested FORTRAN libraries for matrix computations is common practice in C, Java, Julia, Matlab and most other environments. They basically all share the same underlying matrix math code.
- StefanKarpinski 13y ago
  I do think that Kahan's objections to Java's floating-point support were more than just philosophical, although it's unclear to me at this point how many of them still apply. The biggest issue that seems to still remain is that you can't change rounding modes or check flags, both of which are essential for verifying numerical stability and correctness. Far worse than any floating-point issues on the JVM is that your indices and integers are 32-bit, so you're limited to 2GB arrays before you have to take bizarre measures to access larger amounts of data. Although it is standard in high-level systems to call out to a BLAS library [1], for some inexplicable reason it seems that both R and NumPy use the reference BLAS by default, which is quite slow – around 4x slower than better BLASes. Matlab ships with Intel's proprietary MKL, which includes a very fast BLAS implementation, while Julia ships with OpenBLAS, which is a similarly fast open source BLAS implementation derived from the legendary GotoBLAS [2]. Since all BLAS implementations share a common Fortran ABI, it's easy to swap them out, but it's not quite true that all of these systems are using the exact same Fortran code. [1] https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprogra... [2] https://en.wikipedia.org/wiki/GotoBLAS https://en.wikipedia.org/wiki/GotoBLAS

noahmarc 13y ago

The Relite mention at the end looks very worthwhile. We were just about to begin a large rewrite of analysis code from R to CUDA; Relite has the potential to save the effort of rewriting our existing code.

tiarkrompf 13y ago
Relite author here. I'd love to learn more about your use cases, feel free to get in touch!
- noahmarc 13y ago
  Definitely, thank you for the offer. I'm running through the install now but will send you an email once I've run a couple tests. I see your contact info is listed on your home page.

moondowner 13y ago

Worth noting is that FastR is a JVM implementation of R that uses Truffle and Graal. http://openjdk.java.net/projects/graal/ http://openjdk.java.net/projects/graal/ https://wiki.openjdk.java.net/display/Graal/Publications+and+Presentations https://wiki.openjdk.java.net/display/Graal/Publications+and...