5 ms·

Renjin: a JVM-based interpreter for the R language

118 points by phillc73 9y ago

stewbrew 9y ago
IMHO the crucial question is whether users can use libraries that load native libraries and/or make use of rcpp. The docs make me think it's not possible.
- phillc73 9y ago
  Regarding Rcpp, their package library says "A Renjin-specific version of this package is available."[1] Looks like Renjin can also be used from GNU R as an R package[2]. I'd assume any installed R package should also work in conjunction with it. Admittedly, I haven't tried Renjin, but only just found it and thought it was quite interesting. [1] http://packages.renjin.org/package/org.renjin.cran/Rcpp http://packages.renjin.org/package/org.renjin.cran/Rcpp [2] http://docs.renjin.org/en/latest/package/index.html#using-renjin-as-an-r-package http://docs.renjin.org/en/latest/package/index.html#using-re...
  - michaelmior 9y ago
    Even just using Renjin to be able to access Java code within R seems super useful.
    - kprybol 9y ago
      Wasn’t that already possible via the rJava library?
      - michaelmior 9y ago
        True. Although Renjin seems like it provides a bit of a nicer interface in some cases. (Disclaimer: not an R developer)
- sorokod 9y ago
  From the documentation [1]: "As a service, BeDataDriven provides a repository with all CRAN (the Comprehensive R Archive Network) and BioConductor packages at http://packages.renjin.org http://packages.renjin.org. The packages in this repository are built and packaged for use with Renjin. Not all packages can be built for Renjin so please consult the repository to see if your favorite package is available for Renjin." [1] http://docs.renjin.org/en/latest/interactive/index.html#prerequisites http://docs.renjin.org/en/latest/interactive/index.html#prer...
  - Annatar 9y ago
    Yes. Renjin isn’t even a complete implementation of R and I don’t believe that a Java JIT will ever beat Fortran in performance. I’d sure like to see some evidence of JIT R beating the Fortran implementation. And then there is the memory inefficiency of Java. So again I ask: why?
    - mwexler 9y ago
      Because there is value in having a completely jvm stack in some places. Sometimes performance takes a back seat to consistency. It may not be right, but that's the way it is in some orgs.
    - acaloiar 9y ago
      When I was implementing Renjin's PRNG based on GNR R's, I read a fair amount of GNU R code. I think it goes without saying that C and Fortran perform well. However, I do believe that R's flexibility as a language comes at a performance cost. The implementation of that flexibility is not always done in an efficient manner; and as a result performance suffers. I.e. R is not particular performant in certain circumstances despite being largely implemented in C and Fortran. I only have anecdote to offer in this regard, but it is worth considering that GNU R's good performance is not a foregone conclusion.
    - phillc73 9y ago
      Hadley's book, "Advanced R" has a specific performance related chapter[1]. As well as techniques to boost performance within R, there's a section on "Alternative R implementations," which says in part, "There are some exciting new implementations of R. While they all try to stick as closely as possible to the existing language definition, they improve speed by using ideas from modern interpreter design." Renjin is listed in this section. While I don't have direct experience of Renjin, or any other alternative R interpreters, I'm inclined to believe Hadley has and if he believes they improve speed, I'll take him at his word. [1] http://adv-r.had.co.nz/Performance.html http://adv-r.had.co.nz/Performance.html
eggy 9y ago
R's wealth of libraries and syntax was always appealing to me, so maybe now I will look at how compatible and faster Renjin might be.
haifeng 9y ago
There is a java/scala project providing R like statistical computation: http://haifengl.github.io/smile/linear-algebra.html http://haifengl.github.io/smile/linear-algebra.html
Annatar 9y ago
I build and maintain the R stack in my organization. A good portion of the R core is written in Fortran for maximum performance and efficiency; the rest is in C and C++. Why would I want an interpreted, just-in-time, garbage collected version of R in a JVM when I can run at full speed on the real thing?
- kitd 9y ago
  > interpreted, just-in-time, For clarification, the JVM JIT-compiles the most frequently used blocks of bytecode to machine code, so the bulk of the heavy lifting is not interpreted. The benefit of doing it this way is that compilation can be optimised based on the runtime-characteristics of the code, not compiler judgement. Cases of JVM code running faster than equivalent C/C++ code are not uncommon. Aside from that, access to Java libraries (eg for integration) may be important to some shops.
- Insanity 9y ago
  I suppose their reasons are here: http://www.renjin.org/about.html http://www.renjin.org/about.html I've only used R for a short time, so I can't comment on it myself.
  - Annatar 9y ago
    “R has been traditionally limited by the need to fit data sets into memory, and working with even modest sets of data can quickly exhaust memory due to historical limitations in GNU R interpreter’s implementation.” I see that the propaganda machine never rests, no matter which language. Either that, or they haven’t heard of Vertica’s R, which is excellent and counters these claims. The rest with the “fragmentation of R” is a poor marketing tack: people who run R are perfectly capable of thinking with their own brains, thank you very much; Renjin has no place patronizing. We’re not sheep.
    - kirillseva 9y ago
      that's a fairly toxic and silly attitude towards a free software project. Don't like it - don't use it. I personally think it's great that there are people who are working on making R faster/better/popular, even though I may disagree with the methods they've chosen to achieve those goals.
ianopolous 9y ago
FastR [1], an R implementation for the JVM using Truffle aims to be both fast and compatible, including C and fortran calls from R using Truffle's polyglot framework. The thing that makes this even more awesome is inter language optimization - inline your C code into your R code at runtime, which can result in zero overhead inter language calls. [1] https://github.com/graalvm/fastr https://github.com/graalvm/fastr
- peatmoss 9y ago
  One of the interesting things I note from the readme, is that they have an implementation of Grid graphics. Grid is pretty awesome for lower level graphics, and is part of the reason ggplot is able to be as good as it is. Looking forward to poking around at their implementation. If it’s a JVM friendly implementation, that could open up some nicer visualization options from other languges like clojure, scala, etc.
  - mj_kallen 9y ago
    Indeed, they presented this at the last RIOT workshop in July [1] and it's a great way to support graphics without dealing with the grDevices package from GNU R. [1] https://youtu.be/otXTGBTb-3w?t=985 https://youtu.be/otXTGBTb-3w?t=985
    - peatmoss 9y ago
      Oh cheers! Looking forward to watching that.
peatmoss 9y ago
It's been a while since I've looked at Renjin. Last I looked, it was a worthy replacement for tasks that you'd do in base R, but the library story was incomplete. Nowadays, my R usage has gone full tidyverse. Anyone know if dplyr and the gang work here? I know dplyr has a lot of C++ code, and don't know how well the "Renjin-specific" Rcpp interop story is.
- baldfat 9y ago
  First thing I did was look at packages and see if tidyverse, purrr, lubridate, stringr and feather. They are all there. http://packages.renjin.org/packages/search?q=tidyverse http://packages.renjin.org/packages/search?q=tidyverse
  - phillc73 9y ago
    Yes, but some of them aren't available yet, specifically dplyr: http://packages.renjin.org/package/org.renjin.cran/tidyverse http://packages.renjin.org/package/org.renjin.cran/tidyverse
deleted 9y ago
[deleted]
Myrmornis 9y ago
How does Renjin run the BLAS and LAPACK Fortean code, and the C code in R?
- psk0 9y ago
  We transpile all C, C++, and Fortran code directly to Java-bytecode using our open source tool called gcc-bridge (https://github.com/bedatadriven/renjin/tree/master/tools/gcc-bridge https://github.com/bedatadriven/renjin/tree/master/tools/gcc...). This is included as part of Renjin, but you can use it independently as part of your project as well.
  - yazr 9y ago
    This is nice. Does anyone know of a similar tool for c# ?
tranwen 9y ago
Tried a few basic things, some of them did not work: 1. x = rnorm(1000) 2. plot(density(x)) --> not work 3. stem(x) --> not work 4. summary(x) --> works For R data handling, I always use data.table for its efficiency and power. 1. library(data.table) 2. x = data.table(x=rnorm(1000)) --> not work > x = data.table(x=rnorm(1000)) ERROR: Exception calling Calloccolwrapper : Unimplemented GNU R API function 'DUPLICATE_ATTRIB'
- kirillseva 9y ago
  Well supposedly you'd use native data.frame's if you're using a faster R implementation. However, I would agree with you that at this point in time it's fair to assume that R is no longer a language in and of itself - it's more akin to a collection of APIs around years of optimized C++ code, somewhat separated into a few universes (bioinformatics, time series analysis with xtz, data.table aficionados and tidyverse acolytes). And thus making a faster R alternative should focus on 100% package compatibility along with speeding up native R code. Renjin, pqR [1], et. al. all seem to be compromises that work for some workflows but not all of them. The project that I'm excited for is fastR [2] - bringing R, C, C++ and FORTRAN code together into one Truffle/Graal environment would be absolutely huge for this ecosystem. However, I've tested fastR on my employer's [3] workflows and it does not work yet. [1] http://www.pqr-project.org/ http://www.pqr-project.org/ [2] https://github.com/graalvm/fastr https://github.com/graalvm/fastr [3] http://syberia.io/ http://syberia.io/
kazinator 9y ago
> Renjin allows the user to plugin best-of-class implementations of BLAS, LAPACK, and FFT, Makes sense. Why, if you want numerics that were traditionally written in Fortran, of course you have to go to JVM. :)
deleted 9y ago
[deleted]

118 points by phillc73 9y ago

IMHO the crucial question is whether users can use libraries that load native libraries and/or make use of rcpp. The docs make me think it's not possible.

phillc73 9y ago
Regarding Rcpp, their package library says "A Renjin-specific version of this package is available."[1] Looks like Renjin can also be used from GNU R as an R package[2]. I'd assume any installed R package should also work in conjunction with it. Admittedly, I haven't tried Renjin, but only just found it and thought it was quite interesting. [1] http://packages.renjin.org/package/org.renjin.cran/Rcpp http://packages.renjin.org/package/org.renjin.cran/Rcpp [2] http://docs.renjin.org/en/latest/package/index.html#using-renjin-as-an-r-package http://docs.renjin.org/en/latest/package/index.html#using-re...
- michaelmior 9y ago
  Even just using Renjin to be able to access Java code within R seems super useful.
  - kprybol 9y ago
    Wasn’t that already possible via the rJava library?
    - michaelmior 9y ago
      True. Although Renjin seems like it provides a bit of a nicer interface in some cases. (Disclaimer: not an R developer)
sorokod 9y ago
From the documentation [1]: "As a service, BeDataDriven provides a repository with all CRAN (the Comprehensive R Archive Network) and BioConductor packages at http://packages.renjin.org http://packages.renjin.org. The packages in this repository are built and packaged for use with Renjin. Not all packages can be built for Renjin so please consult the repository to see if your favorite package is available for Renjin." [1] http://docs.renjin.org/en/latest/interactive/index.html#prerequisites http://docs.renjin.org/en/latest/interactive/index.html#prer...
- Annatar 9y ago
  Yes. Renjin isn’t even a complete implementation of R and I don’t believe that a Java JIT will ever beat Fortran in performance. I’d sure like to see some evidence of JIT R beating the Fortran implementation. And then there is the memory inefficiency of Java. So again I ask: why?
  - mwexler 9y ago
    Because there is value in having a completely jvm stack in some places. Sometimes performance takes a back seat to consistency. It may not be right, but that's the way it is in some orgs.
  - acaloiar 9y ago
    When I was implementing Renjin's PRNG based on GNR R's, I read a fair amount of GNU R code. I think it goes without saying that C and Fortran perform well. However, I do believe that R's flexibility as a language comes at a performance cost. The implementation of that flexibility is not always done in an efficient manner; and as a result performance suffers. I.e. R is not particular performant in certain circumstances despite being largely implemented in C and Fortran. I only have anecdote to offer in this regard, but it is worth considering that GNU R's good performance is not a foregone conclusion.
  - phillc73 9y ago
    Hadley's book, "Advanced R" has a specific performance related chapter[1]. As well as techniques to boost performance within R, there's a section on "Alternative R implementations," which says in part, "There are some exciting new implementations of R. While they all try to stick as closely as possible to the existing language definition, they improve speed by using ideas from modern interpreter design." Renjin is listed in this section. While I don't have direct experience of Renjin, or any other alternative R interpreters, I'm inclined to believe Hadley has and if he believes they improve speed, I'll take him at his word. [1] http://adv-r.had.co.nz/Performance.html http://adv-r.had.co.nz/Performance.html

eggy 9y ago

R's wealth of libraries and syntax was always appealing to me, so maybe now I will look at how compatible and faster Renjin might be.

haifeng 9y ago

There is a java/scala project providing R like statistical computation: http://haifengl.github.io/smile/linear-algebra.html http://haifengl.github.io/smile/linear-algebra.html

Annatar 9y ago

I build and maintain the R stack in my organization. A good portion of the R core is written in Fortran for maximum performance and efficiency; the rest is in C and C++. Why would I want an interpreted, just-in-time, garbage collected version of R in a JVM when I can run at full speed on the real thing?

kitd 9y ago
> interpreted, just-in-time, For clarification, the JVM JIT-compiles the most frequently used blocks of bytecode to machine code, so the bulk of the heavy lifting is not interpreted. The benefit of doing it this way is that compilation can be optimised based on the runtime-characteristics of the code, not compiler judgement. Cases of JVM code running faster than equivalent C/C++ code are not uncommon. Aside from that, access to Java libraries (eg for integration) may be important to some shops.
Insanity 9y ago
I suppose their reasons are here: http://www.renjin.org/about.html http://www.renjin.org/about.html I've only used R for a short time, so I can't comment on it myself.
- Annatar 9y ago
  “R has been traditionally limited by the need to fit data sets into memory, and working with even modest sets of data can quickly exhaust memory due to historical limitations in GNU R interpreter’s implementation.” I see that the propaganda machine never rests, no matter which language. Either that, or they haven’t heard of Vertica’s R, which is excellent and counters these claims. The rest with the “fragmentation of R” is a poor marketing tack: people who run R are perfectly capable of thinking with their own brains, thank you very much; Renjin has no place patronizing. We’re not sheep.
  - kirillseva 9y ago
    that's a fairly toxic and silly attitude towards a free software project. Don't like it - don't use it. I personally think it's great that there are people who are working on making R faster/better/popular, even though I may disagree with the methods they've chosen to achieve those goals.

ianopolous 9y ago

FastR [1], an R implementation for the JVM using Truffle aims to be both fast and compatible, including C and fortran calls from R using Truffle's polyglot framework. The thing that makes this even more awesome is inter language optimization - inline your C code into your R code at runtime, which can result in zero overhead inter language calls. [1] https://github.com/graalvm/fastr https://github.com/graalvm/fastr

peatmoss 9y ago
One of the interesting things I note from the readme, is that they have an implementation of Grid graphics. Grid is pretty awesome for lower level graphics, and is part of the reason ggplot is able to be as good as it is. Looking forward to poking around at their implementation. If it’s a JVM friendly implementation, that could open up some nicer visualization options from other languges like clojure, scala, etc.
- mj_kallen 9y ago
  Indeed, they presented this at the last RIOT workshop in July [1] and it's a great way to support graphics without dealing with the grDevices package from GNU R. [1] https://youtu.be/otXTGBTb-3w?t=985 https://youtu.be/otXTGBTb-3w?t=985
  - peatmoss 9y ago
    Oh cheers! Looking forward to watching that.

peatmoss 9y ago

It's been a while since I've looked at Renjin. Last I looked, it was a worthy replacement for tasks that you'd do in base R, but the library story was incomplete. Nowadays, my R usage has gone full tidyverse. Anyone know if dplyr and the gang work here? I know dplyr has a lot of C++ code, and don't know how well the "Renjin-specific" Rcpp interop story is.

baldfat 9y ago
First thing I did was look at packages and see if tidyverse, purrr, lubridate, stringr and feather. They are all there. http://packages.renjin.org/packages/search?q=tidyverse http://packages.renjin.org/packages/search?q=tidyverse
- phillc73 9y ago
  Yes, but some of them aren't available yet, specifically dplyr: http://packages.renjin.org/package/org.renjin.cran/tidyverse http://packages.renjin.org/package/org.renjin.cran/tidyverse

deleted 9y ago

[deleted]

Myrmornis 9y ago

How does Renjin run the BLAS and LAPACK Fortean code, and the C code in R?

psk0 9y ago
We transpile all C, C++, and Fortran code directly to Java-bytecode using our open source tool called gcc-bridge (https://github.com/bedatadriven/renjin/tree/master/tools/gcc-bridge https://github.com/bedatadriven/renjin/tree/master/tools/gcc...). This is included as part of Renjin, but you can use it independently as part of your project as well.
- yazr 9y ago
  This is nice. Does anyone know of a similar tool for c# ?

tranwen 9y ago

Tried a few basic things, some of them did not work: 1. x = rnorm(1000) 2. plot(density(x)) --> not work 3. stem(x) --> not work 4. summary(x) --> works For R data handling, I always use data.table for its efficiency and power. 1. library(data.table) 2. x = data.table(x=rnorm(1000)) --> not work > x = data.table(x=rnorm(1000)) ERROR: Exception calling Calloccolwrapper : Unimplemented GNU R API function 'DUPLICATE_ATTRIB'

kirillseva 9y ago
Well supposedly you'd use native data.frame's if you're using a faster R implementation. However, I would agree with you that at this point in time it's fair to assume that R is no longer a language in and of itself - it's more akin to a collection of APIs around years of optimized C++ code, somewhat separated into a few universes (bioinformatics, time series analysis with xtz, data.table aficionados and tidyverse acolytes). And thus making a faster R alternative should focus on 100% package compatibility along with speeding up native R code. Renjin, pqR [1], et. al. all seem to be compromises that work for some workflows but not all of them. The project that I'm excited for is fastR [2] - bringing R, C, C++ and FORTRAN code together into one Truffle/Graal environment would be absolutely huge for this ecosystem. However, I've tested fastR on my employer's [3] workflows and it does not work yet. [1] http://www.pqr-project.org/ http://www.pqr-project.org/ [2] https://github.com/graalvm/fastr https://github.com/graalvm/fastr [3] http://syberia.io/ http://syberia.io/

kazinator 9y ago

> Renjin allows the user to plugin best-of-class implementations of BLAS, LAPACK, and FFT, Makes sense. Why, if you want numerics that were traditionally written in Fortran, of course you have to go to JVM. :)

deleted 9y ago

[deleted]