3 ms·

A little bit off topic, but has anyone run a r based api in production? What do you guys use - something like renjin...or do you put up reserve behind a connect

A little bit off topic, but has anyone run a r based api in production? What do you guys use - something like renjin...or do you put up reserve behind a connection pool ? It's really hard to find something around this.

jmalicki 11y ago
I've used rpy to have the web-api side done in python, then make a call to the R code. That way R can do the math it's good at, and python can do the logging/API/etc. stuff it's good at.
Hansi 11y ago
We have a C# WebAPI in front of R for our use. Each application can either do ad-hoc requests or register itself with the api, reserve N instances of R and a set of packages to be loaded on each and they get enacted as a queue for the app to handle incoming requests. I built the first version using rApache (http://rapache.net/ http://rapache.net/) and then did some testing with OpenCPU (https://www.opencpu.org/ https://www.opencpu.org/) but we had to make it run on Windows and IIS so we built our own in the end. http://deployr.revolutionanalytics.com/ http://deployr.revolutionanalytics.com/ could be a ready made option and seems to be pretty solid but we deemed it to complicated for our needs when we looked at it.
- sandGorgon 11y ago
  did you evaluate Renjin ? it seems that Java would have been a good fit for you . Also - what do you mean by "reserving N instances of R". I'm not sure what that means (coming from a Python/Java background).
  - Hansi 11y ago
    Yes, we looked at Renjin and it's just not what we want, they re-implemented R based off 2.14 and don't do feature parity with that version or newer versions and since the actual analytics code is developed in vanilla R it didn't seem attractive. Also this was a few years back and Renjin as is has sparse documentation and seemingly not too many users and back then the situation on those fronts was worse. To the instance question, I simply mean spinning up a N-amount (usually 4-6) of R processes (there are multiple apps and hundreds of users so we couldn't make it a single linear queue for processing). We mainly use it for optimization calls so we prime the optimization engine in R with the required packages etc and then the R processes just lingers around ready to do an optimization when required.
trestletech 11y ago
YHat and Domino Data Labs both have commercial solutions for exactly this. On the open-source front, OpenCPU (https://www.opencpu.org/ https://www.opencpu.org/), Jug (http://bart6114.github.io/jug/ http://bart6114.github.io/jug/) or plumber (http://plumber.trestletech.com/ http://plumber.trestletech.com/) are all built for this. Plumber has a page to guide you through setting it up on a server using PM2 here: http://plumber.trestletech.com/docs/hosting/ http://plumber.trestletech.com/docs/hosting/ (Conflict of interest warning: I wrote plumber.) I've been running an API on plumber (with admittedly low volumes of traffic) for months now unattended using a strategy similar to the guide mentioned above. Haven't had any issues yet.
- sandGorgon 11y ago
  Very cool! How do you manage this in production? We use supervisors for other services. Is plumber aware of USR1, SIGHUP,etc signals to trigger restarts,etc

jmalicki 11y ago

I've used rpy to have the web-api side done in python, then make a call to the R code. That way R can do the math it's good at, and python can do the logging/API/etc. stuff it's good at.

Hansi 11y ago

We have a C# WebAPI in front of R for our use. Each application can either do ad-hoc requests or register itself with the api, reserve N instances of R and a set of packages to be loaded on each and they get enacted as a queue for the app to handle incoming requests. I built the first version using rApache (http://rapache.net/ http://rapache.net/) and then did some testing with OpenCPU (https://www.opencpu.org/ https://www.opencpu.org/) but we had to make it run on Windows and IIS so we built our own in the end. http://deployr.revolutionanalytics.com/ http://deployr.revolutionanalytics.com/ could be a ready made option and seems to be pretty solid but we deemed it to complicated for our needs when we looked at it.

sandGorgon 11y ago
did you evaluate Renjin ? it seems that Java would have been a good fit for you . Also - what do you mean by "reserving N instances of R". I'm not sure what that means (coming from a Python/Java background).
- Hansi 11y ago
  Yes, we looked at Renjin and it's just not what we want, they re-implemented R based off 2.14 and don't do feature parity with that version or newer versions and since the actual analytics code is developed in vanilla R it didn't seem attractive. Also this was a few years back and Renjin as is has sparse documentation and seemingly not too many users and back then the situation on those fronts was worse. To the instance question, I simply mean spinning up a N-amount (usually 4-6) of R processes (there are multiple apps and hundreds of users so we couldn't make it a single linear queue for processing). We mainly use it for optimization calls so we prime the optimization engine in R with the required packages etc and then the R processes just lingers around ready to do an optimization when required.

trestletech 11y ago

YHat and Domino Data Labs both have commercial solutions for exactly this. On the open-source front, OpenCPU (https://www.opencpu.org/ https://www.opencpu.org/), Jug (http://bart6114.github.io/jug/ http://bart6114.github.io/jug/) or plumber (http://plumber.trestletech.com/ http://plumber.trestletech.com/) are all built for this. Plumber has a page to guide you through setting it up on a server using PM2 here: http://plumber.trestletech.com/docs/hosting/ http://plumber.trestletech.com/docs/hosting/ (Conflict of interest warning: I wrote plumber.) I've been running an API on plumber (with admittedly low volumes of traffic) for months now unattended using a strategy similar to the guide mentioned above. Haven't had any issues yet.

sandGorgon 11y ago
Very cool! How do you manage this in production? We use supervisors for other services. Is plumber aware of USR1, SIGHUP,etc signals to trigger restarts,etc