5 ms·
That statement is overly vague and sounds like marketing BS. I worked on a project where we scored streaming data in R. The biggest bottleneck was getting the
by apohn 10y ago
That statement is overly vague and sounds like marketing BS.
I worked on a project where we scored streaming data in R. The biggest bottleneck was getting the data into and out of the R session. We started out using disk based I/O and ended up using using rJava so our streaming system could communicate with R. In that case we did get a 100X speed up between our first iteration and the final version which used rJava to serialize the data.
So basically, the major bottleneck was not R. It was the communication with R. In the article R is installed on the same hardware as SQL Server, which should automatically give it a speedup with streaming data.
If Microsoft also has an optimized way to get data from SQL Server to R I can see how they got a 100X speedup. In certain cases using the MKL libraries can give you that as well from faster scoring, but I suspect the speedup just comes from improving the data transfer method.
- buro9 10y ago> If Microsoft also has an optimized way to get data from SQL Server to R I can see how they got a 100X speedup. In certain cases using the MKL libraries can give you that as well, but I suspect the speedup just comes from improving the data transfer method. The optimized method is that you can run R inside the database in the latest version of SQL Server. I've actually installed Windows again, just to play with this feature (though I cannot make claims to actually putting it to good use yet).
- deleted 10y ago[deleted]
- apohn 10y agoThis is what the docs say "When you select this feature, extensions are installed in the database engine to support execution of R scripts, and a new service is created, the SQL Server Trusted Launchpad, to manage communications between the R runtime and the SQL Server instance." So basically SQL Server is talking to the R session. The speedup is coming from R being installed locally and the "communication" which I've yet to figure out.
- baldfat 10y ago> I've actually installed Windows again, just to play with this feature (though I cannot make claims to actually putting it to good use yet). I heard the Linux SQL Server is surprisingly decent. https://blogs.microsoft.com/blog/2016/03/07/announcing- https://blogs.microsoft.com/blog/2016/03/07/announcing- sql-server-on-linux/
- apathy 10y agopostgresql used to have a way to embed R into PostGreSQL too: https://github.com/jconway/plr https://github.com/jconway/plr I guess it still does, though I haven't used it in years. You can, of course, do the same thing with Python.
- Jweb_Guru 10y agoNo need to install Windows to get R in a database, you can run with PL/R on Postgres (unless you have a particular desire to run it within SQL Server, of course, but doesn't it run on Linux now?). http://www.joeconway.com/plr/ http://www.joeconway.com/plr/
- blahi 10y agoThat's a lot less than what Microsoft is offering.
- Jweb_Guru 10y agoThat may be, but it's not clear to me from the article. What's the feature I'm missing that makes the SQL Server offering so much more compelling?
- blahi 10y agoOut of core algorithms, attatchment of data straight into R for a couple of very important things.
- Jweb_Guru 10y ago> Out of core algorithms. Does this just mean third-party modules? If so, doesn't http://www.joeconway.com/plr/doc/plr-module-funcs.html http://www.joeconway.com/plr/doc/plr-module-funcs.html suffice? > attatchment of data straight into R for a couple of very important things. Isn't this doable through http://www.joeconway.com/plr/doc/plr-global-data.html http://www.joeconway.com/plr/doc/plr-global-data.html? Apologies if these are dumb questions, as I'm not very familiar with R.
- blahi 10y agoNo. They have their own highly optimized algorithms. They also have their their own distributed data structures. I am not sure about that 2nd link. Seems like it's just UDFs written in R.