4 ms·
In-database R coming to SQL Server 2016
- saosebastiao 11y agoI hope it's not as bad as PL/R. Seemed like a good idea, but the performance was so terrible that it was essentially useless.
- jeltz 11y agoAny idea what makes PL/R slow? Is it time spent doing data conversions from PostgreSQL types to R types?
- tankenmate 11y agoThis article[0] shows exactly this problem; and it links to patches that handle this issue. The article is 4.5 years old, not sure if the patches have been upstreamed. [0] http://www.credativ.co.uk/credativ-blog/2010/07/postgresql-topic-of-the-day---plr-performance-improvements http://www.credativ.co.uk/credativ-blog/2010/07/postgresql-t...
- atorralb 11y agojust so you know, is already in SAP HANA... is nice to see that programming languages are part of a database a not just an extension
- toddkazakov 11y agoIt's not actually in the database. It's a lousy implementation which does not use the distributed nature of HANA. You fetch the data first and then you do your analyses in R which is hosted near the DB. If you want parallel processing in R, you're on your own.
- fs111 11y agoHow does that work when R is GPL licensed? Doesn't that make SQL server a derived work?
- olh 11y agoNot the same R implementation.
- fs111 11y agoI see. Did they re-implement it from scratch?
- benologist 11y agoMS owns Revolution Analytics, can't they just ignore the license [MS now] provided for public use?
- hadley 11y agoRevolution analytics isn't a copyright holder of R, so no.
- toddkazakov 11y agoRevolution have their own implementation of R called OpenR.
- hadley 11y agoNo, that's not their own implementation. It's GNU r bundled with some extras
- Jake232 11y agoI'm always reluctant to these kinds of ideas, of executing code on/within my database server. I know it's apparently sandboxed, but that didn't work out too well for ElasticSearch recently: https://jordan-wright.github.io/blog/2015/03/08/elasticsearch-rce-vulnerability-cve-2015-1427/ https://jordan-wright.github.io/blog/2015/03/08/elasticsearc....
- SigmundA 11y agoSQL is code running in the database server. Most SQL implementations, TSQL, PL/SQL, PGSQL are turing complete as well.
- bladecatcher 11y agoWhile this is a good concept in theory, I'd be skeptical about building on top of such a system. The primary reason is the slowness of R. I built heavy duty data mining systems using a stack of kdb/q and R. In my experience, R, when used for simple clustering algorithms like k-means and k-medoids slowed down my system by nearly 70 times. This is despite running parallelized versions of these algorithms (by means of the SPRINT R package) using mpiexec. IMO, there is a very big gap in this space. There is an urgent need for high performant data inference languages. MATLAB is decent, but is still clunky for my taste. Plus, I prefer the simplicity of a file mapped column oriented database like the one offered by kdb. As KDB is too expensive for me right now, I'm considering building on top of the excellent J language/JDB database stack for my big data needs.
- gaius 11y agoThis will all be down to the implementation of DataFrame no? In other words if you put q ontop of a R DataFrame would you expect the same performance? Or if you ran R ontop of K.
- bladecatcher 11y agoI'm not sure if I understand your question correctly. kx provides shared libraries to make R and q talk to each other. The specific approach I followed used a shared library that brings R into the kdb+ memory space, meaning all the R statistical routines and graphing capabilities can be invoked directly from kdb+. Using this method means data is not passed between remote processes.
- gaius 11y agoAh I assumed you were loading a vanilla DataFrame from the column store. My bad :-)
- steego 11y agoHow does Julia work in this space? What's it lacking?
- gtrubetskoy 11y agoFor what it's worth, PostgreSQL had this since 2003. http://www.joeconway.com/plr/ http://www.joeconway.com/plr/ IMHO scripts running in a database server never work all that well - debugging is a nightmare. At least this has been my experience from trying PG plpython a few years ago. Link to the original announcement email: http://www.postgresql.org/message-id/3E514A46.2040604@joeconway.com http://www.postgresql.org/message-id/3E514A46.2040604@joecon...
- hadley 11y agoR does have some pretty cool tools for after the fact debugging, like dump.frames, but few people know about them.
- vittore 11y agoWhy Microsoft? Why? Why not python?
- M8 11y agoThey are not gluing layers, they are adding statistical capabilities. Each language surves it's purpose.
- SigmundA 11y agoI would say why not improve the CLR support that has been there since SQL 2005?
- BrentOzar 11y ago> I would say why not improve the CLR support that has been there since SQL 2005? Because it won't cause new license sales, whereas integrating R might.
- modarts 11y agoShudder
- sorokod 11y agoHow does Java inside Oracle DB is doing nowadays?