3 ms·
This is functionally similar to dplyr in R. Although the more SQL like syntax of dplyr is much more handy.
by IndianAstronaut 10y ago
This is functionally similar to dplyr in R. Although the more SQL like syntax of dplyr is much more handy.
- michaelsbradley 10y agoAnd when the data gets bigger, there's data.table[1], which performs amazingly well at certain tasks (vectorized ops ftw!), though the syntax can get a little clunky (if you squint at it hard, it's SQL-ish). On my 2012 macbook pro, I'm able to do (some) transformations of tables containing 10s of millions of rows in only a few seconds (and sometimes faster). It's possible to use dplyr and data.table together, as well, to good effect[2]. [1] https://github.com/Rdatatable/data.table/wiki https://github.com/Rdatatable/data.table/wiki [&] https://github.com/Rdatatable/data.table/wiki/Benchmarks-%3A-Grouping https://github.com/Rdatatable/data.table/wiki/Benchmarks-%3A... [2] http://stackoverflow.com/questions/21435339/data-table-vs-dplyr-can-one-do-something-well-the-other-cant-or-does-poorly/27840349#27840349 http://stackoverflow.com/questions/21435339/data-table-vs-dp... [&] https://twitter.com/hadleywickham/status/553169339751215104 https://twitter.com/hadleywickham/status/553169339751215104
- IndianAstronaut 10y agoAre you able to load data sets into data.table which are larger than memory?
- michaelsbradley 10y agoAFAIK, with data.table it's all in-memory; whereas dplyr has the option of working with a database backend. On the other hand, data.table has robust support for modify and update ops by reference, which can be a big performance saver.