4 ms·
Does any of these support out of memory data? IIRC dplyr can use a db as backend but I could be wrong. Anyway, I wonder why not just use a db & sql for more dem
by stewbrew 10y ago
Does any of these support out of memory data? IIRC dplyr can use a db as backend but I could be wrong. Anyway, I wonder why not just use a db & sql for more demanding data munging?
- IndianAstronaut 10y agoI routinely do my data munging in SQL/Hive/Pig. I think R is more of a complement to that. Mainly for more data munging that needs some programmatic manipulation and of course advanced statistical analysis.
- gnufx 10y agoPresumably that depends at least on the type of data and whether you have a suitable database to use. As another approach, for R there's an interface to parallel NetCDF in the pbdR system that you might use for distributed processing on an HPC system. There doesn't seem to be anything like pdbR for Python, for instance.