3 ms·
Have you seen the R package data.table (https://github.com/rdatatable/data.table https://github.com/rdatatable/data.table)? Not sure whether data.table is in t
by hugh-avherald 8y ago
Have you seen the R package data.table (https://github.com/rdatatable/data.table https://github.com/rdatatable/data.table)?
Not sure whether data.table is in the same domain as xsv, and certainly a lot of it is written in C. But for comparison's sake:
fread("cities.csv") 1.30 s
And then the rest of the computations will be faster of course:
count -- 0.005 ms
freq -- 0.2 s
sort -- 0.1 s
It's so useful that I often just use csvs between 10 and 100GB as a database as the difference in performance between fread and a 'proper' database aren't enough to justify the latter.
- burntsushi 8y agoYes. I've used it lightly in the past and have never been impressed by its speed or memory usage. But there could have been user errors. I am not an R expert. :) In any case, I think R implements all or most of its data table transformations in C, so I don't think it applies well here.
- hugh-avherald 8y agoI'm curious as to what has impressed you with regard to speed or memory usage (at least over this scale). I realize there's a lot I don't know.