4 ms·
"Imagine you have a 4.2GB CSV file." " All you need... is the sum of all values in one particular column." In that case, if speed was paramount, I'd use Kona
by gwu78 13y ago
"Imagine you have a 4.2GB CSV file." "
All you need... is the sum of all values in one particular column."
In that case, if speed was paramount, I'd use Kona or kdb. Unquestionably, k is the best tool for that particular job.
- lotsofcows 13y agoReally? In a recent test of a whole bunch of languages, scripting, compiled and JVM (but not Kona or kdb), our awk test was beaten only by C. awk was so far ahead, its run time beat other's compile + run time.
- gwu78 13y agoYes, really. Nothing I know of beats the speed of k for this column-oriented type of task. Kernighan himself tested it against awk many years ago and if I recall it was generally faster even in that set of tests. Where can I see your experimental design? I'd like to try to replicate your results.