4 ms·

from sh import wc, ls print(wc(ls("-1"), "-l"))

• citrin_ru 6y ago
  Try to do in Python something like: fgrep -f keywords huge.log | cut -f 2- | sort | uniq -c | sort -rn | head Where size of data >> RAM (but less than free space in /tmp). It of course possible, but I suspect would require more effort and would work slower than a shell line above.
  • Too 6y ago
    If you want to compare memory efficiency. The sort+uniq -c would likely be more optimal in a high level language as you would there keep only the count+value in a defaultdict or alike instead of sorting the entire dataset, which requires keeping it all in mem. As for brevity, yeah, this particular example would require 5 lines instead of 1.
    • citrin_ru 6y ago
      For small datasets something like defaultdict may be more efficient, but if it would not fit in RAM you have a problem.