3 ms·
True enough, so I wondered: on a corpus of reasonable size, will the "sophisticated" python solution beat the "classic UNIX shell" solution. #!/usr/bin/env
by jepler 9y ago
True enough, so I wondered: on a corpus of reasonable size, will the "sophisticated" python solution beat the "classic UNIX shell" solution.
#!/usr/bin/env python3
import collections, sys
if len(sys.argv) > 1: N = int(sys.argv[1])
else: N = 10
for line, count in collections.Counter(sys.stdin).most_common(N):
print("%d\t%s" % (count, line.rstrip('\n')))
In either case, the "tr | tr |" part of the pipeline remains the same, but "| python3 commonline.py" replaces the rest of the original pipeline.
On my system, with the input being the 100,117 word "kjv10.txt" (from project gutenberg a decade ago), the Python solution takes about 1/4 of the time, .974s best of 8 "real" time for the original pipeline, and .209s for the python3 pipeline. According to "perf stat", the classic solution has 2,938 page faults instead of 1,877 for the solution that includes Python. So even at this input size, you can measure the difference; and depending how often you ran the procedure, it might even pay off vs the time it took me to research and implement the Python version.
(curiously, pulling the downcasing and splitting into python didn't seem beneficial)