4 ms·
With super small dataset sizes like that it'll fit in the L2 cache and can behave differently. I did test this though, and for 20,000 items generated with your
by justinsaccount 7y ago
With super small dataset sizes like that it'll fit in the L2 cache and can behave differently. I did test this though, and for 20,000 items generated with your loop:
sort | uniq -c takes .017s (fastest out of a few runs)
the awk command I used above takes .013s
A trivial implementation I have in go takes .08s
Additionally, using this 'protos' file which is 1,000,000 lines of tcp,udp,icmp:
$ time (sort protos|uniq -c)
5915 icmp
332003 tcp
662082 udp
real 0m0.232s
user 0m0.739s
sys 0m0.100s
$ # fixed to count 'lines' and not the first column, which makes it faster.
$ time awk '{lines[$0]++} END {for (l in lines) printf("%s %d\n", l, lines[l])}' < protos
icmp 5915
udp 662082
tcp 332003
real 0m0.194s
user 0m0.190s
sys 0m0.004s
$ time ./c < protos
662082 udp
332003 tcp
5915 icmp
real 0m0.088s
user 0m0.084s
sys 0m0.005s
so yes, I do test my assumptions.
- dredmorbius 7y agoI was suggesting @crystaldev may not have. And, more significantly, and as you've confirmed, not all in-memory processing is equivalent. There are faster and slower all-in-memory algorithms and implementations Hrm... Maybe a bogounique implementation might be appropriate here.... (By analogy to bogosort: https://en.wikipedia.org/wiki/Bogosort https://en.wikipedia.org/wiki/Bogosort)