4 ms·
Are you being difficult on purpose? I posted a comment on how 'sort | uniq -c | sort -n' is an interesting and very capable pipeline, but often misused and slo
by justinsaccount 7y ago
Are you being difficult on purpose?
I posted a comment on how 'sort | uniq -c | sort -n' is an interesting and very capable pipeline, but often misused and slower than other alternatives.
> you are comparing
Yes, I am comparing two methods of accomplishing the same thing. That is how comparing things works.
> Please, "huge waste"? How do you sort something that does not fit in memory?
Note how the full sentence included "if you give it 100GB of 5 different strings". If your input is 100GB of 5 different strings, then the hash table will easily fit in memory, and sorting the entire data set only to pass it to 'uniq -c' is indeed a 'huge waste'.
There are tons of large data sets that only have a small number of unique values in particular fields. protocols, ports, http status codes, hour of the day, etc. 'sort | uniq -c | sort -n' will work for all of them, but not nearly as efficient a hash table.
- tuldia 7y ago> Are you being difficult on purpose? Programming is about paying the bare minimum attention to the details. > [...] two methods of accomplishing the same thing [...] Absolutelly not. one prints: 72000000 hello 72000000 world the other hello 72000000 world 72000000 Now try both examples against a file with more than one column to understand what I'm talking about ;)