3 ms·

Was curious how fast the shell was versus something more purpose written. This short bit of (golfed/terse) Perl seems to be 2-3x faster for the inputs I tested

by tyingq 5y ago

Was curious how fast the shell was versus something more purpose written. This short bit of (golfed/terse) Perl seems to be 2-3x faster for the inputs I tested. Though it's also 2x more "code", despite golfing and skipping best practice strict/warnings. And both are wasting some time sorting the whole list versus a partial heap sort or similar. #!/usr/bin/perl $l=shift(@ARGV)||1; while (<>) { for (split) { $c{lc($_)}++; } } for (sort { $c{$b} <=> $c{$a} } (keys %c)) { print "$c{$_} $_\n"; last if ++$i == $l; }

chasil 5y ago
That would depend upon the shell that you used. GNU Bash in particular is not known for speed, although this is just a question of pipes. I remember that cdrecord has explict advice on raising pipe buffer size to avoid an underun (that will render a CD unusable). Perhaps pipe tuning here might also impact performance. I don't know if dash will increase performance in this particular case. $ rpm -qi dash | tail -3 DASH is a POSIX-compliant implementation of /bin/sh that aims to be as small as possible. It does this without sacrificing speed where possible. In fact, it is significantly faster than bash (the GNU Bourne-Again SHell) for most tasks.
- tyingq 5y ago
  Looking at the shell script, I think it will always be somewhat slow as it's sorting twice. One pass to sort the words so that "uniq -c" can work correctly, then the reverse numeric sort. I don't think the shell used matters, as most of the cpu time is in the sorting.
mst 5y ago
I think I'd prefer populating %c using: $c{lc $_}++ for map split(), <ARGV>; (I pondered 'do { local $/; <ARGV> }' to do a full slurp as well, but I don't think it helps with clarity and I'm not actually golfing here even though the resulting code happens to be shorter) Also, if you have List::UtilsBy handy (which I basically always do), rev_nsort_by { $c{$_} } keys %c could be a nicer way to express the sort, although probably as fast because of the block expr versus anon-sub-via-& prototype part. (I have not executed any of this code, I'm just musing here, so no warranty express or implied ;)

by tyingq 5y ago

chasil 5y ago

That would depend upon the shell that you used. GNU Bash in particular is not known for speed, although this is just a question of pipes. I remember that cdrecord has explict advice on raising pipe buffer size to avoid an underun (that will render a CD unusable). Perhaps pipe tuning here might also impact performance. I don't know if dash will increase performance in this particular case. $ rpm -qi dash | tail -3 DASH is a POSIX-compliant implementation of /bin/sh that aims to be as small as possible. It does this without sacrificing speed where possible. In fact, it is significantly faster than bash (the GNU Bourne-Again SHell) for most tasks.

tyingq 5y ago
Looking at the shell script, I think it will always be somewhat slow as it's sorting twice. One pass to sort the words so that "uniq -c" can work correctly, then the reverse numeric sort. I don't think the shell used matters, as most of the cpu time is in the sorting.

mst 5y ago

I think I'd prefer populating %c using: $c{lc $_}++ for map split(), <ARGV>; (I pondered 'do { local $/; <ARGV> }' to do a full slurp as well, but I don't think it helps with clarity and I'm not actually golfing here even though the resulting code happens to be shorter) Also, if you have List::UtilsBy handy (which I basically always do), rev_nsort_by { $c{$_} } keys %c could be a nicer way to express the sort, although probably as fast because of the block expr versus anon-sub-via-& prototype part. (I have not executed any of this code, I'm just musing here, so no warranty express or implied ;)