3 ms·

And every time I see things like that, I feel like the person must be unaware of perl. I've made this point before, but I still find it hilarious. For more th

by ajross 3y ago

And every time I see things like that, I feel like the person must be unaware of perl. I've made this point before, but I still find it hilarious. For more than a decade, awk was dead. Like, dead dead. There was nothing you could do in awk that wasn't cleaner and simpler and vastly more extensible in perl. And, yes, perl was faster than gawk, just like gawk is faster than shell pipelines. Then python got big, people decided that they didn't want to use perl for big projects[1], and so perl went out of vogue and got dropped even for the stuff it did (and continues to do) really well. Then a new generation came along having never learned perl, and... ... have apparently rediscovered awk? [1] Also the perl 5 tree stagnated[2] as all the stakeholders wandered off into the weeds to think about some new language. They're all still out there, AFAIK. [2] Around 2000-2005, perl was The Language to be seen writing your new stuff in, so e.g. bioinformatics landed there and not elsewhere. But by 2015, the TensorFlow people wouldn't be caught dead writing perl.

voidfunc 3y ago
Perl never recovered from its many ways to do things label. It's a tired criticism of the language but it's lodged in the brains of a generation of programmers which is unfortunate. Also the classic sysadmin role which used to lean on Perl heavily sort of evolved with rise of The Cloud and automation tools like Chef, Puppet, and Ansible took over in that 2005-2015 time frame.
sgarland 3y ago
That's a fair criticism. I know Perl can do pretty amazing things with text, but I've never bothered to learn it. EDIT: I decided to ask GPT-4 to translate the gawk script to Perl. I make zero claims that this is ideal (as stated, I don't know Perl at all), but it _does_ produce the same output, but slightly slower than the gawk script. $ hyperfine "perl -lane '\$ips{\$F[0]}++; END {print \"\$ips{\$_} \$_\" for (sort {\$ips{\$b} <=> \$ips{\$a}} keys %ips)[0..9]}' fake_log.txt" Benchmark 1: perl -lane '$ips{$F[0]}++; END {print "$ips{$_} $_" for (sort {$ips{$b} <=> $ips{$a}} keys %ips)[0..9]}' fake_log.txt Time (mean ± σ): 1.499 s ± 0.006 s [User: 1.447 s, System: 0.050 s] Range (min … max): 1.490 s … 1.507 s 10 runs
- ajross 3y ago
  I would have gone with an iteratively-built list, FWIW, and avoided the overhead in parsing fields the script won't use: perl -e 'for $i (<>) { $i =~ s/ .*//; push @list, $i; }; print(sort(@list));'
pclmulqdq 3y ago
I am in the "awk > perl" camp. I think the idea of "vastly more extensible" is a negative for my scripting language, and "cleaner" just doesn't matter - I just want to write it the one time I want to use it and then be done with it. The awk language is really simple and quick to write. By the way, I think this is why Perl lost to Python on larger scripting and programming projects - it's just easier to write (albeit harder to read, to antagoinze the Python lovers out there).
tomjakubowski 3y ago
Sample of one. I came of age on Linux in the late 90s/early 00s. Through other nerds on IRC channels I became familiar with Perl and didn't like it. I also picked up basic awk in the context of one-liners for shell pipelines and it was pretty nice for that. Easier to remember than the flags for cut and friends. Learning awk a bit more deeply in recent years has been good too. I can write one liners that do more. I shipped a full awk script once, for something unimportant, but I would never do that again. For serious text munging these days I'd rather write a Rust program.
tptacek 3y ago
They taught awk to my boy in bioinformatics as part of his degree. I was like Vito Corleone in the funeral home when he showed me the FASTA parsing awk code they were working on.
telotortium 3y ago
I mostly use awk over perl because awk is completely documented in one man page, so it's easy to see whether awk will be fit for purpose or whether I should write it using a real programming language. I learned Perl over a decade ago, but not the really concise dialect you would use on the command line for stuff I'd use awk for, and I've forgotten almost all of it now. At least with awk it's easy to relearn the functions I need when I need it.
- ajross 3y ago
  Right, which is sort of my point. 20 years ago, "everyone" knew perl, at least to the extent of knowing the standard idioms for different environments that you're talking about. And in that world, "everyone" would choose perl for these tasks, knowing that everyone else would be expert enough to read and maintain them. Perl was the natural choice. And in a world where perl is a natural choice for these tasks, awk doesn't have a niche. Because at the end of the day awk is simply an inferior language. Which is the bit I find funny: we threw out and forgot about a great tool, and now we think that the ancestral toy it replaced is a good idea again.
tgv 3y ago
I learned perl around that time, and I thought it was awful. And just about everything about it: the parameter passing, the sigils that made BASIC look like Dijkstra's love child, the funky array/scalar coercion, and the bloody fact that it couldn't read from two files at once even though the docs suggested it should work. They didn't say so explicitly, because perl was pretty badly documented. My boss started writing object oriented perl, and that made perl unreadable even to perl experts. AWK, on the other hand, is simplicity itself. Sure, it misses a few things, but for searching through log files or db dumps it's an excellent tool. And it's fast enough. If you really need much more speed, there are other tools, but I would rather rewrite it in C than try perl again.

by ajross 3y ago

voidfunc 3y ago

Perl never recovered from its many ways to do things label. It's a tired criticism of the language but it's lodged in the brains of a generation of programmers which is unfortunate. Also the classic sysadmin role which used to lean on Perl heavily sort of evolved with rise of The Cloud and automation tools like Chef, Puppet, and Ansible took over in that 2005-2015 time frame.

sgarland 3y ago

That's a fair criticism. I know Perl can do pretty amazing things with text, but I've never bothered to learn it. EDIT: I decided to ask GPT-4 to translate the gawk script to Perl. I make zero claims that this is ideal (as stated, I don't know Perl at all), but it _does_ produce the same output, but slightly slower than the gawk script. $ hyperfine "perl -lane '\$ips{\$F[0]}++; END {print \"\$ips{\$_} \$_\" for (sort {\$ips{\$b} <=> \$ips{\$a}} keys %ips)[0..9]}' fake_log.txt" Benchmark 1: perl -lane '$ips{$F[0]}++; END {print "$ips{$_} $_" for (sort {$ips{$b} <=> $ips{$a}} keys %ips)[0..9]}' fake_log.txt Time (mean ± σ): 1.499 s ± 0.006 s [User: 1.447 s, System: 0.050 s] Range (min … max): 1.490 s … 1.507 s 10 runs

ajross 3y ago
I would have gone with an iteratively-built list, FWIW, and avoided the overhead in parsing fields the script won't use: perl -e 'for $i (<>) { $i =~ s/ .*//; push @list, $i; }; print(sort(@list));'

pclmulqdq 3y ago

I am in the "awk > perl" camp. I think the idea of "vastly more extensible" is a negative for my scripting language, and "cleaner" just doesn't matter - I just want to write it the one time I want to use it and then be done with it. The awk language is really simple and quick to write. By the way, I think this is why Perl lost to Python on larger scripting and programming projects - it's just easier to write (albeit harder to read, to antagoinze the Python lovers out there).

tomjakubowski 3y ago

Sample of one. I came of age on Linux in the late 90s/early 00s. Through other nerds on IRC channels I became familiar with Perl and didn't like it. I also picked up basic awk in the context of one-liners for shell pipelines and it was pretty nice for that. Easier to remember than the flags for cut and friends. Learning awk a bit more deeply in recent years has been good too. I can write one liners that do more. I shipped a full awk script once, for something unimportant, but I would never do that again. For serious text munging these days I'd rather write a Rust program.

tptacek 3y ago

They taught awk to my boy in bioinformatics as part of his degree. I was like Vito Corleone in the funeral home when he showed me the FASTA parsing awk code they were working on.

telotortium 3y ago

I mostly use awk over perl because awk is completely documented in one man page, so it's easy to see whether awk will be fit for purpose or whether I should write it using a real programming language. I learned Perl over a decade ago, but not the really concise dialect you would use on the command line for stuff I'd use awk for, and I've forgotten almost all of it now. At least with awk it's easy to relearn the functions I need when I need it.

ajross 3y ago
Right, which is sort of my point. 20 years ago, "everyone" knew perl, at least to the extent of knowing the standard idioms for different environments that you're talking about. And in that world, "everyone" would choose perl for these tasks, knowing that everyone else would be expert enough to read and maintain them. Perl was the natural choice. And in a world where perl is a natural choice for these tasks, awk doesn't have a niche. Because at the end of the day awk is simply an inferior language. Which is the bit I find funny: we threw out and forgot about a great tool, and now we think that the ancestral toy it replaced is a good idea again.

tgv 3y ago

I learned perl around that time, and I thought it was awful. And just about everything about it: the parameter passing, the sigils that made BASIC look like Dijkstra's love child, the funky array/scalar coercion, and the bloody fact that it couldn't read from two files at once even though the docs suggested it should work. They didn't say so explicitly, because perl was pretty badly documented. My boss started writing object oriented perl, and that made perl unreadable even to perl experts. AWK, on the other hand, is simplicity itself. Sure, it misses a few things, but for searching through log files or db dumps it's an excellent tool. And it's fast enough. If you really need much more speed, there are other tools, but I would rather rewrite it in C than try perl again.