3 ms·

Awk as a major systems programming language, revisited (2018)

116 points by goranmoomin 3y ago

dang 3y ago
Related: Awk As A Major Systems Programming Language, Revisited (2018) - https://news.ycombinator.com/item?id=22304017 https://news.ycombinator.com/item?id=22304017 - Feb 2020 (80 comments)
RodgerTheGreat 3y ago
In my mind, the greatest strength of awk as a scripting language is its widespread availability. Whether you're using a modern Mac fresh out of the box, GitBash on a windows machine, or talking to a router over SSH, you can be confident that some version of awk is close at hand. The cost of this ubiquity, however, is extreme fragmentation. As is usually the case with GNU extensions, gawk provides a variety of shiny baubles which absolutely cannot be counted upon if you want to write a portable script. As a result, they're traps for the unwary to at least the same extent as they are handy tools. There are a variety of small features which could greatly enhance the flexibility of awk, including things described in this article and more ambitious ideas like making associative arrays first-class values, but if they aren't broadly supported, I don't imagine they'd have much impact. If I need to install a specific version of awk to run a script (and I have the ability to do so), I could just as easily install one of dozens of other capable scripting languages instead.
- statquontrarian 3y ago
  My first question to any programming question is, "can I reasonably solve this with awk?" exactly for its ubiquity. I stick to the POSIX standard for the reasons you note: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/a... If POSIX awk is insufficient, then I probably should be using something other than awk, so its limits are actually a good guardrail; however, it's shocking how powerful POSIX awk is with built-in arrays, hashmaps, regular expressions, etc., although no doubt there are quirks.
- classichasclass 3y ago
  And this is where Perl 5 says hi. It's also widespread and easily available, can do anything awk can, and isn't similarly fractured (Raku now being its own spinoff world). The article's complaints about its syntax are a matter of taste, of course.
  - mst 3y ago
    And the article's complaint about its stat() interface, while accurate in theory, misses the existence of https://p3rl.org/File::stat https://p3rl.org/File::stat (and there's some similar other modules for things like localtime()) which gives you named access. Also being able to use https://p3rl.org/App::FatPack https://p3rl.org/App::FatPack to bundle (pure perl) dependencies along with your code into a single script file to ship around makes life significantly easier deployment-wise (and https://p3rl.org/Object::Remote https://p3rl.org/Object::Remote allows for 'no deployment at all', it runs 'ssh <target> perl -' and ships the code it needs across the wire on connect).
dougb5 3y ago
I feel like I might hold the record for the most absurd AWK success story, which I hope to write up some day. I wrote the front-end for what became my highest-traffic website (RhymeZone.com) in AWK back in 1996, and it ran for 27 years with just occasional feature additions every few years, serving up to 2 million pages a day from a single medium-size EC2 instance at its peak. Even in 1996 it seemed like a bad idea to make a website in pure awk, but looking back I'm glad I did.
- j0hnyl 3y ago
  I would love to see the code.
- rjeli 3y ago
  I love rhymezone, I use it all the time! Thanks!
- maxfurman 3y ago
  Any chance you could release the source code some day? I'm very curious how you pulled this off!
- orhmeh09 3y ago
  Thanks for making RhymeZone. I use it for wordplay and jokes when I'm posting.
- williamcotton 3y ago
  You’ve contributed to more than one of my songs and poems! Thanks!
jgord 3y ago
I was impressed with awk generally, having discovered it by accident as an alternative to bash and node.js for various data wrangling tasks. The frawk implementation written in rust seemed very fast. I could not find a good way to emit binary files directly, and settled on using a unix util hex2bin or something like that. If I knew a nice way to handle binary, Id program more in awk.
- 082349872349872 3y ago
  both posix awk and the one true awk have printf: https://github.com/onetrueawk/awk/blob/2bab10b60b3f4d3bbefdcb60410655f317d1a452/awk.1#L189 https://github.com/onetrueawk/awk/blob/2bab10b60b3f4d3bbefdc... (that said, I usually use awk's ability to print through pipelines given in the awk script itself)
ykonstant 3y ago
Here is a yak-shaving challenge I posed for your amusement: https://www.reddit.com/r/commandline/comments/1bnehga/find_combinations_of_letters_forming_the_lagerst/ https://www.reddit.com/r/commandline/comments/1bnehga/find_c...

116 points by goranmoomin 3y ago

dang 3y ago

Related: Awk As A Major Systems Programming Language, Revisited (2018) - https://news.ycombinator.com/item?id=22304017 https://news.ycombinator.com/item?id=22304017 - Feb 2020 (80 comments)

RodgerTheGreat 3y ago

In my mind, the greatest strength of awk as a scripting language is its widespread availability. Whether you're using a modern Mac fresh out of the box, GitBash on a windows machine, or talking to a router over SSH, you can be confident that some version of awk is close at hand. The cost of this ubiquity, however, is extreme fragmentation. As is usually the case with GNU extensions, gawk provides a variety of shiny baubles which absolutely cannot be counted upon if you want to write a portable script. As a result, they're traps for the unwary to at least the same extent as they are handy tools. There are a variety of small features which could greatly enhance the flexibility of awk, including things described in this article and more ambitious ideas like making associative arrays first-class values, but if they aren't broadly supported, I don't imagine they'd have much impact. If I need to install a specific version of awk to run a script (and I have the ability to do so), I could just as easily install one of dozens of other capable scripting languages instead.

statquontrarian 3y ago
My first question to any programming question is, "can I reasonably solve this with awk?" exactly for its ubiquity. I stick to the POSIX standard for the reasons you note: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/a... If POSIX awk is insufficient, then I probably should be using something other than awk, so its limits are actually a good guardrail; however, it's shocking how powerful POSIX awk is with built-in arrays, hashmaps, regular expressions, etc., although no doubt there are quirks.
classichasclass 3y ago
And this is where Perl 5 says hi. It's also widespread and easily available, can do anything awk can, and isn't similarly fractured (Raku now being its own spinoff world). The article's complaints about its syntax are a matter of taste, of course.
- mst 3y ago
  And the article's complaint about its stat() interface, while accurate in theory, misses the existence of https://p3rl.org/File::stat https://p3rl.org/File::stat (and there's some similar other modules for things like localtime()) which gives you named access. Also being able to use https://p3rl.org/App::FatPack https://p3rl.org/App::FatPack to bundle (pure perl) dependencies along with your code into a single script file to ship around makes life significantly easier deployment-wise (and https://p3rl.org/Object::Remote https://p3rl.org/Object::Remote allows for 'no deployment at all', it runs 'ssh <target> perl -' and ships the code it needs across the wire on connect).

dougb5 3y ago

I feel like I might hold the record for the most absurd AWK success story, which I hope to write up some day. I wrote the front-end for what became my highest-traffic website (RhymeZone.com) in AWK back in 1996, and it ran for 27 years with just occasional feature additions every few years, serving up to 2 million pages a day from a single medium-size EC2 instance at its peak. Even in 1996 it seemed like a bad idea to make a website in pure awk, but looking back I'm glad I did.

j0hnyl 3y ago
I would love to see the code.
rjeli 3y ago
I love rhymezone, I use it all the time! Thanks!
maxfurman 3y ago
Any chance you could release the source code some day? I'm very curious how you pulled this off!
orhmeh09 3y ago
Thanks for making RhymeZone. I use it for wordplay and jokes when I'm posting.
williamcotton 3y ago
You’ve contributed to more than one of my songs and poems! Thanks!

jgord 3y ago

I was impressed with awk generally, having discovered it by accident as an alternative to bash and node.js for various data wrangling tasks. The frawk implementation written in rust seemed very fast. I could not find a good way to emit binary files directly, and settled on using a unix util hex2bin or something like that. If I knew a nice way to handle binary, Id program more in awk.

082349872349872 3y ago
both posix awk and the one true awk have printf: https://github.com/onetrueawk/awk/blob/2bab10b60b3f4d3bbefdcb60410655f317d1a452/awk.1#L189 https://github.com/onetrueawk/awk/blob/2bab10b60b3f4d3bbefdc... (that said, I usually use awk's ability to print through pipelines given in the awk script itself)

ykonstant 3y ago

Here is a yak-shaving challenge I posed for your amusement: https://www.reddit.com/r/commandline/comments/1bnehga/find_combinations_of_letters_forming_the_lagerst/ https://www.reddit.com/r/commandline/comments/1bnehga/find_c...