11 ms·

Awk: `Begin { ` Part 1

216 points by quyleanh 6y ago

awild 6y ago
I have been using AWK at work and found it cumbersome for my use-case (run-once analysis/manipulation of 200GB csv datasets). I found out about Miller[1] a year or so back and have been using that instead. I don't know how it stacks up in terms of performance, but for my money, named arguments and one-shot statistics is all I need. For example analysing the number of people per-year over multiple differently formatted files is as easy as `mlr uniq -f pid,year then count -g year`. It has filtering, very extensive manipulation capabilities and a nice documentation. John has also reacted very fast on feature requests I had (so fast that I've not yet implemented using them). [1]: https://miller.readthedocs.io/en/latest/#overview https://miller.readthedocs.io/en/latest/#overview
- Gravityloss 6y ago
  This seems very interesting. Making this reproducable and usable for multiple people has always been an annoyance in semi-ad-hoc data collection.
- charris000 6y ago
  +1 for Miller. It's a really slick tool for exploring large csv datasets. I've typically used it to do some prototyping and exploration of 200-500GB csv datasets before doing more hefty work in Java+PigLatin (our use-case is more long-term than just a single run for analysis, so that's re reason for moving out of just Miller). It's great to get a feel for the data and run some initial analysis before diving into the larger, more cumbersome system and tooling.
mauvehaus 6y ago
Standard awk warning: it's tempting to try to use awk on csv files. You'll even get good results on simple csv files that leave you encouraged to go further. Don't. Csv is not standardized and the quoting rules are weird (and not standardized). If you can live with a certain amount of loss of fidelity in your output, you can get away with using awk. If you want a coarse prototype, use awk. If you need robust, production-grade handling of csv files, use (or write) something else. Csv files are a little bit like like dates: superficially simple, with lots of corner cases. Largely for the same reason: lack of standardization. That said, awk is awesome. It's small enough to fit in your brain, unlike Perl (maybe yours is larger than mine?). It's also pretty universally available, with few massive incompatibilities between versions, unlike shell (provided you avoid the gawk-specific features). I love it.
- genuinebyte 6y ago
  The best description of dates and CSV files I've heard by far. Bravo. "Csv files are a little bit like like dates: superficially simple, with lots of corner cases. Largely for the same reason: lack of standardization."
  - cafard 6y ago
    I just attempted to run some government data through Python using the csv module. It worked like a charm, until it told me that a field had exceeded the maximum allowable length. A close look showed that a field began but did not end with a quotation mark. Using the csv.QUOTE_NONE flag resolved that, but did not remedy such quirks as short or long records. CSV files, depending on who generates them, are a bit like dates if the status of a year as leap or not depended on whether the date of Easter were 0 mod 4.
    - jsdwarf 6y ago
      My gripe with the Python csv.reader module is that it has no skip_lines parameter. Ill-formatted lines at the top/bottom of a CSV file are quite common
- bluedino 6y ago
  CSV standards are whatever the application implemented. We'd pull a "CSV" out of one system, then have to massage it before uploading to system A and then make other changes so it would be recognized by system B. I'm torn. On one hand, it's easy. It lets people work with the data, albeit using error-prone tools like NotePad++ and Excel. On the other hand, it's just flat out lazy to use.
- mehrdadn 6y ago
  Tangent, but another interesting problem with date parsing besides lack of standardization is that it's ridiculously context-sensitive. For example, if I schedule an event for Oct 1-Jan 1 right now, the most likely (least surprising) parse would be Oct 1, 2021 through January 1, 2022. Which is both surprising because parsing Oct 1 depends on the current date, and because the parsing of Jan 1 depends on the parse of Oct 1, and because both of these depend on the fact that you're scheduling (in all likelihood) a future event, rather than describing a past event. So whatever date parser you use (no matter how good it is at handling different notations) will return garbage if it doesn't take all these into account.
asicsp 6y ago
Nice introduction (based on both tutorials). One suggestion would be to use -v FPAT='[^,]*|"[^"]+"' instead of BEGIN { FPAT = "[^,]*|\"[^\"]+\"" } >If you get the awk programming language manual…you’ll read it in about two hours and then you’re done. That’s it. You know all of awk. I can't work my head around this quote. That's a ridiculous claim. Even for a experienced programmer, learning a new programming language in 2 weeks, let alone 2 hours would be nothing short of a miracle. I've been using awk for past 2-3 years or so and I wrote a book on GNU awk one-liners earlier this year (https://learnbyexample.github.io/learn_gnuawk/ https://learnbyexample.github.io/learn_gnuawk/). I'm nowhere close to knowing all of awk
- samoa42 6y ago
  > I've been using awk for past 2-3 years or so and I wrote a book on GNU awk one-liners not to criticise your work, but this does not sit well with me.
  - asicsp 6y ago
    Could you elaborate? I didn't get your point.
- Athas 6y ago
  GNU Awk has lots of extensions and is a relatively big language, but I definitely think you can learn most of the original Awk in two weeks - and you can certainly learn enough to be useful in two hours. It's been a decade now, but my recollection is that when I spent a day or two idly reading the Awk book, and was able to immediately employ it in simple shell pipelines after reading not that many pages. I think Awk is useful once you've learned just a bit about it, and I don't think learning a lot about Awk is very useful - there are languages that are better suited for complex tasks.
  - asicsp 6y ago
    Fair enough, someone else made similar point that I had GNU awk in mind wrt all of awk And yeah, it's possible to start using basic field processing and regexp features (if you already know it) in 2 hours and learn most of it in 2 weeks. But, that quote could've been something like you could get started in 2 hours instead of saying one could know all of awk.
rethab 6y ago
I don't agree that you can learn AWK in its entirety in two hours, but I do (strongly) agree that you can teach yourself the basics in two hours and those two hours are extremely well spent considering that AWK is useful for so many things. Surely handling the intricacies of CSV can be a pain, but let's face it: Most tasks are simple and its results are easily verified for correctness. Obviously, I don't write my production webserver with AWK -- I write simple script to parse data (locally), find semantic issues in CSV files, analyze log data, etc.. AWK is great! If you want a slightly bigger cheatsheet, here's mine: https://github.com/rethab/awk-scripts/blob/master/talk.txt https://github.com/rethab/awk-scripts/blob/master/talk.txt
- kqr 6y ago
  You can also trivially come across data designed for consumption with AWK. Whenever I write a script or utility to dump some data for later analysis, I take into consideration if I can shape the output in a way that will make it easy to work with in AWK. Because I won't know ahead of time what all the interesting values or summaries will be, but if I know I can get it through AWK it will be cheap to extract or compute those values and summaries later, once I know what they are.
- disown 6y ago
  The biggest problem with awk is that unless you use it regularly, you'll easily forget how to use it. So it's important to keep a running cheatsheet like you did. Nothing more frustrating than forgetting the awk command and searching for it in your history only to find it aged out.
cesaref 6y ago
My favourite bit of gawk (ok, an extended version of awk but it's probably what you are running) is the network support, which in the manual includes writing simple HTTP servers with it. I kid you not.
diegocg 6y ago
FYI, LWN has a great article about the current status of awk, latest features and future developments https://lwn.net/Articles/820829/ https://lwn.net/Articles/820829/
- swills 6y ago
  Happy to see this mentions https://github.com/onetrueawk/awk https://github.com/onetrueawk/awk
yesenadam 6y ago
> He said, “If you get the awk programming language manual…you’ll read it in about two hours and then you’re done. That’s it. You know all of awk.” It's hyperlinked to the Gawk manual, but it seems likely he actually meant A, W & K's The Awk Programming Language (1988), which you could conceivably read in 2 hours, as it's a joy to read. I used it and The C Programming Language as exemplars of great documentation when writing my own.
- elvis70 6y ago
  It's one of my favorite technical books, with many examples of the application of the language, including the design of small languages and experimentation with algorithms. It is freely available at archive.org: https://archive.org/details/pdfy-MgN0H1joIoDVoIC7 https://archive.org/details/pdfy-MgN0H1joIoDVoIC7
  - maxioatic 6y ago
    Much thanks
- statquontrarian 6y ago
  There's also the POSIX standard which is quite readable: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/a...
- elvis70 6y ago
  By the way, "Learning Perl" is also a great introductory book to a language.
xashor 6y ago
Instead of the gawk manual, I recommend the OpenBSD manpage: https://man.openbsd.org/awk.1 https://man.openbsd.org/awk.1 It's concise and well written (as most OpenBSD manpages), so you can learn AWK even in less than 2 hours. Of course it handles only AWK, not the gawk extensions.
- nmz 6y ago
  mawk also has a nice and short manual. since GNU decided to make its money selling books, their manuals are usually huge and that's by design.
motiejus 6y ago
Fred Hebert (a known name in Erlang ecosystem) created a tutorial "awk in 20 minutes": https://ferd.ca/awk-in-20-minutes.html https://ferd.ca/awk-in-20-minutes.html 20 minutes for that tutorial is enough to write non-trivial awk programs, and, least or most importantly, impress colleagues.
Zhyl 6y ago
I always link to this blog post [0] by Jonathan Palardy. It was one of those eye opening articles for me that really helped me 'grok' it - and this is after having had a couple of stabs at learning it just from reading the text books on it. Awk for me is part of the layers of tools from grep/sed -> AWK -> perl one liner -> full on script. The further down the ladder you go, the more restricted you are, but this means that programs are terser and usually stay readable despite the terseness. I would also recommend Perl One liners [1] which did the same thing for me for Perl one liners and, subsequently, full Perl. [0] https://blog.jpalardy.com/posts/why-learn-awk/ https://blog.jpalardy.com/posts/why-learn-awk/ [1] https://nostarch.com/perloneliners https://nostarch.com/perloneliners
arminiusreturns 6y ago
My story about learning the power of awk was I was working on a dataset and had written a page or so Worth's of Perl... and kept bungling it. I emailed the scientist who wrote the paper whose technique I was emulating and he said, why not use awk, and gave me a basic example one liner, with a bit of modification it worked! Of course all that really means is I suck at Perl, but it was an eye opener for how powerful awk is.
- bingerman 6y ago
  I learned awk before Perl and it helped a lot. I still use awk (and sed) regularly but for more complex tasks involving regexes and such Perl is my favourite.
js2 6y ago
Here's a basic network calculator I wrote in awk once because I had to do these calculations on a minimal SunOS system that didn't have any other way to do math from the shell. https://gist.github.com/jaysoffian/7bb70e1065f085b46a00 https://gist.github.com/jaysoffian/7bb70e1065f085b46a00 You never know when a bit of awk will save your bacon.
deleted 6y ago
[deleted]
untoxicness 6y ago
Brian Kernighan talked about his love of awk on the Lex Fridman podcast [0]. [0] https://lexfridman.com/brian-kernighan/ https://lexfridman.com/brian-kernighan/
SoSoRoCoCo 6y ago
The O'Reilly book is "Sed & Awk" published in the 90's (https://www.oreilly.com/library/view/sed-awk/1565922255/ https://www.oreilly.com/library/view/sed-awk/1565922255/). The languages work together as a team (like lex and yacc). And the book takes about 2 hours to read. Unix is built on the premise of "lots of simple tools that fit together with pipes". Although this is like 1,000th article on how easy it is to learn awk. Why write yet another one? Awk in 20 Minutes (2015) https://news.ycombinator.com/item?id=23048054 https://news.ycombinator.com/item?id=23048054 Learn just a little Awk (2010) https://news.ycombinator.com/item?id=17322412 https://news.ycombinator.com/item?id=17322412 Learn Awk by Example (2019) https://news.ycombinator.com/item?id=22455779 https://news.ycombinator.com/item?id=22455779 it goes on and on... https://hn.algolia.com/?q=awk https://hn.algolia.com/?q=awk
- celticninja 6y ago
  The reason to write one is to record what you learn, to further embed it into your brain. Why post it? Whoever posted it found it interesting. I had not seen the others so this is an intro to awk for me. But in general if something has made it to the front page of HN it was interesting enough for enough people to put it there.
Xophmeister 6y ago
I love Awk. The idea of a programming language that effectively facilitates first-class state machines over text streams fits so well with the Unix pipeline model.
ensiferum 6y ago
Awk and shell scripting is the rabbit hole that is best avoided IMHO. Often it's just easier to crank out a "real" program using your lost familiar programming language or at very least python instead of starting to work with shell or awk scripting.
- nmz 6y ago
  Awk is a full blown programming language though. comparable to python but allowing for much more compact code. Though I completely agree on avoiding shell. the only good shell I've seen is rc. but the relying on external utilities is a double edged sword. forking is too costly over time.
known 6y ago
Before using AWK we need to sanitize the data files;

216 points by quyleanh 6y ago

I have been using AWK at work and found it cumbersome for my use-case (run-once analysis/manipulation of 200GB csv datasets). I found out about Miller[1] a year or so back and have been using that instead. I don't know how it stacks up in terms of performance, but for my money, named arguments and one-shot statistics is all I need. For example analysing the number of people per-year over multiple differently formatted files is as easy as `mlr uniq -f pid,year then count -g year`. It has filtering, very extensive manipulation capabilities and a nice documentation. John has also reacted very fast on feature requests I had (so fast that I've not yet implemented using them). [1]: https://miller.readthedocs.io/en/latest/#overview https://miller.readthedocs.io/en/latest/#overview

Gravityloss 6y ago
This seems very interesting. Making this reproducable and usable for multiple people has always been an annoyance in semi-ad-hoc data collection.
charris000 6y ago
+1 for Miller. It's a really slick tool for exploring large csv datasets. I've typically used it to do some prototyping and exploration of 200-500GB csv datasets before doing more hefty work in Java+PigLatin (our use-case is more long-term than just a single run for analysis, so that's re reason for moving out of just Miller). It's great to get a feel for the data and run some initial analysis before diving into the larger, more cumbersome system and tooling.

mauvehaus 6y ago

Standard awk warning: it's tempting to try to use awk on csv files. You'll even get good results on simple csv files that leave you encouraged to go further. Don't. Csv is not standardized and the quoting rules are weird (and not standardized). If you can live with a certain amount of loss of fidelity in your output, you can get away with using awk. If you want a coarse prototype, use awk. If you need robust, production-grade handling of csv files, use (or write) something else. Csv files are a little bit like like dates: superficially simple, with lots of corner cases. Largely for the same reason: lack of standardization. That said, awk is awesome. It's small enough to fit in your brain, unlike Perl (maybe yours is larger than mine?). It's also pretty universally available, with few massive incompatibilities between versions, unlike shell (provided you avoid the gawk-specific features). I love it.

genuinebyte 6y ago
The best description of dates and CSV files I've heard by far. Bravo. "Csv files are a little bit like like dates: superficially simple, with lots of corner cases. Largely for the same reason: lack of standardization."
- cafard 6y ago
  I just attempted to run some government data through Python using the csv module. It worked like a charm, until it told me that a field had exceeded the maximum allowable length. A close look showed that a field began but did not end with a quotation mark. Using the csv.QUOTE_NONE flag resolved that, but did not remedy such quirks as short or long records. CSV files, depending on who generates them, are a bit like dates if the status of a year as leap or not depended on whether the date of Easter were 0 mod 4.
  - jsdwarf 6y ago
    My gripe with the Python csv.reader module is that it has no skip_lines parameter. Ill-formatted lines at the top/bottom of a CSV file are quite common
bluedino 6y ago
CSV standards are whatever the application implemented. We'd pull a "CSV" out of one system, then have to massage it before uploading to system A and then make other changes so it would be recognized by system B. I'm torn. On one hand, it's easy. It lets people work with the data, albeit using error-prone tools like NotePad++ and Excel. On the other hand, it's just flat out lazy to use.
mehrdadn 6y ago
Tangent, but another interesting problem with date parsing besides lack of standardization is that it's ridiculously context-sensitive. For example, if I schedule an event for Oct 1-Jan 1 right now, the most likely (least surprising) parse would be Oct 1, 2021 through January 1, 2022. Which is both surprising because parsing Oct 1 depends on the current date, and because the parsing of Jan 1 depends on the parse of Oct 1, and because both of these depend on the fact that you're scheduling (in all likelihood) a future event, rather than describing a past event. So whatever date parser you use (no matter how good it is at handling different notations) will return garbage if it doesn't take all these into account.

asicsp 6y ago

Nice introduction (based on both tutorials). One suggestion would be to use -v FPAT='[^,]*|"[^"]+"' instead of BEGIN { FPAT = "[^,]*|\"[^\"]+\"" } >If you get the awk programming language manual…you’ll read it in about two hours and then you’re done. That’s it. You know all of awk. I can't work my head around this quote. That's a ridiculous claim. Even for a experienced programmer, learning a new programming language in 2 weeks, let alone 2 hours would be nothing short of a miracle. I've been using awk for past 2-3 years or so and I wrote a book on GNU awk one-liners earlier this year (https://learnbyexample.github.io/learn_gnuawk/ https://learnbyexample.github.io/learn_gnuawk/). I'm nowhere close to knowing all of awk

samoa42 6y ago
> I've been using awk for past 2-3 years or so and I wrote a book on GNU awk one-liners not to criticise your work, but this does not sit well with me.
- asicsp 6y ago
  Could you elaborate? I didn't get your point.
Athas 6y ago
GNU Awk has lots of extensions and is a relatively big language, but I definitely think you can learn most of the original Awk in two weeks - and you can certainly learn enough to be useful in two hours. It's been a decade now, but my recollection is that when I spent a day or two idly reading the Awk book, and was able to immediately employ it in simple shell pipelines after reading not that many pages. I think Awk is useful once you've learned just a bit about it, and I don't think learning a lot about Awk is very useful - there are languages that are better suited for complex tasks.
- asicsp 6y ago
  Fair enough, someone else made similar point that I had GNU awk in mind wrt all of awk And yeah, it's possible to start using basic field processing and regexp features (if you already know it) in 2 hours and learn most of it in 2 weeks. But, that quote could've been something like you could get started in 2 hours instead of saying one could know all of awk.

rethab 6y ago

I don't agree that you can learn AWK in its entirety in two hours, but I do (strongly) agree that you can teach yourself the basics in two hours and those two hours are extremely well spent considering that AWK is useful for so many things. Surely handling the intricacies of CSV can be a pain, but let's face it: Most tasks are simple and its results are easily verified for correctness. Obviously, I don't write my production webserver with AWK -- I write simple script to parse data (locally), find semantic issues in CSV files, analyze log data, etc.. AWK is great! If you want a slightly bigger cheatsheet, here's mine: https://github.com/rethab/awk-scripts/blob/master/talk.txt https://github.com/rethab/awk-scripts/blob/master/talk.txt

kqr 6y ago
You can also trivially come across data designed for consumption with AWK. Whenever I write a script or utility to dump some data for later analysis, I take into consideration if I can shape the output in a way that will make it easy to work with in AWK. Because I won't know ahead of time what all the interesting values or summaries will be, but if I know I can get it through AWK it will be cheap to extract or compute those values and summaries later, once I know what they are.
disown 6y ago
The biggest problem with awk is that unless you use it regularly, you'll easily forget how to use it. So it's important to keep a running cheatsheet like you did. Nothing more frustrating than forgetting the awk command and searching for it in your history only to find it aged out.

cesaref 6y ago

My favourite bit of gawk (ok, an extended version of awk but it's probably what you are running) is the network support, which in the manual includes writing simple HTTP servers with it. I kid you not.

diegocg 6y ago

FYI, LWN has a great article about the current status of awk, latest features and future developments https://lwn.net/Articles/820829/ https://lwn.net/Articles/820829/

swills 6y ago
Happy to see this mentions https://github.com/onetrueawk/awk https://github.com/onetrueawk/awk

yesenadam 6y ago

> He said, “If you get the awk programming language manual…you’ll read it in about two hours and then you’re done. That’s it. You know all of awk.” It's hyperlinked to the Gawk manual, but it seems likely he actually meant A, W & K's The Awk Programming Language (1988), which you could conceivably read in 2 hours, as it's a joy to read. I used it and The C Programming Language as exemplars of great documentation when writing my own.

elvis70 6y ago
It's one of my favorite technical books, with many examples of the application of the language, including the design of small languages and experimentation with algorithms. It is freely available at archive.org: https://archive.org/details/pdfy-MgN0H1joIoDVoIC7 https://archive.org/details/pdfy-MgN0H1joIoDVoIC7
- maxioatic 6y ago
  Much thanks
statquontrarian 6y ago
There's also the POSIX standard which is quite readable: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/a...
elvis70 6y ago
By the way, "Learning Perl" is also a great introductory book to a language.

xashor 6y ago

Instead of the gawk manual, I recommend the OpenBSD manpage: https://man.openbsd.org/awk.1 https://man.openbsd.org/awk.1 It's concise and well written (as most OpenBSD manpages), so you can learn AWK even in less than 2 hours. Of course it handles only AWK, not the gawk extensions.

nmz 6y ago
mawk also has a nice and short manual. since GNU decided to make its money selling books, their manuals are usually huge and that's by design.

motiejus 6y ago

Fred Hebert (a known name in Erlang ecosystem) created a tutorial "awk in 20 minutes": https://ferd.ca/awk-in-20-minutes.html https://ferd.ca/awk-in-20-minutes.html 20 minutes for that tutorial is enough to write non-trivial awk programs, and, least or most importantly, impress colleagues.

Zhyl 6y ago

I always link to this blog post [0] by Jonathan Palardy. It was one of those eye opening articles for me that really helped me 'grok' it - and this is after having had a couple of stabs at learning it just from reading the text books on it. Awk for me is part of the layers of tools from grep/sed -> AWK -> perl one liner -> full on script. The further down the ladder you go, the more restricted you are, but this means that programs are terser and usually stay readable despite the terseness. I would also recommend Perl One liners [1] which did the same thing for me for Perl one liners and, subsequently, full Perl. [0] https://blog.jpalardy.com/posts/why-learn-awk/ https://blog.jpalardy.com/posts/why-learn-awk/ [1] https://nostarch.com/perloneliners https://nostarch.com/perloneliners

arminiusreturns 6y ago

My story about learning the power of awk was I was working on a dataset and had written a page or so Worth's of Perl... and kept bungling it. I emailed the scientist who wrote the paper whose technique I was emulating and he said, why not use awk, and gave me a basic example one liner, with a bit of modification it worked! Of course all that really means is I suck at Perl, but it was an eye opener for how powerful awk is.

bingerman 6y ago
I learned awk before Perl and it helped a lot. I still use awk (and sed) regularly but for more complex tasks involving regexes and such Perl is my favourite.

js2 6y ago

Here's a basic network calculator I wrote in awk once because I had to do these calculations on a minimal SunOS system that didn't have any other way to do math from the shell. https://gist.github.com/jaysoffian/7bb70e1065f085b46a00 https://gist.github.com/jaysoffian/7bb70e1065f085b46a00 You never know when a bit of awk will save your bacon.

deleted 6y ago

[deleted]

untoxicness 6y ago

Brian Kernighan talked about his love of awk on the Lex Fridman podcast [0]. [0] https://lexfridman.com/brian-kernighan/ https://lexfridman.com/brian-kernighan/

SoSoRoCoCo 6y ago

The O'Reilly book is "Sed & Awk" published in the 90's (https://www.oreilly.com/library/view/sed-awk/1565922255/ https://www.oreilly.com/library/view/sed-awk/1565922255/). The languages work together as a team (like lex and yacc). And the book takes about 2 hours to read. Unix is built on the premise of "lots of simple tools that fit together with pipes". Although this is like 1,000th article on how easy it is to learn awk. Why write yet another one? Awk in 20 Minutes (2015) https://news.ycombinator.com/item?id=23048054 https://news.ycombinator.com/item?id=23048054 Learn just a little Awk (2010) https://news.ycombinator.com/item?id=17322412 https://news.ycombinator.com/item?id=17322412 Learn Awk by Example (2019) https://news.ycombinator.com/item?id=22455779 https://news.ycombinator.com/item?id=22455779 it goes on and on... https://hn.algolia.com/?q=awk https://hn.algolia.com/?q=awk

celticninja 6y ago
The reason to write one is to record what you learn, to further embed it into your brain. Why post it? Whoever posted it found it interesting. I had not seen the others so this is an intro to awk for me. But in general if something has made it to the front page of HN it was interesting enough for enough people to put it there.

Xophmeister 6y ago

I love Awk. The idea of a programming language that effectively facilitates first-class state machines over text streams fits so well with the Unix pipeline model.

ensiferum 6y ago

Awk and shell scripting is the rabbit hole that is best avoided IMHO. Often it's just easier to crank out a "real" program using your lost familiar programming language or at very least python instead of starting to work with shell or awk scripting.

nmz 6y ago
Awk is a full blown programming language though. comparable to python but allowing for much more compact code. Though I completely agree on avoiding shell. the only good shell I've seen is rc. but the relying on external utilities is a double edged sword. forking is too costly over time.

known 6y ago

Before using AWK we need to sanitize the data files;