9 ms·
The new namespaces look interesting, although I really don't think the lack of namespaces it what has hindered Awk from widespread use. I am speculating of cou
by freedomben 7y ago
The new namespaces look interesting, although I really don't think the lack of namespaces it what has hindered Awk from widespread use.
I am speculating of course, but as someone who evangelizes Awk the most common thing I hear from people as to why they don't use it, is that they just don't know the language all that well. For people that don't know much about Awk, it looks really complex and esoteric.
To address the Awk ignorance, I put together a talk for Linux Fest Northwest last year, and it was so popular that I gave it again (virtually due to COVID-19) this year. The conference was fully remote so the talk is on Youtube.
If you've ever wanted to learn Awk, this will take you from zero to proficient. There are exercises as well for practice:
* Presentation: https://youtu.be/43BNFcOdBlY https://youtu.be/43BNFcOdBlY
* Exercises:
- Source: https://github.com/FreedomBen/awk-hack-the-planet https://github.com/FreedomBen/awk-hack-the-planet
- My solutions with explanation: https://youtu.be/4UGLsRYDfo8 https://youtu.be/4UGLsRYDfo8
- benhoyt 7y ago> although I really don't think the lack of namespaces it what has hindered Awk from widespread use I agree (I side more with Kernighan than Robbins here). My take on what hindered it from larger-scale use: lack of a big standard library (Kernighan more or less says this) -- Python and Go's stdlibs really helped them for example, and some significant quirks like lack of proper local variables (you can get them with extra arguments, but it's weird). Just skimmed your AWK-learning talk -- very good stuff. I'm relatively proficient, but keen to try your exercises anyway!
- 7thaccount 7y agoAlso agreed. I want the built-in Python data structures like dictionaries without getting too much Python DNA.
- davidgould 7y agoAwk arrays are dictionaries, what are you missing?
- 7thaccount 7y agoNever knew it lol, so thanks for pointing it out. It still stands that other functionality is missing.
- anitil 7y agoI love AWK with a passion. It's just perfectly fills the gap between grep, sed and 'I better fire up a text editor'. I guess 'perl -pie' used to sit in that niche, but I've never used it much.
- theamk 7y agoIs there any reason to prefer it over `perl -pie` ? I found out that at the base level, they are pretty comparable; and at advanced level, CPAN (or perl- packages in distro's repo) have saved me more than once. (Some embedded systems often do not have perl, but if they are that limited, they might have a very stripped down version of awk too)
- freedomben 7y agoIf you already know perl -pie well, then it may not be worth it to switch to Awk unless maybe you do a lot of embedded development. Awk is part of posix so is pre-installed and available nearly everywhere, even on many embedded devices. Busybox for example includes an implementation of Awl. Even my super cheap home router had an awk interpreter on it. Perl is also decreasingly commonly installed. On the latest Fedora for example I had to install it explicitly. Awk is still installed by default.
- duskwuff 7y agoNot really. For many years, perl even shipped with a tool ("a2p" [1]) which would automatically translate awk syntax to perl -- there's very little distance between the two, syntactically speaking. [1]: now available as https://metacpan.org/pod/App::a2p https://metacpan.org/pod/App::a2p
- downerending 7y agoOne likely unpopular answer is that learning to use perl-pie well is more likely to lead one to create large perl programs, vs awk. Many of us feel those programs would be better written in Python.
- throwaway_pdp09 7y agoI bought an awk book and read it. Didn't see the point of the language any more. I have regexps if I need them, python if I need more. I rarely do much textual data munging anyway. The exercises look bizarre - "Questions to answer about our payroll data using awk to analyze" - doesn't make sense. That's what DBs are for, or similar tools. It's just an exercise but still. Final coffin nail is that having awk on your CV is useless, because everyone wants full-stack cassandra, spark and hadoop, and other stupid things. Awk won't help get you hired (try to find it in https://news.ycombinator.com/item?id=23042618 https://news.ycombinator.com/item?id=23042618)
- bambataa 7y agoYes, the only time I come across Awk now is as part of some shell pipeline. It just seems to have been crowded out of lots of places. I still think it’s cool though.
- Mediterraneo10 7y agoOften old *nix tools are disparaged here by professional software developers. But not everyone is working in software development professionally, and not everyone needs sexy qualifications in other tools. As an ordinary person whose day job is in the humanities, learning awk, sed, Emacs Lisp and other such tools has allowed me to quickly solve tasks that would have taken me ages to do manually. These tools have docs that are very accessible to the ordinary computer user (after all, they were conceived as something that clerical staff could be trained in). I am happy these tools are still around and I think they should be more widely known.
- jimbokun 7y ago> I rarely do much textual data munging anyway. Well, since that is pretty much AWK's entire reason for being, of course you don't have a use for it. > That's what DBs are for, or similar tools. But if you have a text file with a lot of semi-structured data and just need to answer a few questions, using AWK will almost always be faster than designing a schema and importing into a database first, before answering the question. Oh, and want to know a really good tool for getting data from a text file into a database? AWK! > Awk won't help get you hired AWK just lets you accomplish many tasks quickly and without fuss that can create value for your company, and those are things that can go on your resume. At some point in your career, pointing to specific things you have accomplished becomes more important than the keywords on your resume.
- clan 7y agoIt is a very nice presentation! I did however see a lot of screen tearing. It looks like a partial flash of a terminal window. I see[1]: Title: ben@bensystem76 Dockerfile ngingx-help.txt test.pem Desktop part2 drew.txt .... I is only periodically but seems to be the same terminal tearing in[1]. I saw some in the beginning but now at 22:20 I am so annoyed that I come here to complain! ;-) I checked both 720p60 and 1080p60 - I just pause at 22:20 - but plenty before that as well. Maybe consider re-encode/re-upload if possible. If not: Epileptics beware! [1] https://i.postimg.cc/6Q2RvbCn/AWK-screen-tearing-in-Youtube.png https://i.postimg.cc/6Q2RvbCn/AWK-screen-tearing-in-Youtube....
- clan 7y agoAaaaand 1 minute later in the video you acknowledge the problem yourself. Nothing to see here. Move along...
- freedomben 7y agoHaha yeah, so sorry. My OCD flares when I see that :why:
- BiteCode_dev 7y agoThe reason AWK is not widespread is that it's only great for one case: working on a stream of lines. It's a pretty narrow niche. Sure you can code anything with it, but there is always another language that will be better at this other thing, and still decent at what awk does. Awk looks more like a DSL than a generalist language. So why learn awk to save you a few minutes for the rare cases you do need it? Just learn Python/Ruby/etc, it's a better investment, and it has a better crossplateform story.
- jimbokun 7y ago> It's a pretty narrow niche. If you're working on any kind of Unix, it's a niche that you encounter almost constantly. > Awk looks more like a DSL than a generalist language. Yes, it is pretty much the ultimate text processing DSL. So if you need to process a text file a line at a time, in most cases AWK is the optimal solution. > and it has a better crossplateform story. I think AWK is available on almost every platform where you can install those languages?
- srean 7y agoThe reason AWK is so awesome is that it's only great for one case: working on a stream of lines
- blodorn 7y agoRegarding AWK being cross-platform, it is part of the POSIX standard. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/a...
- lottin 7y agoThe problem I think is that most people only use awk occasionally and that makes the skills hard to retain due to lack of practice.
- dhimes 7y agoI agree. And since you can have php (or whatever) on the command line, you really have to ask yourself "why should I spend 45 minutes or more going through old scripts I wrote to re-learn how to do what I want to do?" I love playing with awk. I think it's really cool. But there's a cost to it.
- Crontab 7y agoThanks for posting this. I'll give it a watch.
- zubspace 7y agoVery eye opening presentation and good exercises in the second video! I had no understanding of awk whatsoever and can now fully grok the following command, which returns the number of different cities in a structured file with a header line, addresses and the city in column 5: awk 'NR != 1 { print $5 }' payroll.tsv | sort | uniq | wc -l This is beautiful.
- davidgould 7y agoAlmost. It doesn't handle multi word cities, eg "San Franciso", and it spawns extra processes. Try: awk -F "\t" 'NR != 1 { if (!cities[$5]++) n++ } ; END { print n }' payroll.tsv
- deleted 7y ago[deleted]