4 ms·
Awk works great in pipes on the commandline, unlike Python where you would be writing a multiline script to accomplish the same task. My 99% use case is extrac
by jandrese 8y ago
Awk works great in pipes on the commandline, unlike Python where you would be writing a multiline script to accomplish the same task.
My 99% use case is extracting particular fields from some input stream.
awk '{print $1,$5}'
That's a tremendously common problem and awk is the best tool for the job.
I once started diving deeper into awk and there is an impressive amount you can do with it, but I hit a point pretty quickly where it would be cleaner to write the script in a more traditional language where my successor won't be cursing my name for writing something complex in an obscure language.
- Spivak 8y agoIf you just want to extract fields the cut utility might be a better fit. cut -d $delim -f $field1,$field2,...
- mmt 8y agoI'll tend to do this in scripts, both for clarity and precision (i.e. when I know the specific, single delimiter). However, for ad-hoc command-line usage, I still prefer awk, because (1) the default delimiter is white space, which means I don't have to worry about if what I'm looking at might contain tabs intermingled with spaces (or how many spaces) and (2) it makes it much easier to incrementally increase the complexity of my ad-hoc operation, such as if I decide I want to take a sum or average of one of those columns, after all.
- kjeetgill 8y agoI first learned awk precisely to replace cut. Not being able to reorder fields make it an especially narrow tool. Is there anything from cut I'm missing out on?
- mmt 8y ago> Is there anything from cut I'm missing out on? It's a narrow tool. Other than the inherent advantage this brings in clarity/simplicity in a script, there's nothing else to it.
- kazinator 8y agoScripts in which you have both cut and awk are not simpler or clearer than ones in which you only use awk.
- mmt 8y agoAlthough I completely agree with this for any given "one-liner" or pipeline (even if it's multiple lines), I don't believe the mere use of awk in one place in the script means using cut elsewhere isn't simpler/clearer there. Consider a script that originally had now awk but just a field=$(somecmd | cut -d: -f3) but now has been modified to have awk elsewhere. Did the above just become less simple or clear? If so, is it worth changing it to the below for clarity? Does it matter if the awk predates the cut, instead? field=$(somecmd | awk -F: '{print $2}') I say "no" to both, although I do recognize the argument for just using the same, consistent tool everywhere, instead.
- jandrese 8y agoThe -w option in cut isn't universal which makes it unreliable for the extremely common use case of fields separated by arbitrary whitespace.
- bcbrown 8y agoI actually wrote a sed utility to collapse arbitrary whitespace before piping into cut before I learned about Awk. I never knew about -w before, but looks like neither of the systems I'm currently using for work support it.
- vram22 8y ago>I actually wrote a sed utility to collapse arbitrary whitespace Can be done with tr, without writing a utility (if by "collapse" you mean what I think you do): This file t: $ cat t the quick brown fox jumped over the lazy dog containing many combinations of spaces, tabs and newlines (whitespace) can be changed to this file t2: $ cat t2 the quick brown fox jumped over the lazy dog by this tr command: tr -cs "[a-zA-Z]" "\012" < t > t2 That also makes the output more amenable to further processing, including common tasks like finding the frequencies of the words in the input, as mentioned in the "More shell, less egg" post mentioned in this post: The Bentley-Knuth problem and solutions: https://jugad2.blogspot.com/2012/07/the-bentley-knuth-problem-and-solutions.html https://jugad2.blogspot.com/2012/07/the-bentley-knuth-proble...
- EvilTerran 8y agoYeah, I usually start with `cut` - it makes the easy cases very easy - then only reach for awk if it turns out I need more power. Thinking of recent cases from my own experience, that tends to be needs like: - outputting my selected fields in an order other than their order in the input lines: you can't `cut -f2,1`, for example. - use a multi-character or regex field separator: -F'<space><star><comma><space><star>' [edit: rewritten in words to dodge HN's formatting] is a real winner when dealing with hand-written CSV. - use a record separator other than \n: thanks, Windows, for all those times I've written ORS="\r\n" :P - you can even use a regex record separator, which not even perl supports; I don't need that often, but it's a godsend when I do. - select based on criteria other than "number of fields from the start": I recently needed the last field from lines of varying field-count, and awk '{print $NF}' was just the thing. But if you don't need anything like that, cut is a lot more succinct - especially if you need to split on tabs only, which it does by default, while awk defaults to splitting on spaces too.
- dredmorbius 8y agoIt's often easy to start with tools such as cut, sed, grep, tr, etc., but as you start building a one-liner (or short script) you realise that what you've cobbled together could all be expressed in awk. You can shell out if necessary: cmd = "whois " $url while ( cmd | getline ) { # stuff } close(cmd) Also: coprocesses.
- kakwa_ 8y agoIt's also use awk in that manner 99% of the time. Another common usage for awk in my case is to do floating operations (*, +, eq, gt, etc) in a shell script. It avoids installing bc, which is not always present by default. I used it for other things, like for example comparing software versions in shell scripts. The most advanced usage of awk in my case was to do some statistic analysis (average, confidence interval, standard deviation) of network simulations.
- lozf 8y agoJust be aware there are some caveats when using awk to operate on large numbers. GNU awk has the -M option to handle precision arithmetic, but where you can't use `gawk`, `bc` may be safer afterall.
- hawski 8y agoI like awk very much, but cut has one big advantage over awk - you can easily output range of fields also open ended ones. Example: $ cat foo foo bar baz foo qux $ < foo cut -d' ' -f 2- bar baz qux As far as I know you have to loop through the fields, which is (haha) awkward.