7 ms·

Let it go, it's 2011. Every time someone uses Awk in this day and age god kills a baby seal.

by moe 15y ago

4ad 15y ago
And why do you think that? Awk is simply the best tool for many tasks.
- moe 15y ago
  Name one?
  - veyron 15y ago
    !x[$1]++ Use that at least 10 times a day Also, Larry Wall (perl) said "I still say awk '{print $1}' a lot."
    - dbbo 15y ago
      Because it's shorter than `perl -lane 'print $F[0]'`.
      - CJefferson 15y ago
        It's certainly easier to remember.. what's the lane?
        telemachos 15y ago
        If you're genuinely curious about -lane, my favorite site for Perl one liners has gone the way of all flesh, but the Wayback Machine can still get you a copy[1]. [1] http://web.archive.org/web/20090602215912/http://sial.org/howto/perl/one-liner/ http://web.archive.org/web/20090602215912/http://sial.org/ho...
        jsrn 15y ago
        "what's the lane?" four options squeezed together: -l: 1. "chomps" the input (which here means: removes newlines (if present) from each line, more general explanation: http://perldoc.perl.org/functions/chomp.html http://perldoc.perl.org/functions/chomp.html 2. and automatically adds a newline to each output newline (see below how to achieve this in a shorter way with a relatively new Perl feature). -a: turns on auto-split-mode: this splits the input (like awk) into the @F (for fields) array. The default separator is one (or many) spaces, i.e. it behaves like AWK. -n: makes Perl process the input in an implicit while loop around the input, which is processed line-wise: "as long as there is another line, process it!" -e: execute the next command line argument as a Perl program (argument in this case beeing 'print $F[0]'). Note that the example can be shortened if you use -E instead of -e. -E enables all extensions and new features of Perl (which aren't enabled with -e because of backwards compatibility). This allows you to use 'say' instead of 'print' which adds a trailing newline automatically and lets you drop the -l option (if you don't need the 'chomp' behaviour explained above): $ perl -anE 'say $F[0]' Of course, the AWK command line is still shorter - and that's expected, because AWK is more specialized than Perl. Still, Perl one liners are often very useful and can do other things better / shorter than AWK - especially if the one-liner uses one of the many libraries that are available for Perl. A thorough reference of all Perl command line options is available at: http://perldoc.perl.org/perlrun.html or just: $ man perlrun
        CJefferson 15y ago
        Thank you, that tempts me to go and look at perl. At the moment I tend to use simple awk, or a full python script. I find python really doesn't lend itself to "one line", or even particularly short, programs however. I keep meaning to go back and look at perl. I was tempted to wait for Perl 6, but I think the time has come to just look at Perl 5 :)
    - burgerbrain 15y ago
      Don't forget about `cut -f1` though! Of course anything more complex than that, but not requiring the complexity of Perl/Ruby is generally best still done with Awk. In my world, that is a lot.
      - a3_nm 15y ago
        cut -f1 and awk '{print $1}' are not equivalent, though: awk does not count empty fields. To be precise: pi:~$ echo 'foo bar' | cut -d ' ' -f 2 pi:~$ echo 'foo bar' | awk '{print $2}' bar
        jsrn 15y ago
        Just for the interested reader: when the first command line is changed into pi:~$ echo 'foo bar' | tr -s ' ' | cut -d ' ' -f 2 it also outputs 'bar'. The -s (for "squeeze") option of tr turns every sequence of the specified character (space in this case) into one instance of this character. Of course, the awk solution is more succint and elegant in this case - I just think that tr -s / cut -d is handy to know from time to time, too.
        a3_nm 15y ago
        The interested reader should not assume hastily, however, that tr -s is enough to make cut behave exactly like awk. Hint: leading spaces. pi:~$ echo ' foo' | tr -s ' ' | cut -d ' ' -f 1 pi:~$ echo ' foo' | awk '{print $1}' foo
  - 4ad 15y ago
    Hacking together something to parse text? I used it to find a house[1]. I needed to rent two apartments close to each other. I wrote some shell/awk scripts that take several real estate web sites as input, parse them, extract all data about apartments including URL, address and price, pipe the addresses into Google maps API, find the exact coordinates of all apartments, calculate distances between them and produce a list of pairs of apartments sorted by the distance between them. It works incredibly well, it routinely finds houses that are on the same street or apartments that are in the same building. The first version of the code was written when I was in Vienna, looking for a house. It took me an evening. The only thing that comes even close to this in power is perl, but I don't like perl, I think it's overkill when you only need to parse text, not do some other computation, and I did all this work under Plan9 which doesn't have perl anyway. Oh, and what was most useful after all this was creating one liners for doing filtering, like finding pairs that had total cost under some value, total number of rooms above some value and distance between them in some interval. I found this one liners to be easier in awk then perl. Your argument against awk because is 2011 is the same as saying let's drop C because it's 2011. Both C and awk solve some things well, for other things there are other languages. [1] http://code.google.com/p/operation-housefinder/ http://code.google.com/p/operation-housefinder/
    - burgerbrain 15y ago
      I find that any argument along the lines of "Oh goodness, it's [YEAR] for crying out loud" is generally rubbish. Not always, but often enough that I've noticed the correlation. I know Perl, I know Ruby, I know Brainfuck and I know Awk. I'm also not a masochist. I don't use Brainfuck but I use the hell out of Awk. Myself and other Awk users like 4ad here aren't using Awk because we're crazy, we've calculated effort/rewards/tradeoffs and come upon a solution. I find people suggesting that others should or should not use various tools to be incredibly condescending. In doing so you are effectively refusing to recognize others as your peers. If you're Dijkstra and everyone around you has a hardon for GOTO, then be my guest, but you are not.
  - telemachos 15y ago
    I find it quicker and easier to create one-off text filters in awk than Perl or Ruby. That is, I may have a set of csv files for teachers, administrators and students by grade. I want to extract fields x, y and z from all these files, transform the data in some specific ways and then create a single output file (maybe to upload somewhere else or to use as part of a larger program). I certainly could write a script in an interpreted language to do this, but I often find it faster to do it in awk. (There's a sweet spot here: if the job is too complex or I'm going to do it over and over, a script may become the better choice.) I use awk for this kind of thing 3-5 times a week, easily.
    - jrockway 15y ago
      By the time you've dealt with all the special cases of the CSV format, your awk script isn't going to look very clean. A few lines of Perl with Text::CSV is going to be much more robust, run faster, and take less time to get working.
      - telemachos 15y ago
        (I know your work, and I respect you. I mean the following as a compliment - truly, no sarcasm or snark.) Sounds to me like you're being a professional programmer. I'm not. I'm a self-taught amateur. I program a little for myself, a little for the school I work for and a bunch to do sysadmin tasks (for myself and the school). I don't worry all that much about the special cases of CSV. I look at the data, massage it a little before if I need to (usually that with awk, sed, etc. as well), then run it through awk and do the job. If the result isn't all perfect, I clean it up quickly by hand using Vim. In this kind of context, 'robust' doesn't mean anything to me. I'm not expecting to do the exact same task ever again. As for running faster, I call bullshit and remind you of this[1]. awk runs plenty fast - seconds or less for such cases. As for less time to get it working, I doubt it. I'm pretty quick with a hackish little awk script. The bottom line for me is that I like awk. It's defaults make sense to me, it's fast, it's very flexible and powerful. Finding the right Perl module or Ruby gem, learning its API, opening my editor, etc. - that all takes time. For small, one-off jobs, it's not worth it. [1] http://news.ycombinator.com/item?id=1116224 http://news.ycombinator.com/item?id=1116224
pestaa 15y ago
Care to elaborate? I'd love to hear alternatives you use for the same kind of task.
- moe 15y ago
  Mostly chains of sort, uniq, sed, tr, comm and friends. Anything that doesn't fit into 80 chars that way is usually a sure sign that it's time to reach for a real scripting language rather than brewing up one of those line-noise blobs. Oh, and of course I don't mind when people use it on their shell-prompt. I do mind when I run into crap like this in a shell-script: awk -F'>' '{ pack[$1]=pack[$1] $2 } END {for (val in pack) print val ">", "(" pack[val] ")"}'
  - 4ad 15y ago
    Crap can be written in any language, no language is imune and no language is better in this regard.
    - moe 15y ago
      Except that in awk the middle-ground between "could be trivially done without awk" and "illegible" is extremely narrow. When I 'grep awk /usr/bin/*' then I see only two types of usages. Most commonly it's used in place of grep, sed or cut for no obvious reason. Gladly less common but all the more annoying is when someone tries to go all "fancy" with it...
      - keenerd 15y ago
        Awk has everything "real" languages have. The same math, flow control and data structures. It also has a subset of string mangling ops that give it the "illegible" appearance. But it is fine as a general purpose dynamic scripting language. As an example: http://kmkeen.com/awk-music/ http://kmkeen.com/awk-music/ It could be clearer without the variable-name golfing and as a seperate .awk file (instead of a CLI one-liner) but it reads pretty much the same as C without any pesky type declarations.
  - 15y ago

Let it go, it's 2011. Every time someone uses Awk in this day and age god kills a baby seal.

by moe 15y ago

Let it go, it's 2011. Every time someone uses Awk in this day and age god kills a baby seal.

4ad 15y ago

And why do you think that? Awk is simply the best tool for many tasks.

moe 15y ago
Name one?
- veyron 15y ago
  !x[$1]++ Use that at least 10 times a day Also, Larry Wall (perl) said "I still say awk '{print $1}' a lot."
  - dbbo 15y ago
    Because it's shorter than `perl -lane 'print $F[0]'`.
    - CJefferson 15y ago
      It's certainly easier to remember.. what's the lane?
      - telemachos 15y ago
        If you're genuinely curious about -lane, my favorite site for Perl one liners has gone the way of all flesh, but the Wayback Machine can still get you a copy[1]. [1] http://web.archive.org/web/20090602215912/http://sial.org/howto/perl/one-liner/ http://web.archive.org/web/20090602215912/http://sial.org/ho...
      - jsrn 15y ago
        "what's the lane?" four options squeezed together: -l: 1. "chomps" the input (which here means: removes newlines (if present) from each line, more general explanation: http://perldoc.perl.org/functions/chomp.html http://perldoc.perl.org/functions/chomp.html 2. and automatically adds a newline to each output newline (see below how to achieve this in a shorter way with a relatively new Perl feature). -a: turns on auto-split-mode: this splits the input (like awk) into the @F (for fields) array. The default separator is one (or many) spaces, i.e. it behaves like AWK. -n: makes Perl process the input in an implicit while loop around the input, which is processed line-wise: "as long as there is another line, process it!" -e: execute the next command line argument as a Perl program (argument in this case beeing 'print $F[0]'). Note that the example can be shortened if you use -E instead of -e. -E enables all extensions and new features of Perl (which aren't enabled with -e because of backwards compatibility). This allows you to use 'say' instead of 'print' which adds a trailing newline automatically and lets you drop the -l option (if you don't need the 'chomp' behaviour explained above): $ perl -anE 'say $F[0]' Of course, the AWK command line is still shorter - and that's expected, because AWK is more specialized than Perl. Still, Perl one liners are often very useful and can do other things better / shorter than AWK - especially if the one-liner uses one of the many libraries that are available for Perl. A thorough reference of all Perl command line options is available at: http://perldoc.perl.org/perlrun.html or just: $ man perlrun
        CJefferson 15y ago
        Thank you, that tempts me to go and look at perl. At the moment I tend to use simple awk, or a full python script. I find python really doesn't lend itself to "one line", or even particularly short, programs however. I keep meaning to go back and look at perl. I was tempted to wait for Perl 6, but I think the time has come to just look at Perl 5 :)
  - burgerbrain 15y ago
    Don't forget about `cut -f1` though! Of course anything more complex than that, but not requiring the complexity of Perl/Ruby is generally best still done with Awk. In my world, that is a lot.
    - a3_nm 15y ago
      cut -f1 and awk '{print $1}' are not equivalent, though: awk does not count empty fields. To be precise: pi:~$ echo 'foo bar' | cut -d ' ' -f 2 pi:~$ echo 'foo bar' | awk '{print $2}' bar
      - jsrn 15y ago
        Just for the interested reader: when the first command line is changed into pi:~$ echo 'foo bar' | tr -s ' ' | cut -d ' ' -f 2 it also outputs 'bar'. The -s (for "squeeze") option of tr turns every sequence of the specified character (space in this case) into one instance of this character. Of course, the awk solution is more succint and elegant in this case - I just think that tr -s / cut -d is handy to know from time to time, too.
        a3_nm 15y ago
        The interested reader should not assume hastily, however, that tr -s is enough to make cut behave exactly like awk. Hint: leading spaces. pi:~$ echo ' foo' | tr -s ' ' | cut -d ' ' -f 1 pi:~$ echo ' foo' | awk '{print $1}' foo
- 4ad 15y ago
  Hacking together something to parse text? I used it to find a house[1]. I needed to rent two apartments close to each other. I wrote some shell/awk scripts that take several real estate web sites as input, parse them, extract all data about apartments including URL, address and price, pipe the addresses into Google maps API, find the exact coordinates of all apartments, calculate distances between them and produce a list of pairs of apartments sorted by the distance between them. It works incredibly well, it routinely finds houses that are on the same street or apartments that are in the same building. The first version of the code was written when I was in Vienna, looking for a house. It took me an evening. The only thing that comes even close to this in power is perl, but I don't like perl, I think it's overkill when you only need to parse text, not do some other computation, and I did all this work under Plan9 which doesn't have perl anyway. Oh, and what was most useful after all this was creating one liners for doing filtering, like finding pairs that had total cost under some value, total number of rooms above some value and distance between them in some interval. I found this one liners to be easier in awk then perl. Your argument against awk because is 2011 is the same as saying let's drop C because it's 2011. Both C and awk solve some things well, for other things there are other languages. [1] http://code.google.com/p/operation-housefinder/ http://code.google.com/p/operation-housefinder/
  - burgerbrain 15y ago
    I find that any argument along the lines of "Oh goodness, it's [YEAR] for crying out loud" is generally rubbish. Not always, but often enough that I've noticed the correlation. I know Perl, I know Ruby, I know Brainfuck and I know Awk. I'm also not a masochist. I don't use Brainfuck but I use the hell out of Awk. Myself and other Awk users like 4ad here aren't using Awk because we're crazy, we've calculated effort/rewards/tradeoffs and come upon a solution. I find people suggesting that others should or should not use various tools to be incredibly condescending. In doing so you are effectively refusing to recognize others as your peers. If you're Dijkstra and everyone around you has a hardon for GOTO, then be my guest, but you are not.
- telemachos 15y ago
  I find it quicker and easier to create one-off text filters in awk than Perl or Ruby. That is, I may have a set of csv files for teachers, administrators and students by grade. I want to extract fields x, y and z from all these files, transform the data in some specific ways and then create a single output file (maybe to upload somewhere else or to use as part of a larger program). I certainly could write a script in an interpreted language to do this, but I often find it faster to do it in awk. (There's a sweet spot here: if the job is too complex or I'm going to do it over and over, a script may become the better choice.) I use awk for this kind of thing 3-5 times a week, easily.
  - jrockway 15y ago
    By the time you've dealt with all the special cases of the CSV format, your awk script isn't going to look very clean. A few lines of Perl with Text::CSV is going to be much more robust, run faster, and take less time to get working.
    - telemachos 15y ago
      (I know your work, and I respect you. I mean the following as a compliment - truly, no sarcasm or snark.) Sounds to me like you're being a professional programmer. I'm not. I'm a self-taught amateur. I program a little for myself, a little for the school I work for and a bunch to do sysadmin tasks (for myself and the school). I don't worry all that much about the special cases of CSV. I look at the data, massage it a little before if I need to (usually that with awk, sed, etc. as well), then run it through awk and do the job. If the result isn't all perfect, I clean it up quickly by hand using Vim. In this kind of context, 'robust' doesn't mean anything to me. I'm not expecting to do the exact same task ever again. As for running faster, I call bullshit and remind you of this[1]. awk runs plenty fast - seconds or less for such cases. As for less time to get it working, I doubt it. I'm pretty quick with a hackish little awk script. The bottom line for me is that I like awk. It's defaults make sense to me, it's fast, it's very flexible and powerful. Finding the right Perl module or Ruby gem, learning its API, opening my editor, etc. - that all takes time. For small, one-off jobs, it's not worth it. [1] http://news.ycombinator.com/item?id=1116224 http://news.ycombinator.com/item?id=1116224

pestaa 15y ago

Care to elaborate? I'd love to hear alternatives you use for the same kind of task.

moe 15y ago
Mostly chains of sort, uniq, sed, tr, comm and friends. Anything that doesn't fit into 80 chars that way is usually a sure sign that it's time to reach for a real scripting language rather than brewing up one of those line-noise blobs. Oh, and of course I don't mind when people use it on their shell-prompt. I do mind when I run into crap like this in a shell-script: awk -F'>' '{ pack[$1]=pack[$1] $2 } END {for (val in pack) print val ">", "(" pack[val] ")"}'
- 4ad 15y ago
  Crap can be written in any language, no language is imune and no language is better in this regard.
  - moe 15y ago
    Except that in awk the middle-ground between "could be trivially done without awk" and "illegible" is extremely narrow. When I 'grep awk /usr/bin/*' then I see only two types of usages. Most commonly it's used in place of grep, sed or cut for no obvious reason. Gladly less common but all the more annoying is when someone tries to go all "fancy" with it...
    - keenerd 15y ago
      Awk has everything "real" languages have. The same math, flow control and data structures. It also has a subset of string mangling ops that give it the "illegible" appearance. But it is fine as a general purpose dynamic scripting language. As an example: http://kmkeen.com/awk-music/ http://kmkeen.com/awk-music/ It could be clearer without the variable-name golfing and as a seperate .awk file (instead of a CLI one-liner) but it reads pretty much the same as C without any pesky type declarations.
- 15y ago