5 ms·

Awk has always been a language that I loved but I have struggled to use besides quick jobs for parsing text files. I understand it is meant to be use for exactl

Awk has always been a language that I loved but I have struggled to use besides quick jobs for parsing text files. I understand it is meant to be use for exactly that, but the fact that is simple, fast and lightweight sometimes makes me want to do something more with it, but when I start trying to do something besides parsing text I find that it starts becoming awkward (pun intented?).

usrbinbash 3y ago
> but the fact that is simple, fast and lightweight I see awk as a DSL to be honest. Yes, it can be used as a general purpose language, but that quickly becomes, as you say, awkward :D Like many DSLs, it is simple, fast and lightweight as long as it is used for it's intended purpose. Once you start using it for something else, these advantages evaporate pretty quickly, because then you have to essentially work around the DSL design to get it to do what you want.
- snitty 3y ago
  DSL == Domain Specific Language?
  - Rediscover 3y ago
    Yes
    - coliveira 3y ago
      One simple thing I do with awk is to create a command processor: read one line at a time and do things on my data as a response. This is very useful because you can make your command as powerful as needed and call other unix tools as a result.
      - rsolva 3y ago
        Do you have an example of this that is available somewhere?
bluetomcat 3y ago
It's a language for creating quick alternative views from line- and column-oriented text streams. That means, take the output of another tool and represent it in a different way.
asicsp 3y ago
I use awk mostly for one-liners and resort to Python when I need more than a few lines of code.
tripflag 3y ago
I have found a handful of unconventional applications for awk -- I once needed a tiny pcm pulsewave generator, and awk was surprisingly decent for the job [1]. Aside from that I've mostly been using it for quick statistics [2], but it quickly moves into perl territory... 1: https://github.com/9001/asm/blob/hovudstraum/etc/bin/beeps#L69-L88 https://github.com/9001/asm/blob/hovudstraum/etc/bin/beeps#L... 2: https://ocv.me/doc/unix/oneliners/#965bfcb8 https://ocv.me/doc/unix/oneliners/#965bfcb8
kqr 3y ago
This is exactly why I moved from AWK to Perl for these quick jobs a couple of years ago. If you stick to an AWK-like subset, Perl is also simple, fast and lightweight. If you want to grow your scripts (and you have a lot of discipline) Perl – in contrast to AWK – gives you enough noose to hang^W^W^W^Wthe tools you need.
- joeythedolphin 3y ago
  Perl? Wow. Is that better than bash, python or even nodejs? Why write in Perl over these? Serious question, was propaghandized to hate Perl.
  - gpvos 3y ago
    Absolutely. It is comparable to python in some ways, but makes it much easier to write quick one-liners using regexes and data manipulation, and to scale those up to real programs. It fills the gap between bash scripts using awk, grep and sed, and C/java/C#. Compared to bash scripting, perl is a real programming language. The documentation and library ecosystem are excellent, backwards compatibility is legendary, yet it supports modern Unicode. The syntax is weird, but try it for a bit, read the man pages, it's not that hard. The OO system is weirder, and I wouldn't make complex class hierarchies in it, but it is usable.
    - marttt 3y ago
      I like how Awk is just a single executable. A single-executable Perl that includes only the core library would be great. There is Microperl [0, 1], but no idea how well it compiles with more up-to-date Perl versions. 0: https://github.com/bentxt/microperl-standalone https://github.com/bentxt/microperl-standalone 1: Original article from 2000 by the author Simon Cozens: https://www.foo.be/docs/tpj/issues/vol5_3/tpj0503-0003.html https://www.foo.be/docs/tpj/issues/vol5_3/tpj0503-0003.html
  - throw0101a 3y ago
    > Perl? Wow. Is that better than bash, python or even nodejs? Why write in Perl over these? It depends on scale. If you have some quick parsing to do, then awk will get you started quickly, but as you expand your experimentation on what you want to extract/manipulate, it may not be easy to add onto the awk beginnings of your "one liner". But if you start with awk-like† syntax but invoking it with Perl, then if you find you have to expand, Perl has more elbow room. The intention is not to 'go big', which those other languages may be better at, but to more easily 'start small'. † IIRC, Larry Wall wanted a utility that had awk/(s)ed-like syntax for text manipulation, just 'with more'.
PhilipRoman 3y ago
I find it pretty nice for writing simple preprocessors. For example I have one which takes anything between two marker lines and pipes it through a command (one invocation per block). Awk has an amazing pipe operator which lets you do something like this: ... { print $0 | "command" } "command" is executed once, and the pipe is kept open until closed explicitly by close("command"), at which point the next invocation will execute it again. The command string itself acts as a key for the pipe file descriptor. And of course, no mention of awk is complete without the "uniq" implementation, which beats the coreutils uniq in every way possible (by supporting arbitrary expressions as keys and not requiring sorted input): !a[$0]++
- aktau 3y ago
  I had no idea about this "keep the pipe open" behaviour. I thought it would spawn the binary on every print statement and thus didn't consider it in the past. But now...

usrbinbash 3y ago

> but the fact that is simple, fast and lightweight I see awk as a DSL to be honest. Yes, it can be used as a general purpose language, but that quickly becomes, as you say, awkward :D Like many DSLs, it is simple, fast and lightweight as long as it is used for it's intended purpose. Once you start using it for something else, these advantages evaporate pretty quickly, because then you have to essentially work around the DSL design to get it to do what you want.

snitty 3y ago
DSL == Domain Specific Language?
- Rediscover 3y ago
  Yes
  - coliveira 3y ago
    One simple thing I do with awk is to create a command processor: read one line at a time and do things on my data as a response. This is very useful because you can make your command as powerful as needed and call other unix tools as a result.
    - rsolva 3y ago
      Do you have an example of this that is available somewhere?

bluetomcat 3y ago

It's a language for creating quick alternative views from line- and column-oriented text streams. That means, take the output of another tool and represent it in a different way.

asicsp 3y ago

I use awk mostly for one-liners and resort to Python when I need more than a few lines of code.

tripflag 3y ago

I have found a handful of unconventional applications for awk -- I once needed a tiny pcm pulsewave generator, and awk was surprisingly decent for the job [1]. Aside from that I've mostly been using it for quick statistics [2], but it quickly moves into perl territory... 1: https://github.com/9001/asm/blob/hovudstraum/etc/bin/beeps#L69-L88 https://github.com/9001/asm/blob/hovudstraum/etc/bin/beeps#L... 2: https://ocv.me/doc/unix/oneliners/#965bfcb8 https://ocv.me/doc/unix/oneliners/#965bfcb8

kqr 3y ago

This is exactly why I moved from AWK to Perl for these quick jobs a couple of years ago. If you stick to an AWK-like subset, Perl is also simple, fast and lightweight. If you want to grow your scripts (and you have a lot of discipline) Perl – in contrast to AWK – gives you enough noose to hang^W^W^W^Wthe tools you need.

joeythedolphin 3y ago
Perl? Wow. Is that better than bash, python or even nodejs? Why write in Perl over these? Serious question, was propaghandized to hate Perl.
- gpvos 3y ago
  Absolutely. It is comparable to python in some ways, but makes it much easier to write quick one-liners using regexes and data manipulation, and to scale those up to real programs. It fills the gap between bash scripts using awk, grep and sed, and C/java/C#. Compared to bash scripting, perl is a real programming language. The documentation and library ecosystem are excellent, backwards compatibility is legendary, yet it supports modern Unicode. The syntax is weird, but try it for a bit, read the man pages, it's not that hard. The OO system is weirder, and I wouldn't make complex class hierarchies in it, but it is usable.
  - marttt 3y ago
    I like how Awk is just a single executable. A single-executable Perl that includes only the core library would be great. There is Microperl [0, 1], but no idea how well it compiles with more up-to-date Perl versions. 0: https://github.com/bentxt/microperl-standalone https://github.com/bentxt/microperl-standalone 1: Original article from 2000 by the author Simon Cozens: https://www.foo.be/docs/tpj/issues/vol5_3/tpj0503-0003.html https://www.foo.be/docs/tpj/issues/vol5_3/tpj0503-0003.html
- throw0101a 3y ago
  > Perl? Wow. Is that better than bash, python or even nodejs? Why write in Perl over these? It depends on scale. If you have some quick parsing to do, then awk will get you started quickly, but as you expand your experimentation on what you want to extract/manipulate, it may not be easy to add onto the awk beginnings of your "one liner". But if you start with awk-like† syntax but invoking it with Perl, then if you find you have to expand, Perl has more elbow room. The intention is not to 'go big', which those other languages may be better at, but to more easily 'start small'. † IIRC, Larry Wall wanted a utility that had awk/(s)ed-like syntax for text manipulation, just 'with more'.

PhilipRoman 3y ago

I find it pretty nice for writing simple preprocessors. For example I have one which takes anything between two marker lines and pipes it through a command (one invocation per block). Awk has an amazing pipe operator which lets you do something like this: ... { print $0 | "command" } "command" is executed once, and the pipe is kept open until closed explicitly by close("command"), at which point the next invocation will execute it again. The command string itself acts as a key for the pipe file descriptor. And of course, no mention of awk is complete without the "uniq" implementation, which beats the coreutils uniq in every way possible (by supporting arbitrary expressions as keys and not requiring sorted input): !a[$0]++

aktau 3y ago
I had no idea about this "keep the pipe open" behaviour. I thought it would spawn the binary on every print statement and thus didn't consider it in the past. But now...