5 ms·

I noticed that I became so much more quick after taking 1 hour to properly learn awk. Yes, it literally takes about 1 hour.

by j0hnyl 4y ago

ketanmaheshwari 4y ago
Awk is awesome but saying it literally takes 1 hour to properly learn it is a bit overselling.
- j0hnyl 4y ago
  I really don't think so! If you have experience with any scripting, you can fully grok the fundamentals of awk in 1 hour. You might not memorize all the nuances, but you can establish the fundamentals to a degree that most things you would try to achieve would take just a few minutes of brushing up. For those that haven't taken the time yet, I think this is a good place to start: https://learnxinyminutes.com/docs/awk/ https://learnxinyminutes.com/docs/awk/ Of course, some people do very advanced things in awk and I absolutely agree that 1 hour of study isn't going to make you a ninja, but it's absolutely enough to learn the awk programming paradigm so that when the need arises you can quickly mobilize the solution you need. For example: If you're quick to the draw, it can take less time to write an awk one liner to calculate the average of a column in a csv than it does to copy the csv into excel and highlight the column. It's a massive productivity booster.
  - 2devnull 4y ago
    I feel like I do this every three years then proceed to never use it. Then I read a post on hn and think about how great it could be; rinse and repeat
    - boredtofears 4y ago
      yeah thats exactly right. it may only take an hour to learn, but every time i need to use awk it seems like i have to spend an hour to re-learn its goofy syntax.
      - tejtm 4y ago
        alas this is true, I never correctly recall the order of particular function args as they are fairly random, still beats the alternative of having to continually internalize entire fragile ecosystems to achieve the same goal.
        boredtofears 4y ago
        yeah you're definitely right. im sure if it was something i had to use more consistently i'd be able to commit it to memory. maybe...
  - chasil 4y ago
    Brian Kernighan covers the entire [new] awk language in 40 pages - chapter 2. There are people who have asked me scripting questions for over a decade, who will not read this for some reason. It could be read in an hour, but not fully retained. https://archive.org/download/pdfy-MgN0H1joIoDVoIC7/ https://archive.org/download/pdfy-MgN0H1joIoDVoIC7/
- foobarian 4y ago
  All you need to do is learn that cmd | awk '{ $5 }' will print out the 5th word as delimited by one or more whitespace characters. Regexes support this easily but are cumbersome to write on the command line.
  - bravetraveler 4y ago
    Doing that, maybe with some inline concatenation to make a new structure, and this are about all I use: Printing based on another field, example gets UIDs >= 1000: awk -F: '$3 >= 1000 {print $0}' /etc/passwd It can do plenty of magic, but knowing how to pull fields, concat them together, and select based on them cover like 99% of the things I hope to do with it
  - every 4y ago
    And don't forget the invisible $0 field in awk...
    - fluential 4y ago
      And $NF
- deleted 4y ago
  [deleted]
- metadat 4y ago
  Awk is an amazingly effective tool for getting things done quickly. Submitted yesterday: Learn to use Awk with hundreds of examples https://github.com/learnbyexample/Command-line-text-processing/blob/master/gnu_awk.md https://github.com/learnbyexample/Command-line-text-processi... https://news.ycombinator.com/item?id=33349930 https://news.ycombinator.com/item?id=33349930
- mmh0000 4y ago
  What? The awk manual is only 827 highly technical pages[1]. If you can't read and interalize that in an hour, I suspect you're a much worse programmer than the OP. [1] https://www.gnu.org/software/gawk/manual/gawk.html https://www.gnu.org/software/gawk/manual/gawk.html For the sarcasm impared among us, everything above this, but possibly including this sentence is sarcasm.
  - sargstuff 4y ago
    Think the more relevant script equivalent of 'Everything in this statement is false.' is 'All output must return false to have true side effects.' The quick one ~ true ~ fix was ! or #! without the 1024k copyright. s-expression notation avoids the issue with (."contents") MS Windows interpretation is much more terse & colorful.
devonkim 4y ago
It takes a lot less time to learn to be fairly productive with awk than with, say, vi / vim. Over time I've realized that gluing these text manipulation tools together is only an intermediate step toward learning how to write and use them in a manner that is maintainable across several generations of engineers as well as portable across many different environments, and that's still a mostly unsolved problem IMO for not just shell scripts but programming languages in general. For example, the same shell script that does something as seemingly simple as performing a sha256 checksum on macOS won't work on most Linux distributions. So in the end one winds up writing a lot of utilities all over again in yet another language for the sake of portability which ironically hurts maintainability and readability for sure because it's simply more code that can rot.
qwertywert_ 4y ago
The only thing I use AWK for is getting at columns from output, (possibly processing or conditionally doing something on each) what would be the next big use-case?
- 5e92cb50239222b 4y ago
  I use it frequently to calculate some basic statistics on log file data. Here's a nice example of something similar: https://drewdevault.com/dynlib https://drewdevault.com/dynlib
- yls 4y ago
  I have used it to extract a table to restore from a MySQL database dump.
- sargstuff 4y ago
  awk autonoma theory and oop the results. add unicode for extra tics! Scripted cholmskey grammar ( https://en.wikipedia.org/wiki/Universal_grammar https://en.wikipedia.org/wiki/Universal_grammar ) to unleash the power of regular expressions.

I noticed that I became so much more quick after taking 1 hour to properly learn awk. Yes, it literally takes about 1 hour.

by j0hnyl 4y ago

I noticed that I became so much more quick after taking 1 hour to properly learn awk. Yes, it literally takes about 1 hour.

ketanmaheshwari 4y ago

Awk is awesome but saying it literally takes 1 hour to properly learn it is a bit overselling.

j0hnyl 4y ago
I really don't think so! If you have experience with any scripting, you can fully grok the fundamentals of awk in 1 hour. You might not memorize all the nuances, but you can establish the fundamentals to a degree that most things you would try to achieve would take just a few minutes of brushing up. For those that haven't taken the time yet, I think this is a good place to start: https://learnxinyminutes.com/docs/awk/ https://learnxinyminutes.com/docs/awk/ Of course, some people do very advanced things in awk and I absolutely agree that 1 hour of study isn't going to make you a ninja, but it's absolutely enough to learn the awk programming paradigm so that when the need arises you can quickly mobilize the solution you need. For example: If you're quick to the draw, it can take less time to write an awk one liner to calculate the average of a column in a csv than it does to copy the csv into excel and highlight the column. It's a massive productivity booster.
- 2devnull 4y ago
  I feel like I do this every three years then proceed to never use it. Then I read a post on hn and think about how great it could be; rinse and repeat
  - boredtofears 4y ago
    yeah thats exactly right. it may only take an hour to learn, but every time i need to use awk it seems like i have to spend an hour to re-learn its goofy syntax.
    - tejtm 4y ago
      alas this is true, I never correctly recall the order of particular function args as they are fairly random, still beats the alternative of having to continually internalize entire fragile ecosystems to achieve the same goal.
      - boredtofears 4y ago
        yeah you're definitely right. im sure if it was something i had to use more consistently i'd be able to commit it to memory. maybe...
- chasil 4y ago
  Brian Kernighan covers the entire [new] awk language in 40 pages - chapter 2. There are people who have asked me scripting questions for over a decade, who will not read this for some reason. It could be read in an hour, but not fully retained. https://archive.org/download/pdfy-MgN0H1joIoDVoIC7/ https://archive.org/download/pdfy-MgN0H1joIoDVoIC7/
foobarian 4y ago
All you need to do is learn that cmd | awk '{ $5 }' will print out the 5th word as delimited by one or more whitespace characters. Regexes support this easily but are cumbersome to write on the command line.
- bravetraveler 4y ago
  Doing that, maybe with some inline concatenation to make a new structure, and this are about all I use: Printing based on another field, example gets UIDs >= 1000: awk -F: '$3 >= 1000 {print $0}' /etc/passwd It can do plenty of magic, but knowing how to pull fields, concat them together, and select based on them cover like 99% of the things I hope to do with it
- every 4y ago
  And don't forget the invisible $0 field in awk...
  - fluential 4y ago
    And $NF
deleted 4y ago
[deleted]
metadat 4y ago
Awk is an amazingly effective tool for getting things done quickly. Submitted yesterday: Learn to use Awk with hundreds of examples https://github.com/learnbyexample/Command-line-text-processing/blob/master/gnu_awk.md https://github.com/learnbyexample/Command-line-text-processi... https://news.ycombinator.com/item?id=33349930 https://news.ycombinator.com/item?id=33349930
mmh0000 4y ago
What? The awk manual is only 827 highly technical pages[1]. If you can't read and interalize that in an hour, I suspect you're a much worse programmer than the OP. [1] https://www.gnu.org/software/gawk/manual/gawk.html https://www.gnu.org/software/gawk/manual/gawk.html For the sarcasm impared among us, everything above this, but possibly including this sentence is sarcasm.
- sargstuff 4y ago
  Think the more relevant script equivalent of 'Everything in this statement is false.' is 'All output must return false to have true side effects.' The quick one ~ true ~ fix was ! or #! without the 1024k copyright. s-expression notation avoids the issue with (."contents") MS Windows interpretation is much more terse & colorful.

devonkim 4y ago

It takes a lot less time to learn to be fairly productive with awk than with, say, vi / vim. Over time I've realized that gluing these text manipulation tools together is only an intermediate step toward learning how to write and use them in a manner that is maintainable across several generations of engineers as well as portable across many different environments, and that's still a mostly unsolved problem IMO for not just shell scripts but programming languages in general. For example, the same shell script that does something as seemingly simple as performing a sha256 checksum on macOS won't work on most Linux distributions. So in the end one winds up writing a lot of utilities all over again in yet another language for the sake of portability which ironically hurts maintainability and readability for sure because it's simply more code that can rot.

qwertywert_ 4y ago

The only thing I use AWK for is getting at columns from output, (possibly processing or conditionally doing something on each) what would be the next big use-case?

5e92cb50239222b 4y ago
I use it frequently to calculate some basic statistics on log file data. Here's a nice example of something similar: https://drewdevault.com/dynlib https://drewdevault.com/dynlib
yls 4y ago
I have used it to extract a table to restore from a MySQL database dump.
sargstuff 4y ago
awk autonoma theory and oop the results. add unicode for extra tics! Scripted cholmskey grammar ( https://en.wikipedia.org/wiki/Universal_grammar https://en.wikipedia.org/wiki/Universal_grammar ) to unleash the power of regular expressions.