6 ms·

Knowing bash, sed (single-line), grep quite well, I still regularly run into problems (mostly multiline complex regex substitutions I guess) where I could use a

by jkbyc 12y ago

Knowing bash, sed (single-line), grep quite well, I still regularly run into problems (mostly multiline complex regex substitutions I guess) where I could use a more expressive or more efficient tool. I am thinking that I should perhaps invest in learning AWK. But then, why not go all the way and learn Perl? What are the advantages and disadvantages?

cstross 12y ago
If you've got bash, sed, grep, and add awk, then you are 80-90% of the way to Perl 4 proficiency -- that's basic Perl circa 1990, adequate for short scripts (and more efficient that bash/sed/awk scripts). Perl started life as a superset of awk, with sed, grep, and some weird extras (streams) thrown in. Indeed, the perl distro ships with a2p, a program for automatically converting awk scripts into perl scripts. Perl 5 adds variable scoping, pointers, modules, OOP and a bunch of powerful expressive stuff and abstractions -- but if you learn awk on top of what you've already got it means you're about 90% of the way to introductory Perl proficiency, and you can add the advanced stuff later. So by all means learn awk! Just remember that it's not suitable for really big jobs -- if you need to create awk scripts that go beyond a couple of dozen lines of code, that's when you'll probably want to upgrade to Perl.
- dredmorbius 12y ago
  Awk and sysadmin tips from Charlie Stross. Life is good.
- jkbyc 12y ago
  I didn't know that awk and Perl are so closely related, thanks!
  - emmelaich 12y ago
    Perl got it's inspiration from many languages. See Wikipedia or the original Perl book. Including BASIC, but that's not mentioned in Wikipedia.
- microtherion 12y ago
  While I agree that awk and Perl 4 have roughly comparable powers, I would not call Perl "a superset of awk" (At least not Perl 3/4; I'm not familiar with the syntax of earlier versions). Where awk really shines is situations that call for a set of record based production rules, and if in fact your problem is in that domain, you could write hundreds or even thousands of rules without the problem becoming unsuitable for awk. Like for any domain specific language, you could consider the limitations imposed by awk as valuable discipline to keep you in the problem domain. While perl can be written in an awk-like style, it's really much more of a general purpose language, so it can accommodate workflows that deviate substantially from the "sequentially process records" style that awk excels at.
  - emmelaich 12y ago
    The autosplit and loop options to Perl make it very suitable for an awk replacemnt. It even has BEGIN and END pseudo-patterns. Once you've tried Perl you don't want to flip back and forth to awk because the inconsistencies in syntax (within and to each other) will drive you mad.
    - cstross 12y ago
      However, IIRC BEGIN and END showed up in Perl 5.003 or thereabouts, circa 1995-97.
- emmelaich 12y ago
  What you said, but really once you've started Perl there's no reason to go back to sed or awk. Any program you ever want to distribute has to have some decent error handling and option and arg parsing and that precludes them both. Even when I think in sed or awk, I'll generally write in Perl; or, if I have an old awk script I'll use a2p to convert it. a2p (awk to perl), s2p (sed to perl) and find2perl (find to perl) are all excellent ways to get you into Perl. Nowadays my preferred language for almost anything is Python, but I still find myself writing Perl for some throwaways.
username42 12y ago
I think there is no use to master bash, sed and awk. When you have a task that becomes tricky (beyond entry level) in sed, it is often easy in awk. When you have a task that becomes tricky in bash or awk, switch to a more powerful scripting language like perl/python/ruby/?. I love perl, but I feel it is getting outdated. I try to improve my programming speed in python but I am still far from my speed in perl.

by jkbyc 12y ago

cstross 12y ago

If you've got bash, sed, grep, and add awk, then you are 80-90% of the way to Perl 4 proficiency -- that's basic Perl circa 1990, adequate for short scripts (and more efficient that bash/sed/awk scripts). Perl started life as a superset of awk, with sed, grep, and some weird extras (streams) thrown in. Indeed, the perl distro ships with a2p, a program for automatically converting awk scripts into perl scripts. Perl 5 adds variable scoping, pointers, modules, OOP and a bunch of powerful expressive stuff and abstractions -- but if you learn awk on top of what you've already got it means you're about 90% of the way to introductory Perl proficiency, and you can add the advanced stuff later. So by all means learn awk! Just remember that it's not suitable for really big jobs -- if you need to create awk scripts that go beyond a couple of dozen lines of code, that's when you'll probably want to upgrade to Perl.

dredmorbius 12y ago
Awk and sysadmin tips from Charlie Stross. Life is good.
jkbyc 12y ago
I didn't know that awk and Perl are so closely related, thanks!
- emmelaich 12y ago
  Perl got it's inspiration from many languages. See Wikipedia or the original Perl book. Including BASIC, but that's not mentioned in Wikipedia.
microtherion 12y ago
While I agree that awk and Perl 4 have roughly comparable powers, I would not call Perl "a superset of awk" (At least not Perl 3/4; I'm not familiar with the syntax of earlier versions). Where awk really shines is situations that call for a set of record based production rules, and if in fact your problem is in that domain, you could write hundreds or even thousands of rules without the problem becoming unsuitable for awk. Like for any domain specific language, you could consider the limitations imposed by awk as valuable discipline to keep you in the problem domain. While perl can be written in an awk-like style, it's really much more of a general purpose language, so it can accommodate workflows that deviate substantially from the "sequentially process records" style that awk excels at.
- emmelaich 12y ago
  The autosplit and loop options to Perl make it very suitable for an awk replacemnt. It even has BEGIN and END pseudo-patterns. Once you've tried Perl you don't want to flip back and forth to awk because the inconsistencies in syntax (within and to each other) will drive you mad.
  - cstross 12y ago
    However, IIRC BEGIN and END showed up in Perl 5.003 or thereabouts, circa 1995-97.
emmelaich 12y ago
What you said, but really once you've started Perl there's no reason to go back to sed or awk. Any program you ever want to distribute has to have some decent error handling and option and arg parsing and that precludes them both. Even when I think in sed or awk, I'll generally write in Perl; or, if I have an old awk script I'll use a2p to convert it. a2p (awk to perl), s2p (sed to perl) and find2perl (find to perl) are all excellent ways to get you into Perl. Nowadays my preferred language for almost anything is Python, but I still find myself writing Perl for some throwaways.

username42 12y ago

I think there is no use to master bash, sed and awk. When you have a task that becomes tricky (beyond entry level) in sed, it is often easy in awk. When you have a task that becomes tricky in bash or awk, switch to a more powerful scripting language like perl/python/ruby/?. I love perl, but I feel it is getting outdated. I try to improve my programming speed in python but I am still far from my speed in perl.