4 ms·

What is your opinion on using other languages to augment bash? Awk, perl etc.

by antris 8y ago

freedomben 8y ago
I personally consider awk, sed, grep, cut, etc. to be "part" of bash. I don't think I ever write a script without invoking those venerable tools at least once. I often have many pipes, such as some_command \ | grep -E 'some.*regex$' \ | sed -e 's/erase_text//g' \ | sed -e 's/erase_more//g' \ | awk '{ print $2 }' \ | cut -d ':' -f 1 I used to reach for Perl one-liners a lot, and still do sometimes when I need a gross regex that `sed -E` can't handle (See Perl Pie[1]), but Ruby has been making an appearance more and more often for handy one-liners. Roblog has a great post on using Ruby [2]. If the script is complex enough and I'm willing to take a dependency on Ruby, that is usually where I turn. Where that line is does change from time to time. Ruby makes it super trivial to call shell commands and process them easily as well, which makes it easy to integrate into existing scripting [3]. Hopefully that answered your question :-) [1] http://technosophos.com/2009/05/21/perl-pie-if-you-only-learn-how-do-one-thing-perl-it.html http://technosophos.com/2009/05/21/perl-pie-if-you-only-lear... [2] https://robm.me.uk/ruby/2013/11/20/ruby-enp.html https://robm.me.uk/ruby/2013/11/20/ruby-enp.html [3] https://stackoverflow.com/a/2400/2062384 https://stackoverflow.com/a/2400/2062384
- rdtsc 8y ago
  > sed -e 's/erase_text//g' \ I like the pattern. Copied it to my notes to try next time I am grepping through logs. I usually just string a few 'grep -v ignore_this' with pipes but then it ignores the whole line. But I can see how erasing some parts of the line would be very helpful sometimes.
- laumars 8y ago
  You might already be aware of this, but you can chain multiple sed functions together into one command. eg sed -e 's/erase_text//g; s/erase_more//g' But I appreciate your example there is more for illustrative purposes rather than an actual pipeline you have in production.
- zippitydoodah68 8y ago
  These types of pipelines are why bash and the traditional tools are so maligned. Expert knowledge of sed|(g)awk|bash tools is necessary otherwise you end up with companies telling you that you can't use them. some_command | gawk ' /some.*regex$/ {gsub(erase_text,"");gsub(erase_more,"");split($2,a,":"); print (length(a[1]) ? a[1] : "STRING ERROR")}'
  - freedomben 8y ago
    I'll definitely agree that it can get out of hand. If it's a script/tool that will have many eyes on it and people that need to understand it, bash tools probably aren't the best way to go. I usually turn to ruby in those cases. But even for one off commands, I find myself using pipes like that all the time and I'm the only one that will ever see it, so if it's greek it's no problem :-)
- sa46 8y ago
  Drive by code review, you can end a line with the pipe (also && and ||) to avoid a backslash. some_command | grep -E 'some.*regex$' | sed -e 's/erase_text//g' | sed -e 's/erase_more//g' | awk '{ print $2 }' | cut -d ':' -f 1

What is your opinion on using other languages to augment bash? Awk, perl etc.

by antris 8y ago

What is your opinion on using other languages to augment bash? Awk, perl etc.

I personally consider awk, sed, grep, cut, etc. to be "part" of bash. I don't think I ever write a script without invoking those venerable tools at least once. I often have many pipes, such as some_command \ | grep -E 'some.*regex$' \ | sed -e 's/erase_text//g' \ | sed -e 's/erase_more//g' \ | awk '{ print $2 }' \ | cut -d ':' -f 1 I used to reach for Perl one-liners a lot, and still do sometimes when I need a gross regex that `sed -E` can't handle (See Perl Pie[1]), but Ruby has been making an appearance more and more often for handy one-liners. Roblog has a great post on using Ruby [2]. If the script is complex enough and I'm willing to take a dependency on Ruby, that is usually where I turn. Where that line is does change from time to time. Ruby makes it super trivial to call shell commands and process them easily as well, which makes it easy to integrate into existing scripting [3]. Hopefully that answered your question :-) [1] http://technosophos.com/2009/05/21/perl-pie-if-you-only-learn-how-do-one-thing-perl-it.html http://technosophos.com/2009/05/21/perl-pie-if-you-only-lear... [2] https://robm.me.uk/ruby/2013/11/20/ruby-enp.html https://robm.me.uk/ruby/2013/11/20/ruby-enp.html [3] https://stackoverflow.com/a/2400/2062384 https://stackoverflow.com/a/2400/2062384

rdtsc 8y ago
> sed -e 's/erase_text//g' \ I like the pattern. Copied it to my notes to try next time I am grepping through logs. I usually just string a few 'grep -v ignore_this' with pipes but then it ignores the whole line. But I can see how erasing some parts of the line would be very helpful sometimes.
laumars 8y ago
You might already be aware of this, but you can chain multiple sed functions together into one command. eg sed -e 's/erase_text//g; s/erase_more//g' But I appreciate your example there is more for illustrative purposes rather than an actual pipeline you have in production.
zippitydoodah68 8y ago
These types of pipelines are why bash and the traditional tools are so maligned. Expert knowledge of sed|(g)awk|bash tools is necessary otherwise you end up with companies telling you that you can't use them. some_command | gawk ' /some.*regex$/ {gsub(erase_text,"");gsub(erase_more,"");split($2,a,":"); print (length(a[1]) ? a[1] : "STRING ERROR")}'
- freedomben 8y ago
  I'll definitely agree that it can get out of hand. If it's a script/tool that will have many eyes on it and people that need to understand it, bash tools probably aren't the best way to go. I usually turn to ruby in those cases. But even for one off commands, I find myself using pipes like that all the time and I'm the only one that will ever see it, so if it's greek it's no problem :-)
sa46 8y ago
Drive by code review, you can end a line with the pipe (also && and ||) to avoid a backslash. some_command | grep -E 'some.*regex$' | sed -e 's/erase_text//g' | sed -e 's/erase_more//g' | awk '{ print $2 }' | cut -d ':' -f 1