5 ms·
Skip awk, use perl.... The alias below sets perl to loop over STDIN splitting each line on more than one whitespace character and populate an array F. The -nE
by Ultimatt 9y ago
Skip awk, use perl....
The alias below sets perl to loop over STDIN splitting each line on more than one whitespace character and populate an array F. The -nE will then Evaluate an expression from the command line looping over the input line-by-line.
alias glorp='perl -aF"/\s+/" -nE'
So now we have the command `glorp` to play with which has more familiar syntax than awk and all of CPAN available to play with!
$ [data is generated] | glorp '/Something/ and say $F[2]'
We have access to any Perl module by putting -MModule::Name=function after the command, the following will parse a JSON record per line and glorp out what we wanted:
$ echo -e '{"hello":"world"}\n{"hello":"cat"}' | glorp 'say decode_json($_)->{hello};' -MJSON=decode_json
world
cat
Maybe you are used to using curl too. There is a nice web framework in Perl called Mojolicious (http://mojolicious.org http://mojolicious.org) that provides a convenience module called 'ojo' for command line use. So grabbing the summary sentence from Wikipedia articles is as straight forward as below. Notice Mojolicious lets us use CSS selectors!
$ echo -e 'grep\nawk\nperl' \
| glorp 'say g("wikipedia.org/wiki/$F[0]")->dom->at("#mw-content-text > div > p")->all_text' -Mojo
- d33 9y agoHey, it would be awesome to have something like this for Python!
- Ultimatt 9y agoNot really possible in Python for anything overly useful/complex because of the significant white space, and regular expressions are not a first class object in the language. ipython is the equivalent solution I suppose. However, Ruby completely inherited this aspect of Perl and is another good candidate. It's just even old systems have a perl installed that will perform the awk like functionality.
- phaemon 9y agoThere is "Pyed Piper" aka `pyp` https://code.google.com/archive/p/pyp/ https://code.google.com/archive/p/pyp/ "Pyp is a linux command line text manipulation tool similar to awk or sed, but which uses standard python string and list methods as well as custom functions evolved to generate fast results in an intense production environment."
- Ultimatt 9y agoThat's cool, but I feel like you're unlikely to run into it very often in the wild? The vast majority of Linux systems come with ruby/perl out of the box. Plus if Python did support this sort of hackery it would quickly garner most of the bad press Perl has over the years :'(
- bobpaul 9y agoWho cares? It's on pypi (https://pypi.python.org/pypi/pyp/2.11 https://pypi.python.org/pypi/pyp/2.11) so just `pip install pyp` and then you have it. You're probably not going to want to distribute shell scripts that rely on pyp, but if you're distributing scripts you can just distribute scripts written entirely in Python and use pyp for your own one-liners.
- tyingq 9y agoThe lack of something like Perl/Ruby's -n and -p switches is a barrier as well. Those are handy because they automatically work for either piped input, or files named on the command line. Of course, you can do all of this in Python, just not as a one liner.
- deleted 9y ago[deleted]
- gmfawcett 9y agoI wrote a little tool called "pyline" about a hundred years ago... I still reach for it often when I'm in a hurry, and don't have time to futz around with awk/sed. https://code.activestate.com/recipes/437932-pyline-a-grep-like-sed-like-command-line-tool/ https://code.activestate.com/recipes/437932-pyline-a-grep-li...
- edraferi 9y agoThat's pretty cool, thanks!
- geophile 9y agoCheck out osh: https://github.com/geophile/osh https://github.com/geophile/osh
- prathiks 9y agoso is the intention to be python equivalent of awk
- vidarh 9y agoHere's the equivalent for Ruby: alias glorp='ruby -ane ' $ [data is generated] | glorp ' ~ /Something/ and puts $F[2]' Or: $ echo -e '{"hello":"world"}\n{"hello":"cat"}' | glorp 'puts JSON.load($_)["hello"] ' -rjson (Of course Ruby got the -a autosplit-mode and the -n assumed 'while gets(); ... end' loop from Perl along with $_ and $F, so it's very intentional that they're similar)
- Ultimatt 9y agoAwesome thanks for sharing this! I was too lazy to give a Ruby example alongside.
- kolodny 9y agoSomewhat related nodejs self plug: Use nip https://github.com/kolodny/nip https://github.com/kolodny/nip $ echo -e 'this\nis\na\nwhatever foo' | nip 'return /whatever/.test(line) && cols[1]' # foo
- jedisct1 9y agoAnd Ruby regexes are amazing.
- omaranto 9y agoI thought that they were amazing because they were just like Perl's. Are there any differences?
- codesnik 9y agowhat's the reason of using -F option? -a defaults " " separator, which already emulates awk behavior. If I'm not wrong, only difference between perl -aF"/\s+/" and perl -a is treating of leading whitespace in lines
- Ultimatt 9y agoTo show you could use other separators defined by any regex you like.
- ShannonAlther 9y agoThere was an awk/perl thread yesterday too.
- leovonl 9y agoSorry, but I strongly disagree. I can think of two very strong reasons to prefer awk over perl: * busybox implements awk, so its (usually) available even on embedded platforms * awk syntax is way clearer than perl, anyone that has a few experience in C + shell can figure out what is being done with some googling awk is also powerful enough to separate output to different files, accumulate inputs, etc. And if you need anything more complex, there are tons of other languages to choose from then (including perl).