5 ms·

I just use | perl -pi -e 's/foo/bar/g' , etc for this kind of stuff. Is there anything I can't do with perl on a line that sed will do? I can see how perl is

I just use | perl -pi -e 's/foo/bar/g' , etc for this kind of stuff. Is there anything I can't do with perl on a line that sed will do? I can see how perl is a lot more complex than sed, but I went through the whole perl learning curve back in the late 90's so it doesn't bother me that much..

dogweather 12y ago
For me, this is the whole point of perl: You can learn one tool which replaces three, each with their own syntax: Sed, Awk, and Grep.
- cogburnd02 12y ago
  Wait, why not just use awk? It can do pretty much everything the other two do (and can, like sed, be used to replace other things (like 'head')) and runs faster than perl at some things. (https://news.ycombinator.com/item?id=8858342 https://news.ycombinator.com/item?id=8858342)
  - dogweather 12y ago
    I learned awk in school, and like it. But I like to choose tools which increase programmer-efficiency before machine-efficiency. Perl is just one tool and one easy syntax to learn. Its regex language is now the standard as well. Only if runtime efficiency turns out to be too slow would I re-examine choice of tools. This is like avoiding premature optimization. (I've never found perl to be too slow in practice, btw.)
sbi 12y ago
Probably not, but in some cases sed and AWK might be faster. I am a big fan of AWK. It is limited in some ways that make it impractical to use for really serious programs, but it is very expressive. Look at [1, 2]. [1]: http://c2.com/doc/expense/ http://c2.com/doc/expense/ [2]: http://www.pement.org/awk/awk1line.txt http://www.pement.org/awk/awk1line.txt
- senorsmile 12y ago
  I have tested converting sed/awk lines to perl in a few base scripts that worked on a fairly large amount of data. Oddly enough, in every case, perl 5.18 performed at LEAST 1.5x faster, sometimes as much as 3.5x faster. Obviously anecdotal evidence, but recent versions of Perl seemed to have gained some good speed.
  - ronjouch 12y ago
    Rafe Colburn from Etsy wrote about this performance oddity: http://rc3.org/2014/08/28/surprisingly-perl-outperforms-sed-and-awk/ http://rc3.org/2014/08/28/surprisingly-perl-outperforms-sed-... EDIT to manage expectations: the article doesn't explain why, it just provides benchmarks and one commenter made a suggestion about character handling. More insight still welcome :)
    - sbi 12y ago
      There are many different versions of awk: gawk, (BSD) nawk, mawk, etc. I think OS X uses nawk, but mawk is reputedly faster. Gawk is definitely slower than both mawk. I'm not surprised that Perl is faster though.
    - Tobu 12y ago
      > I used the default OS X versions of these tools. The versions were Perl 5.16.2, Awk 20070501, and some version of BSD sed from 2005
  - _delirium 12y ago
    I've had a similar experience, with Perl up to about 7x faster. I had sed in a few data-mangling pipelines because I assumed simpler=faster, but replacing it with Perl was either a wash or a speedup in every case. This with the versions of perl and sed in Debian (looks like it's GNU sed), so ymmv with other seds. The case where I saw a 7x speedup was doing many-times-per-line, fixed-string search/replace on a file consisting of very long lines (an SQL dump where some lines had >1m characters). Perl was IO-bound (so presumably would've been even faster if I'd had better disks), while sed was CPU-bound at a pretty low fraction of the possible IO performance.
scdlbx 12y ago
It's a lot easier to delete specific lines using sed. Also you can have sed do replacements to the n'th instance of something. Doing that in Perl is a bit more complicated and a lot less succint. $ echo "foo foo foo foo" | sed 's/foo/bar/3' foo foo bar foo
- emmelaich 12y ago
  deleting (i.e) not printing lines Here are the examples for not printing the 4th line: sed: sed -n '4!p' awk: awk 'NR != 4' perl: perl -ne '$. != 4 && print' Not much between them really.
  - scdlbx 12y ago
    I would consider 'sed 4d' to be significantly easier than doing the same in Perl, but I don't disagree that it's not that hard to do in either.
- raiph 12y ago
  The Rakudo Perl 6 compiler is still immature and slow, and the -i option (in-place edit) hasn't yet been implemented, but, at least for comparison's sake: $ perl6 -pe 'next if ++$ == 2' example.txt ... prints all lines except line 2. This is an example from Perl 6 One Liners[1]. The `$` is just just an unnamed variable that is getting incremented once per evaluation (-e is for `evaluate`) which in this case happens once per line (-p is for printing each line of input after eval'ing the code -- unless a `next` applies, in which case that line gets skipped). And... $ echo "foo foo foo foo" | perl6 -pe 's:3rd/foo/bar/' ... replaces the third foo with bar. P6 regexes are far easier to read and way more powerful than P5 regexes. The `:3rd` bit is a general language feature called "Adverbs", in this case applied to the regex focused s/// built in.[2] [1] https://github.com/sillymoose/Perl6-One-Liners https://github.com/sillymoose/Perl6-One-Liners [2] http://doc.perl6.org/language/regexes#Adverbs http://doc.perl6.org/language/regexes#Adverbs
  - cogburnd02 12y ago
    I've read literally nothing about perl 6 but what David Skoll wrote here: http://david.skoll.ca/blog/2010-07-29-perl-sss.html http://david.skoll.ca/blog/2010-07-29-perl-sss.html Quote: "I asked on a forum what the goals are for relative size and speed of Perl 6 vs. Perl 5, and a Perl 6 developer responded that a reasonable goal would be to have Perl 6 be twice as big as Perl 5 and take twice as long to start up. "To achieve this goal, the Perl 6 developers will have to shrink the program size by a factor of 6.1 (that is, get rid of about 84% of the code.) They'll need to reduce startup memory consumption by a factor of 13.7 (that is, cut out 93.7% of their memory use) and reduce startup time by a factor of over 275. "Oh, and this is after they add in all the missing features required to bring Perl 6 up to production-level." Has the situation gotten better since 2010?
kerkeslager 12y ago
I'm a firm believer that there are many ways to work effectively. If Perl fills the same needs for you that sed does for some people, then by all means stick to Perl. However, for someone who knows neither, here are some reasons you might want to choose sed over Perl: 1. sed syntax is pervasive in other tools. For example, to run a substitution from early on in the tutorial in vim, type :%s/abc/(&)/<enter> from insert mode. 2. sed is simpler than Perl. It used to be that Perl filled a unique role as a scripting language but now there are a bunch of languages in that space (most notably Python and Ruby). Python + sed for example would fulfill most of the same functions that Perl does (and there are reasons to choose Python over Perl as scripting languages, although that's a much more complicated domain to discuss). 3. sed is more performant (or so I hear). This has never been a real concern for me, but some people cite this is a concern. 4. sed usually is a bit more terse. For the length of expressions you'll typically be writing with sed, this isn't a big concern. Disclaimer: There are probably good reasons to choose Perl over sed, too. Not being a Perl guy, I don't know those reasons. I'll leave that to someone who knows more about Perl. When I learned sed it was because I was taking a *nix class in college, and a professor pointed me at sed and not Perl. I learned sed instead of Perl because it was put in front of me, not because of any weighing of pros and cons. That's how a lot of learning happens. Sometimes simply learning what's put in front of you leads turns out to be an obvious mistake in retrospect (I was stuck writing VBA for a little while). I don't know whether learning Perl or sed is better for what sed does, but I do know that after maybe 6ish years using sed quite frequently it hasn't turned out to be an obvious mistake.
- cogburnd02 12y ago
  > "3. sed is more performant (or so I hear)." There are regexes for which awk (maybe sed too?) will perform at a reasonable speed while perl is incredibly slow. Graph: http://pdos.csail.mit.edu/~rsc/regexp-img/grep1p.png http://pdos.csail.mit.edu/~rsc/regexp-img/grep1p.png Article: http://swtch.com/~rsc/regexp/regexp1.html http://swtch.com/~rsc/regexp/regexp1.html Perl...could...employ much faster algorithms when presented with regular expressions that don't have backreferences.

dogweather 12y ago

For me, this is the whole point of perl: You can learn one tool which replaces three, each with their own syntax: Sed, Awk, and Grep.

cogburnd02 12y ago
Wait, why not just use awk? It can do pretty much everything the other two do (and can, like sed, be used to replace other things (like 'head')) and runs faster than perl at some things. (https://news.ycombinator.com/item?id=8858342 https://news.ycombinator.com/item?id=8858342)
- dogweather 12y ago
  I learned awk in school, and like it. But I like to choose tools which increase programmer-efficiency before machine-efficiency. Perl is just one tool and one easy syntax to learn. Its regex language is now the standard as well. Only if runtime efficiency turns out to be too slow would I re-examine choice of tools. This is like avoiding premature optimization. (I've never found perl to be too slow in practice, btw.)

sbi 12y ago

Probably not, but in some cases sed and AWK might be faster. I am a big fan of AWK. It is limited in some ways that make it impractical to use for really serious programs, but it is very expressive. Look at [1, 2]. [1]: http://c2.com/doc/expense/ http://c2.com/doc/expense/ [2]: http://www.pement.org/awk/awk1line.txt http://www.pement.org/awk/awk1line.txt

senorsmile 12y ago
I have tested converting sed/awk lines to perl in a few base scripts that worked on a fairly large amount of data. Oddly enough, in every case, perl 5.18 performed at LEAST 1.5x faster, sometimes as much as 3.5x faster. Obviously anecdotal evidence, but recent versions of Perl seemed to have gained some good speed.
- ronjouch 12y ago
  Rafe Colburn from Etsy wrote about this performance oddity: http://rc3.org/2014/08/28/surprisingly-perl-outperforms-sed-and-awk/ http://rc3.org/2014/08/28/surprisingly-perl-outperforms-sed-... EDIT to manage expectations: the article doesn't explain why, it just provides benchmarks and one commenter made a suggestion about character handling. More insight still welcome :)
  - sbi 12y ago
    There are many different versions of awk: gawk, (BSD) nawk, mawk, etc. I think OS X uses nawk, but mawk is reputedly faster. Gawk is definitely slower than both mawk. I'm not surprised that Perl is faster though.
  - Tobu 12y ago
    > I used the default OS X versions of these tools. The versions were Perl 5.16.2, Awk 20070501, and some version of BSD sed from 2005
- _delirium 12y ago
  I've had a similar experience, with Perl up to about 7x faster. I had sed in a few data-mangling pipelines because I assumed simpler=faster, but replacing it with Perl was either a wash or a speedup in every case. This with the versions of perl and sed in Debian (looks like it's GNU sed), so ymmv with other seds. The case where I saw a 7x speedup was doing many-times-per-line, fixed-string search/replace on a file consisting of very long lines (an SQL dump where some lines had >1m characters). Perl was IO-bound (so presumably would've been even faster if I'd had better disks), while sed was CPU-bound at a pretty low fraction of the possible IO performance.

scdlbx 12y ago

It's a lot easier to delete specific lines using sed. Also you can have sed do replacements to the n'th instance of something. Doing that in Perl is a bit more complicated and a lot less succint. $ echo "foo foo foo foo" | sed 's/foo/bar/3' foo foo bar foo

emmelaich 12y ago
deleting (i.e) not printing lines Here are the examples for not printing the 4th line: sed: sed -n '4!p' awk: awk 'NR != 4' perl: perl -ne '$. != 4 && print' Not much between them really.
- scdlbx 12y ago
  I would consider 'sed 4d' to be significantly easier than doing the same in Perl, but I don't disagree that it's not that hard to do in either.
raiph 12y ago
The Rakudo Perl 6 compiler is still immature and slow, and the -i option (in-place edit) hasn't yet been implemented, but, at least for comparison's sake: $ perl6 -pe 'next if ++$ == 2' example.txt ... prints all lines except line 2. This is an example from Perl 6 One Liners[1]. The `$` is just just an unnamed variable that is getting incremented once per evaluation (-e is for `evaluate`) which in this case happens once per line (-p is for printing each line of input after eval'ing the code -- unless a `next` applies, in which case that line gets skipped). And... $ echo "foo foo foo foo" | perl6 -pe 's:3rd/foo/bar/' ... replaces the third foo with bar. P6 regexes are far easier to read and way more powerful than P5 regexes. The `:3rd` bit is a general language feature called "Adverbs", in this case applied to the regex focused s/// built in.[2] [1] https://github.com/sillymoose/Perl6-One-Liners https://github.com/sillymoose/Perl6-One-Liners [2] http://doc.perl6.org/language/regexes#Adverbs http://doc.perl6.org/language/regexes#Adverbs
- cogburnd02 12y ago
  I've read literally nothing about perl 6 but what David Skoll wrote here: http://david.skoll.ca/blog/2010-07-29-perl-sss.html http://david.skoll.ca/blog/2010-07-29-perl-sss.html Quote: "I asked on a forum what the goals are for relative size and speed of Perl 6 vs. Perl 5, and a Perl 6 developer responded that a reasonable goal would be to have Perl 6 be twice as big as Perl 5 and take twice as long to start up. "To achieve this goal, the Perl 6 developers will have to shrink the program size by a factor of 6.1 (that is, get rid of about 84% of the code.) They'll need to reduce startup memory consumption by a factor of 13.7 (that is, cut out 93.7% of their memory use) and reduce startup time by a factor of over 275. "Oh, and this is after they add in all the missing features required to bring Perl 6 up to production-level." Has the situation gotten better since 2010?

kerkeslager 12y ago

I'm a firm believer that there are many ways to work effectively. If Perl fills the same needs for you that sed does for some people, then by all means stick to Perl. However, for someone who knows neither, here are some reasons you might want to choose sed over Perl: 1. sed syntax is pervasive in other tools. For example, to run a substitution from early on in the tutorial in vim, type :%s/abc/(&)/<enter> from insert mode. 2. sed is simpler than Perl. It used to be that Perl filled a unique role as a scripting language but now there are a bunch of languages in that space (most notably Python and Ruby). Python + sed for example would fulfill most of the same functions that Perl does (and there are reasons to choose Python over Perl as scripting languages, although that's a much more complicated domain to discuss). 3. sed is more performant (or so I hear). This has never been a real concern for me, but some people cite this is a concern. 4. sed usually is a bit more terse. For the length of expressions you'll typically be writing with sed, this isn't a big concern. Disclaimer: There are probably good reasons to choose Perl over sed, too. Not being a Perl guy, I don't know those reasons. I'll leave that to someone who knows more about Perl. When I learned sed it was because I was taking a *nix class in college, and a professor pointed me at sed and not Perl. I learned sed instead of Perl because it was put in front of me, not because of any weighing of pros and cons. That's how a lot of learning happens. Sometimes simply learning what's put in front of you leads turns out to be an obvious mistake in retrospect (I was stuck writing VBA for a little while). I don't know whether learning Perl or sed is better for what sed does, but I do know that after maybe 6ish years using sed quite frequently it hasn't turned out to be an obvious mistake.

cogburnd02 12y ago
> "3. sed is more performant (or so I hear)." There are regexes for which awk (maybe sed too?) will perform at a reasonable speed while perl is incredibly slow. Graph: http://pdos.csail.mit.edu/~rsc/regexp-img/grep1p.png http://pdos.csail.mit.edu/~rsc/regexp-img/grep1p.png Article: http://swtch.com/~rsc/regexp/regexp1.html http://swtch.com/~rsc/regexp/regexp1.html Perl...could...employ much faster algorithms when presented with regular expressions that don't have backreferences.