4 ms·
> For a Perl-type problem (scanning and parsing big files), Perl is very fast. I think it's a matter of what you're comparing it to. Compared to using Perl fo
by haberman 12y ago
> For a Perl-type problem (scanning and parsing big files), Perl is very fast.
I think it's a matter of what you're comparing it to.
Compared to using Perl for a general-purpose problem, Perl for scanning/parsing is fast.
Compared to scanning/parsing with C, Perl is not fast.
$ ruby -e '1.upto(1000000) { |n| puts "This is line number #{n}" }' > file
$ time perl -ne 'print if /number 12345/' < file
[...]
real 0m0.193s
user 0m0.189s
sys 0m0.004s
$ time grep "number 12345" file
[...]
real 0m0.023s
user 0m0.019s
sys 0m0.005s
I gave Perl every possible advantage here. I didn't actually even write any Perl except a regular expression, which is delegated immediately to C. I didn't even write the loop in Perl, I like the Perl main() function handle that. And still the C program is almost 10x faster.
Note: these test runs are from Linux. On OS X the Perl results were almost the same, but "grep" was unexplainably way slower. It seems to hang after it's already dumped all of its output. Basically grep on OS X appears to be badly broken somehow.
- Igglyboo 12y agoI think the takeaway is that perl is "fast enough" for perl type problems. Writing a complicated file parser in C would be a nightmare.
- visarga 12y agoWhen I have a one-time computation job that takes an hour to write and two hours to run in Perl, but in C takes 10 hours to write and half an hour to run, then Perl is faster than C. And these one-time/rare/short jobs are much more frequent than intense, high throughput C code like the nginix web server or the node javascript interpreter.
- Igglyboo 12y agoI see your point, C is definitely the choice for long running jobs or jobs that will be run more than a handful of times, in my experience however I'm writing a lot of one-off scripts that take 30 seconds tops to run so Perl wins out pretty hard over C.
- vlisivka 12y agoYou are comparing two interpreters. One is slow, because it is written in C, second is fast, because it is written in C.
- cleaver 12y agoI recall anecdotal reports that Perl was faster than egrep in some cases. I never tested it myself and that was a long time ago—could be a bug that is long since fixed.
- csirac2 12y agoI'm always wary of these kinds of sub-second benchmarks because more often than not you've only accidentally measured just the compilation and startup times. I might have some bias though from speeding up a crusty old Perl CGI web apps with multi-second request times down to less than 100ms simply by keeping the perl processes persistent with mod_fcgid or whatever.
- haberman 12y ago> I'm always wary of these kinds of sub-second benchmarks because more often than not you've only accidentally measured just the compilation and startup times. I increased the iteration count by 10x and observed exactly the same pattern: $ time perl -ne 'print if /number 12345/' < file [...] real 0m1.661s user 0m1.586s sys 0m0.076s $ time grep "number 12345" file [...] real 0m0.188s user 0m0.132s sys 0m0.056s
- csirac2 12y agoCool, thanks for entertaining my superstitions :)
- kbenson 12y agoWhat you are seeing is different regex engines and capabilities, and grep's focus on pure speed and optimization of a common case and Perl's focus on versatility. I see very similar results between Perl and grep, and you can see this by also including egrep, which allows slightly more complex expressions: [root@stats ~]# time perl -ne 'print if /number 123456/' < /tmp/file [...] real 0m1.990s user 0m1.937s sys 0m0.049s [root@stats ~]# time grep "number 123456" /tmp/file [...] real 0m0.158s user 0m0.115s sys 0m0.035s [root@stats ~]# time egrep "number 123456" /tmp/file [...] real 0m0.150s user 0m0.127s sys 0m0.023s But what happens if we use a slightly more complex expression? [root@stats ~]# time perl -ne 'print if /number [1]23456/' < /tmp/file [...] real 0m1.989s user 0m1.925s sys 0m0.047s [root@stats ~]# time grep "number [1]23456" /tmp/file [...] real 0m1.402s user 0m1.366s sys 0m0.022s [root@stats ~]# time egrep "number [1]23456" /tmp/file [...] real 0m1.414s user 0m1.382s sys 0m0.031s The difference becomes much less pronounced. What if we make the expression just a bit more complex? [root@stats ~]# time perl -ne 'print if /number [1]23456[0-9]*/' < /tmp/file [...] real 0m1.950s user 0m1.910s sys 0m0.039s [root@stats ~]# time grep "number [1]23456[0-9]*" /tmp/file [...] real 0m9.353s user 0m9.307s sys 0m0.037s [root@stats ~]# time egrep "number [1]23456[0-9]*" /tmp/file [...] real 0m9.539s user 0m9.483s sys 0m0.045s So, now we have the Perl regex engine fairly static across extra complexity while grep and egrep are seeing order of magnitude time increases, and are much slower than Perl at this point. I suspect your first benchmark was the result of a specific optimization grep has that Perl doesn't, or it may be that grep was able to switch to using a DFA regex for that first case, while Perl doesn't both with a completely different regex implementation for special cases like that. Anecdata: I needed to process a large amount of XML a while back, to the point where a week spent testing and optimizing XML parsing libraries in Perl was worth it, because it could shave weeks or months off the processing time. The winner? A regex that captured attributes and content and assigned name/value pairs directly out to a hash. This was only possible because the XML was highly normalized, but it was actually over 10 times faster than the closest competitor for XML parsing I could fine, and I checked all the libXML libXML2, and SAX libraries I could get my hands on. In the end, it was something as simple as the following approximation: while (my ($doc) = $xml =~ /$get_record_xml_re/) { my %hash = $get_record_xml =~ /$record_begin_re$capture_name_and_value_pairs_re$record_end_re/; process_record( \%hash ); }