5 ms·

As someone who wrote a lot of Perl in the late 90's and early 2000's and moved to Python for scientific work (simulations, etc.) as well as other tasks (web app

As someone who wrote a lot of Perl in the late 90's and early 2000's and moved to Python for scientific work (simulations, etc.) as well as other tasks (web apps, etc), what I saw was: - Perl actually was really popular for a while back then, especially in the Bioinformatics/Genomics space. It was all over that field, I think partially because it's really easy to think of a genome as just a text string of ATCGs and Perl was really convenient for manipulating text. - I originally switched to Python for some projects because I had to do simple GUIs and visualization. Pygame and TKinter were much nicer to deal with than Perl's options. If you were just reading data in from one file and writing it out to another, Perl was fine, but the GUI toolkits were miserable. - Numpy ("Numeric" at the time) was what really sealed the deal and probably pushed a lot of others to switch. Perl had PDL and similar but they weren't as fast or as easy to understand and use. Even if Perl is "Naturally far more efficient" than Python, it's still slow enough that for non-trivial calculations you would not want to use it directly. You have to use one of these other libraries where the actual number crunching is happening in highly tuned FORTRAN or C libraries. Python's options back at that time just leapfrogged Perl's and Numpy had really nice integration with Pygame, which made visualization in your app really smooth. Perl might've caught back up, but that was around when Perl6 was announced and sucked all the air out of the room.

simtel20 5y ago
I built and maintained the python stack at a financial firm that had a sizeable investment in perl and PDL, and iirc PDL was unreliable (crashy, not sure about other aspects of reliability) and I don't recall that it was making the transition to 64 bit in the mid/late 2000s at the speed numpy was. Do you remember any more detail from the perl side of the fence?
- thraxil 5y ago
  Not really. I vaguely remember trying out PDL and not getting very far. I was pretty comfortable writing C back then, so any time I'd run into performance issues in Perl, I'd usually just switch to C. Getting PDL working properly was painful enough that it didn't really seem worth spending a lot of time on when I could just write a small C program, especially for commandline data pipeline type situations. Once I started hitting projects with requirements for interactive GUIs though, keeping everything in the same language becomes more valuable and Python had the better complete package.
skulk 5y ago
> especially in the Bioinformatics/Genomics space My first internship was re-writing a Perl program that parsed the output of `samtools mpileup`[0] into a C program that used htslib[1] to directly read the BAM file and extract the relevant data. This preprocessing (the Perl version) was the slowest part of the pipeline, even slower than the actual data analysis. [0]: http://www.htslib.org/doc/samtools-mpileup.html http://www.htslib.org/doc/samtools-mpileup.html [1]: http://www.htslib.org/ http://www.htslib.org/
- olliej 5y ago
  oof I can't imagine converting perl string manipulation to C was fun. Did you try to replicate what the perl code was doing, or did you just redo the entire thing in a more C like way?
- thraxil 5y ago
  Yeah, I feel like that was a pretty common workflow. Someone would get a new algorithm working in Perl, test it on fairly small datasets, maybe publish their paper, then some poor chump would have to carefully rewrite it in C to make it actually useful for real data. I was at a university, so our chumps were grad students. I'm sure interns were the industry equivalent :)

simtel20 5y ago

I built and maintained the python stack at a financial firm that had a sizeable investment in perl and PDL, and iirc PDL was unreliable (crashy, not sure about other aspects of reliability) and I don't recall that it was making the transition to 64 bit in the mid/late 2000s at the speed numpy was. Do you remember any more detail from the perl side of the fence?

thraxil 5y ago
Not really. I vaguely remember trying out PDL and not getting very far. I was pretty comfortable writing C back then, so any time I'd run into performance issues in Perl, I'd usually just switch to C. Getting PDL working properly was painful enough that it didn't really seem worth spending a lot of time on when I could just write a small C program, especially for commandline data pipeline type situations. Once I started hitting projects with requirements for interactive GUIs though, keeping everything in the same language becomes more valuable and Python had the better complete package.

skulk 5y ago

> especially in the Bioinformatics/Genomics space My first internship was re-writing a Perl program that parsed the output of `samtools mpileup`[0] into a C program that used htslib[1] to directly read the BAM file and extract the relevant data. This preprocessing (the Perl version) was the slowest part of the pipeline, even slower than the actual data analysis. [0]: http://www.htslib.org/doc/samtools-mpileup.html http://www.htslib.org/doc/samtools-mpileup.html [1]: http://www.htslib.org/ http://www.htslib.org/

olliej 5y ago
oof I can't imagine converting perl string manipulation to C was fun. Did you try to replicate what the perl code was doing, or did you just redo the entire thing in a more C like way?
thraxil 5y ago
Yeah, I feel like that was a pretty common workflow. Someone would get a new algorithm working in Perl, test it on fairly small datasets, maybe publish their paper, then some poor chump would have to carefully rewrite it in C to make it actually useful for real data. I was at a university, so our chumps were grad students. I'm sure interns were the industry equivalent :)