8 ms·

Bioinformatics and the joy of Perl6

49 points by kamaal 12y ago

grondilu 12y ago
For those who want to test their skills in Bioinformatics: http://rosalind.info/ http://rosalind.info/ The neat things is that you can use any language you want.
- SilasX 12y ago
  On that note, awk shows up a lot in the shortest solutions because of the ease with which it parses input files. I haven't used perl myself, so can anyone comment on the merits of perl vs awk here? They often compete for this use case.
  - grondilu 12y ago
    Oversimplifying a bit, I'm going to tell you that Perl was initially conceived as a mix between C, sed and awk. So, there are quite a few awk idioms that are directly translatable in Perl. As a matter of fact, there even is a section dedicated to awk in perltrap[1], enumerating the differences between the two languages. 1. http://perldoc.perl.org/perltrap.html#Awk-Traps http://perldoc.perl.org/perltrap.html#Awk-Traps
dalke 12y ago
I'm a bit surprised at the implied superiority to the other Bio* projects. None of them started as a copy of BioPerl (and btw, it's "Biopython", not "BioPython"), and none of them are trying to implement all of the features of BioPerl. For that matter, the other projects implement features that BioPerl doesn't have. It's interesting that the example supports a comment in the FASTA record. I wonder why the author included it. I've never seen a comment used in the wild, but then again I haven't worked with FASTA files for some time. My experience was that nearly every tool would break if you give it a FASTA file with a comment. For example, the BioPerl FASTA parser is at https://github.com/bioperl/bioperl-live/blob/master/Bio/SeqIO/fasta.pm#L139 https://github.com/bioperl/bioperl-live/blob/master/Bio/SeqI... , where you can see it doesn't support a comment. Since the chance of breakage is so high, my belief 10 years ago was that this original feature from Pearson is dead, and will never be revived.
- Ultimatt 12y ago
  Update for the superiority, though that is part my flippant language use and some projection :P The comment section in the grammar isn't comments as in ; line comments but the per sequence comments usually used to encode extra meta information such as with GenBank. No idea if they should be called something else, I've always thought of them as comments.
  - cjfields1 12y ago
    Okay, I think I see the discrepancy. It's a nomenclature thing. The Bio* refer to the additional stuff after the ID as the sequence 'description'. There is a subtle difference between that and a simple comment, though I have seen even NCBI use this as the 'junk drawer' for any additional seq information (such as the headers in the nr database).
- chaosfox 12y ago
  comments go after blank space in the sequence name, for example: >ENST00000517143 ncrna:snRNA ATGC... "ncrna:snRNA" is a comment
  - cjfields1 12y ago
    Actually, in the Bio* langs that would be referred to as the 'description'. @dalke is referring to the very rarely-used ';' I believe, which I don't recall any modern-day FASTA parser supporting.
    - Ultimatt 12y ago
      Oh I've seen them used in the wild... I'd rather have a parser throw an exception than have that data go through though lol
      - cjfields1 12y ago
        Curious. Where have you run into them (so I can run the other way :)
deleted 12y ago
[deleted]
jghn 12y ago
I'm really surprised to see modern discussions of Perl in bioinformatics, normally when I see any mention I assume that either a) it's an outsider who is snickering at how backwards we are or b) it's an old timer who has never gotten with the times. I've been in the bioinformatics space for about 13 years now and I've only come a small handful of perl scripts, all of which have been legacy programs. Even when I started in the field, perl was viewed as somewhat anachronistic. All of the end-user software I've written (read: For researchers) has been in R, Java and a wee bit of scala. These days all of the researchers I know are using some combo of R, Python and to a lesser extent MATLAB. Many have useful C and/or Java (and/or Scala) skills as well. My personal experience is obviously not exhaustive but I find it hard to believe that many people are using Perl these days, and IMO that's a matter of good riddance.
- Snoooze 12y ago
  > b) it's an old timer who has never gotten with the times. As someone who knows the author of the post, that is mostly true ;) In all seriousness, while I generally agree with your sentiments I don't think that Perl is dead within Bioinformatics. Not least since major data providers, e.g. Ensembl, write their tools in Perl.
  - jghn 12y ago
    Heh. Oh I know it still exists, but while this is admittedly anecdotal on my part (and likely heavily biased by the institutions I've been a part of) I haven't seen a single new person who defaults to Perl in a very, very, very long time. And to be clear, I'm not trying to rip the author or the article, it just seems ... oddly timed, like writing a contemporary article on how to use COBOL in the business computing space.
    - Snoooze 12y ago
      I think the suggestions is that Perl 6 is a very different beast to Perl 5. Or to put it another way, you could read the title of the post as "Bioinformatics and the joy of Foo Programming Language". I'm not sure I agree with that, however.
coriny 12y ago
Just one technical comment - that parser will break on many FASTA files. They can also include '.' & * characters ('.' is often output by sequence aligners to indicate sections outside of the model, * is there for aa but not dna/rna and I have seen them in sequencer output for uncertain positions). I think this kind of edge case variance is possibly why the various Bio-* tools have not been very successful in all, you end up spending hours trying to work out why it's not working with your particular inputs. On a personal viewpoint: "Take a look at the REPL dump below and see if you can work out where we are implicitly using $seq as a Str rather than Seq:" - this section pretty much sums up why I've always hated working with Other Peoples' Perl. To accurately follow someone else's perl requires either an encyclopedic knowledge of the language and it's edge cases or very heavy documentation and a set standard of coding practices. It's a shame that this looks true for Perl 6 as well. Great for personal use, I guess. But by and large I'm glad to see more projects switch over to python.
- kamaal 12y ago
  >>this section pretty much sums up why I've always hated working with Other Peoples' Perl. I think that statement was really more of 'The remainder is left for the users as an assignment' kind of statement. >>To accurately follow someone else's perl requires either an encyclopedic knowledge of the language and it's edge cases or very heavy documentation and a set standard of coding practices. To accurately follow anything non-trivial written by anybody in any programming language will require you a good grip over the language and its corner cases. There is no magic here, if a language doesn't offer facilities to heavy lift a certain thing you will end up writing those yourself.
  - coriny 12y ago
    >> To accurately follow anything non-trivial written by anybody in any programming language will require you a good grip over the language and its corner cases. There is no magic here, if a language doesn't offer facilities to heavy lift a certain thing you will end up writing those yourself. I'm going to have speak subjectively and relative to my own skills here. Without rigorous use of standards Perl 5 is a nightmare to read. With very clear and consistent style then it can be readable, but this has to be enforced extremely strictly for any code base beyond trivial. Perl's 100 ways to do things is not a positive in this context. Case in point, I was given a bit of perl two weeks ago by a bioinformatics lecturer. Did not use strict or warnings; variables weren't declared in scope; no subroutines, let alone Moose objects. And this is typical of ~90% of bioinformatics perl that gets written. What's more, it was a trivial script that was simply parsing some BLAST matches from CSV, checking for overlaps and assigning a type based on the match gene combination - and was completely indecipherable. It took me two days of refactoring to pick out all the details. And he managed to use a couple of magic symbols I'd never seen before. The first time I was given a bit of python, without learning the language at all, I was able to extend the script and get it do some new things (it was considerably more complex than the aforementioned perl script). And even when it is well written and documented - I did work with some very good perl programmers - it's tended to end up as very fragile and difficult to extend or even substantially redevelop due to the heavy use of context for determining behaviour (and the actual context can be difficult to determine). Unit testing had to be very extensive since it was difficult to isolate changes. I'm going to have to be precise here - I'm attacking Perl 5. It just seemed from that section that Perl 6 doesn't solve the major problem (difficult to deduce the context and work out the resulting effect) I had with Perl 5. I may be completely missing the mark and actually readability is much better. I really hope so. >> Perl 6 isn't production ready yet. But even a cursory glance over the features tells, Python isn't even in the same league of tools Perl 6 will be. Vapourware is vapourware. Perl 6 was meant to be production ready a decade ago. On the other hand, I'm interested in knowing what you think Perl 6 brings that will give it the edge over Python? Do you see it displacing Python as rapidly as Python replaced Perl 5 as the science "glue" language of choice? With the apparent rise of Julia as well, could it just be too late?

49 points by kamaal 12y ago

For those who want to test their skills in Bioinformatics: http://rosalind.info/ http://rosalind.info/ The neat things is that you can use any language you want.

SilasX 12y ago
On that note, awk shows up a lot in the shortest solutions because of the ease with which it parses input files. I haven't used perl myself, so can anyone comment on the merits of perl vs awk here? They often compete for this use case.
- grondilu 12y ago
  Oversimplifying a bit, I'm going to tell you that Perl was initially conceived as a mix between C, sed and awk. So, there are quite a few awk idioms that are directly translatable in Perl. As a matter of fact, there even is a section dedicated to awk in perltrap[1], enumerating the differences between the two languages. 1. http://perldoc.perl.org/perltrap.html#Awk-Traps http://perldoc.perl.org/perltrap.html#Awk-Traps

dalke 12y ago

I'm a bit surprised at the implied superiority to the other Bio* projects. None of them started as a copy of BioPerl (and btw, it's "Biopython", not "BioPython"), and none of them are trying to implement all of the features of BioPerl. For that matter, the other projects implement features that BioPerl doesn't have. It's interesting that the example supports a comment in the FASTA record. I wonder why the author included it. I've never seen a comment used in the wild, but then again I haven't worked with FASTA files for some time. My experience was that nearly every tool would break if you give it a FASTA file with a comment. For example, the BioPerl FASTA parser is at https://github.com/bioperl/bioperl-live/blob/master/Bio/SeqIO/fasta.pm#L139 https://github.com/bioperl/bioperl-live/blob/master/Bio/SeqI... , where you can see it doesn't support a comment. Since the chance of breakage is so high, my belief 10 years ago was that this original feature from Pearson is dead, and will never be revived.

Ultimatt 12y ago
Update for the superiority, though that is part my flippant language use and some projection :P The comment section in the grammar isn't comments as in ; line comments but the per sequence comments usually used to encode extra meta information such as with GenBank. No idea if they should be called something else, I've always thought of them as comments.
- cjfields1 12y ago
  Okay, I think I see the discrepancy. It's a nomenclature thing. The Bio* refer to the additional stuff after the ID as the sequence 'description'. There is a subtle difference between that and a simple comment, though I have seen even NCBI use this as the 'junk drawer' for any additional seq information (such as the headers in the nr database).
chaosfox 12y ago
comments go after blank space in the sequence name, for example: >ENST00000517143 ncrna:snRNA ATGC... "ncrna:snRNA" is a comment
- cjfields1 12y ago
  Actually, in the Bio* langs that would be referred to as the 'description'. @dalke is referring to the very rarely-used ';' I believe, which I don't recall any modern-day FASTA parser supporting.
  - Ultimatt 12y ago
    Oh I've seen them used in the wild... I'd rather have a parser throw an exception than have that data go through though lol
    - cjfields1 12y ago
      Curious. Where have you run into them (so I can run the other way :)

deleted 12y ago

[deleted]

jghn 12y ago

I'm really surprised to see modern discussions of Perl in bioinformatics, normally when I see any mention I assume that either a) it's an outsider who is snickering at how backwards we are or b) it's an old timer who has never gotten with the times. I've been in the bioinformatics space for about 13 years now and I've only come a small handful of perl scripts, all of which have been legacy programs. Even when I started in the field, perl was viewed as somewhat anachronistic. All of the end-user software I've written (read: For researchers) has been in R, Java and a wee bit of scala. These days all of the researchers I know are using some combo of R, Python and to a lesser extent MATLAB. Many have useful C and/or Java (and/or Scala) skills as well. My personal experience is obviously not exhaustive but I find it hard to believe that many people are using Perl these days, and IMO that's a matter of good riddance.

Snoooze 12y ago
> b) it's an old timer who has never gotten with the times. As someone who knows the author of the post, that is mostly true ;) In all seriousness, while I generally agree with your sentiments I don't think that Perl is dead within Bioinformatics. Not least since major data providers, e.g. Ensembl, write their tools in Perl.
- jghn 12y ago
  Heh. Oh I know it still exists, but while this is admittedly anecdotal on my part (and likely heavily biased by the institutions I've been a part of) I haven't seen a single new person who defaults to Perl in a very, very, very long time. And to be clear, I'm not trying to rip the author or the article, it just seems ... oddly timed, like writing a contemporary article on how to use COBOL in the business computing space.
  - Snoooze 12y ago
    I think the suggestions is that Perl 6 is a very different beast to Perl 5. Or to put it another way, you could read the title of the post as "Bioinformatics and the joy of Foo Programming Language". I'm not sure I agree with that, however.

coriny 12y ago

Just one technical comment - that parser will break on many FASTA files. They can also include '.' & * characters ('.' is often output by sequence aligners to indicate sections outside of the model, * is there for aa but not dna/rna and I have seen them in sequencer output for uncertain positions). I think this kind of edge case variance is possibly why the various Bio-* tools have not been very successful in all, you end up spending hours trying to work out why it's not working with your particular inputs. On a personal viewpoint: "Take a look at the REPL dump below and see if you can work out where we are implicitly using $seq as a Str rather than Seq:" - this section pretty much sums up why I've always hated working with Other Peoples' Perl. To accurately follow someone else's perl requires either an encyclopedic knowledge of the language and it's edge cases or very heavy documentation and a set standard of coding practices. It's a shame that this looks true for Perl 6 as well. Great for personal use, I guess. But by and large I'm glad to see more projects switch over to python.

kamaal 12y ago
>>this section pretty much sums up why I've always hated working with Other Peoples' Perl. I think that statement was really more of 'The remainder is left for the users as an assignment' kind of statement. >>To accurately follow someone else's perl requires either an encyclopedic knowledge of the language and it's edge cases or very heavy documentation and a set standard of coding practices. To accurately follow anything non-trivial written by anybody in any programming language will require you a good grip over the language and its corner cases. There is no magic here, if a language doesn't offer facilities to heavy lift a certain thing you will end up writing those yourself.
- coriny 12y ago
  >> To accurately follow anything non-trivial written by anybody in any programming language will require you a good grip over the language and its corner cases. There is no magic here, if a language doesn't offer facilities to heavy lift a certain thing you will end up writing those yourself. I'm going to have speak subjectively and relative to my own skills here. Without rigorous use of standards Perl 5 is a nightmare to read. With very clear and consistent style then it can be readable, but this has to be enforced extremely strictly for any code base beyond trivial. Perl's 100 ways to do things is not a positive in this context. Case in point, I was given a bit of perl two weeks ago by a bioinformatics lecturer. Did not use strict or warnings; variables weren't declared in scope; no subroutines, let alone Moose objects. And this is typical of ~90% of bioinformatics perl that gets written. What's more, it was a trivial script that was simply parsing some BLAST matches from CSV, checking for overlaps and assigning a type based on the match gene combination - and was completely indecipherable. It took me two days of refactoring to pick out all the details. And he managed to use a couple of magic symbols I'd never seen before. The first time I was given a bit of python, without learning the language at all, I was able to extend the script and get it do some new things (it was considerably more complex than the aforementioned perl script). And even when it is well written and documented - I did work with some very good perl programmers - it's tended to end up as very fragile and difficult to extend or even substantially redevelop due to the heavy use of context for determining behaviour (and the actual context can be difficult to determine). Unit testing had to be very extensive since it was difficult to isolate changes. I'm going to have to be precise here - I'm attacking Perl 5. It just seemed from that section that Perl 6 doesn't solve the major problem (difficult to deduce the context and work out the resulting effect) I had with Perl 5. I may be completely missing the mark and actually readability is much better. I really hope so. >> Perl 6 isn't production ready yet. But even a cursory glance over the features tells, Python isn't even in the same league of tools Perl 6 will be. Vapourware is vapourware. Perl 6 was meant to be production ready a decade ago. On the other hand, I'm interested in knowing what you think Perl 6 brings that will give it the edge over Python? Do you see it displacing Python as rapidly as Python replaced Perl 5 as the science "glue" language of choice? With the apparent rise of Julia as well, could it just be too late?