5 ms·

When I started working in bioinformatics I was excited to use Lisp for my work. I wrote a considerable amount of library code and programs at first. But I alway

When I started working in bioinformatics I was excited to use Lisp for my work. I wrote a considerable amount of library code and programs at first. But I always found resistance when it came to other people using my programs and even more if the program was to be published and released for general use. There is a large amount of really, really bad Perl code still in use in biology. Trust me when I say to those outside of the field, I've seen things you people wouldn't believe. I try to discourage Perl use between it makes it so easy for biologists to write terrible code. I've found a compromise in Python. People will happily use it because it's trendy and ubiquitous. And I think it makes it easier to write OK code. But I miss using Lisp. The style of programming when you have a REPL is perfect for this kind of thing. The ability to quickly write prototypes which can later be improved is an advantage which cannot be stressed enough. C is fantastic when you have a good idea of the algorithms and data structures you want to implement and are concerned with implementing them efficiently. But that's rarely the case when writing bioinformatics code. I need to know if the damn thing works first before I spend time making it work fast. And why would you not use a language where both things are equally easy? I'm happy to see this paper because it means that I have a way to justify my usage of Lisp in the future. Big thanks to the authors.

open_bear 10y ago
I think that IPython is one of the best tools for scientists. Jupyter Notebooks that allow reproducible results, a lot of data-wrangling libraries, visualizations, distributed computing. I wish more scientists knew of IPython.
- svwobwbou 10y ago
  yes - and the thing that makes python approachable is the syntax and standard formatting. I love LISP, I'm not confused or annoyed by the parens, but many people are. LISP would be more popular if there was a good (browser-based) hybrid structural editor with the keyboard-navigation familiarity of a text editor, in a notebook format where the LISP code is displayed like Python (the parens hidden, and handled automatically, but serialized as normal LISP code behind the scenes). This is not that hard to do, and we're nearly there: * http://gorilla-repl.org/ http://gorilla-repl.org/ * https://github.com/kovasb/session https://github.com/kovasb/session (Also my own early abandoned experiments: http://celeriac.net/iiiiioiooooo/ http://celeriac.net/iiiiioiooooo/ http://celeriac.net/iiiiioiooooo/public/ http://celeriac.net/iiiiioiooooo/public/ http://celeriac.net/iiiiioiooooo-dom/ http://celeriac.net/iiiiioiooooo-dom/ http://celeriac.net/io/public/ http://celeriac.net/io/public/)
  - greggyb 10y ago
    See Wisp Scheme: http://srfi.schemers.org/srfi-119/srfi-119.html http://srfi.schemers.org/srfi-119/srfi-119.html It doesn't address the browser-based point.
  - open_bear 10y ago
    GorillaREPL is very nice. There is also a Proto-REPL that looks very promising (https://github.com/jasongilman/proto-repl https://github.com/jasongilman/proto-repl)
hatmatrix 10y ago
Building a community around lisp is tricky. While there are many criticisms to this article [1], there are some elements of the "Lisp Curse" which is true - each programmer tends to rewrite his/her own libraries and so a collaborative framework seldom emerges even while many programmers may work in the same domain. This can be a disadvantage in a domain like the natural sciences where deep collaborations are necessary. Furthermore, the disconnect with traditional mathematical syntax has also turned off many users. There was a predecessor to R called Lisp-Stat [2] which never achieved the acceptance that S/R did, and the Lisp-based syntax of Lisp-Stat was cited as one of the reasons. Following the example of a high-profile Lisp supporter like Peter Norvig who accepted Python as a "Lispy" alternative early on, I think there are a lot of us who have come to accept Python as a necessary compromise. While Python is not homoiconic, functional programming is encouraged through its libraries rather than its core language definition, and the power of macros are missing, the inherent popularity and all the benefits that come with it is tough to beat. [1] http://winestockwebdesign.com/Essays/Lisp_Curse.html http://winestockwebdesign.com/Essays/Lisp_Curse.html [2] http://homepage.divms.uiowa.edu/~luke/xls/xlsinfo/xlsinfo.html http://homepage.divms.uiowa.edu/~luke/xls/xlsinfo/xlsinfo.ht...
- throwanem 10y ago
  > each programmer tends to rewrite his/her own libraries and so a collaborative framework seldom emerges even while many programmers may work in the same domain So, exactly like bioinformatics, then. Trivial tasks are supported by packages of libraries like BioPerl, but all the original research is sui generis, with the attitude in general being that each problem is unique and only results get published, so who cares if the code is a write-only mess as long as it does the one job after which no one will care about it anyway. In such an environment, the "Lisp Curse" already exists anyway, so why not get the benefit of a more powerful and expressive language when you're already paying its cost? (Granted, Lisp for bioinformatics is never going to become a generally accepted thing. But if you're not releasing code anyway, no one cares what language you write it in...)
  - hatmatrix 10y ago
    What about Bioconductor and Openbabel/Pybel?
yvdriess 10y ago
A benefit of introducing LISP in bioinformatics is the cultural element that tools are expected to work on the command line using simple linux streams for I/O. (As well as a systemic allergy to dependencies, preferring to do everything from scratch.) You can make drop-in Lisp replacements of shitty Perl tools and no one will be the wiser, as long as you are producing bit-compatible results. I imagine establishing trust in your tool will be harder if you provide them with Lisp vs. Perl source. Other communities (e.g. statistics, machine learning) have a culture of ramming all their tools in a toolbox-library/framework (R, sklearn). This makes for a much tightly coupled environment, making it much harder to introduce Lisp work.
GFK_of_xmaspast 10y ago
> I've found a compromise in Python. .... But I miss using Lisp. The style of programming when you have a REPL 95% of my python programming time these days is spent inside a repl.
nerdponx 10y ago
You might be interested in Hy [1], which is a Lisp implemented in Python that can import Python libraries. I'm not sure if it cross-compiles to Python or just to Python bytecode, though. 1: https://github.com/hylang/hy https://github.com/hylang/hy
- agumonkey 10y ago
  IIRC it's sexp to python AST. There's also Pixie, a python lisp to LLVM system, that was specifically made for high performance.
  - nerdponx 10y ago
    Is it this? https://github.com/pixie-lang/pixie https://github.com/pixie-lang/pixie
    - agumonkey 10y ago
      It seems so.. which makes my LLVM mention false. There's a talk by the author online http://www.youtube.com/watch?v=1AjhFZVfB9c http://www.youtube.com/watch?v=1AjhFZVfB9c worth it
- willismichael 10y ago
  For a moment you had me wondering if there would be a way to use Hy and google's new Grumpy together, and then I dismissed the idea as probably utter madness. For a long time I've been pining for a Clojure-like lisp that can compile to machine code.
  - nerdponx 10y ago
    Why wouldn't it work?
    - willismichael 10y ago
      I'm not certain that it wouldn't work, but it would have multiple compile steps: Hy => Python => Go => Native, which would either cause a lot of pain or make nigh impossible many of the advantages of using a lisp, like connecting a repl to a running process and being able to evaluate and rewrite code from there.
gwu78 10y ago
What Lisp did you use?
xaa 10y ago
Also in bioinformatics, and I use Python too, but not for the same reasons. Among languages with mature statistics/ML/numeric libraries, your choices are R and Python (MATLAB/Mathematica if you're willing to consider non-open-source). R is just terrible, so Python is all that's left if you want a sane language with batteries. I used to write a decent amount of Clojure. I would be surprised if you would have problems with reviewers complaining about a nonstandard language -- unless you're publishing a library, then you might rightfully get dinged because the target audience would be small. But if you're just running a routine analysis, I have always found the reviewers don't care what language it's in. Anyway, I dropped Clojure because the numeric support is just terrible. There's Java libraries, of course, but you lose the conciseness and functional aspect. You just can't performantly use arrays or matrices as if they were generic sequences, as appealing as it appears from a distance. Considered Racket, but I consider lazy evaluation a must (a la Python's generators), and I could never figure out how to use Racket's lazy mode. So, Python it is... Maybe choosing a programming language is like marriage: better to have something you can tolerate for a long period with all its warts than something you're infatuated with. > I have a way to justify my usage of Lisp in the future Haha, OK. I found this paper really unconvincing. No mention of the terms "numeric" or "array", and the only mentions of "statistics" and "machine learning" are in the form of "Lisp would be a really great foundation for stats/ML if someone would just write a library for it".
- abrax3141 10y ago
  Why would one need to mention numeric arrays? Lisp has all the usual capabilities in this area, and with a little effort is as fast as C, sometime faster. See stats comments elsewhere in this thread.

open_bear 10y ago

I think that IPython is one of the best tools for scientists. Jupyter Notebooks that allow reproducible results, a lot of data-wrangling libraries, visualizations, distributed computing. I wish more scientists knew of IPython.

svwobwbou 10y ago
yes - and the thing that makes python approachable is the syntax and standard formatting. I love LISP, I'm not confused or annoyed by the parens, but many people are. LISP would be more popular if there was a good (browser-based) hybrid structural editor with the keyboard-navigation familiarity of a text editor, in a notebook format where the LISP code is displayed like Python (the parens hidden, and handled automatically, but serialized as normal LISP code behind the scenes). This is not that hard to do, and we're nearly there: * http://gorilla-repl.org/ http://gorilla-repl.org/ * https://github.com/kovasb/session https://github.com/kovasb/session (Also my own early abandoned experiments: http://celeriac.net/iiiiioiooooo/ http://celeriac.net/iiiiioiooooo/ http://celeriac.net/iiiiioiooooo/public/ http://celeriac.net/iiiiioiooooo/public/ http://celeriac.net/iiiiioiooooo-dom/ http://celeriac.net/iiiiioiooooo-dom/ http://celeriac.net/io/public/ http://celeriac.net/io/public/)
- greggyb 10y ago
  See Wisp Scheme: http://srfi.schemers.org/srfi-119/srfi-119.html http://srfi.schemers.org/srfi-119/srfi-119.html It doesn't address the browser-based point.
- open_bear 10y ago
  GorillaREPL is very nice. There is also a Proto-REPL that looks very promising (https://github.com/jasongilman/proto-repl https://github.com/jasongilman/proto-repl)

hatmatrix 10y ago

Building a community around lisp is tricky. While there are many criticisms to this article [1], there are some elements of the "Lisp Curse" which is true - each programmer tends to rewrite his/her own libraries and so a collaborative framework seldom emerges even while many programmers may work in the same domain. This can be a disadvantage in a domain like the natural sciences where deep collaborations are necessary. Furthermore, the disconnect with traditional mathematical syntax has also turned off many users. There was a predecessor to R called Lisp-Stat [2] which never achieved the acceptance that S/R did, and the Lisp-based syntax of Lisp-Stat was cited as one of the reasons. Following the example of a high-profile Lisp supporter like Peter Norvig who accepted Python as a "Lispy" alternative early on, I think there are a lot of us who have come to accept Python as a necessary compromise. While Python is not homoiconic, functional programming is encouraged through its libraries rather than its core language definition, and the power of macros are missing, the inherent popularity and all the benefits that come with it is tough to beat. [1] http://winestockwebdesign.com/Essays/Lisp_Curse.html http://winestockwebdesign.com/Essays/Lisp_Curse.html [2] http://homepage.divms.uiowa.edu/~luke/xls/xlsinfo/xlsinfo.html http://homepage.divms.uiowa.edu/~luke/xls/xlsinfo/xlsinfo.ht...

throwanem 10y ago
> each programmer tends to rewrite his/her own libraries and so a collaborative framework seldom emerges even while many programmers may work in the same domain So, exactly like bioinformatics, then. Trivial tasks are supported by packages of libraries like BioPerl, but all the original research is sui generis, with the attitude in general being that each problem is unique and only results get published, so who cares if the code is a write-only mess as long as it does the one job after which no one will care about it anyway. In such an environment, the "Lisp Curse" already exists anyway, so why not get the benefit of a more powerful and expressive language when you're already paying its cost? (Granted, Lisp for bioinformatics is never going to become a generally accepted thing. But if you're not releasing code anyway, no one cares what language you write it in...)
- hatmatrix 10y ago
  What about Bioconductor and Openbabel/Pybel?

yvdriess 10y ago

A benefit of introducing LISP in bioinformatics is the cultural element that tools are expected to work on the command line using simple linux streams for I/O. (As well as a systemic allergy to dependencies, preferring to do everything from scratch.) You can make drop-in Lisp replacements of shitty Perl tools and no one will be the wiser, as long as you are producing bit-compatible results. I imagine establishing trust in your tool will be harder if you provide them with Lisp vs. Perl source. Other communities (e.g. statistics, machine learning) have a culture of ramming all their tools in a toolbox-library/framework (R, sklearn). This makes for a much tightly coupled environment, making it much harder to introduce Lisp work.

GFK_of_xmaspast 10y ago

> I've found a compromise in Python. .... But I miss using Lisp. The style of programming when you have a REPL 95% of my python programming time these days is spent inside a repl.

nerdponx 10y ago

You might be interested in Hy [1], which is a Lisp implemented in Python that can import Python libraries. I'm not sure if it cross-compiles to Python or just to Python bytecode, though. 1: https://github.com/hylang/hy https://github.com/hylang/hy

agumonkey 10y ago
IIRC it's sexp to python AST. There's also Pixie, a python lisp to LLVM system, that was specifically made for high performance.
- nerdponx 10y ago
  Is it this? https://github.com/pixie-lang/pixie https://github.com/pixie-lang/pixie
  - agumonkey 10y ago
    It seems so.. which makes my LLVM mention false. There's a talk by the author online http://www.youtube.com/watch?v=1AjhFZVfB9c http://www.youtube.com/watch?v=1AjhFZVfB9c worth it
willismichael 10y ago
For a moment you had me wondering if there would be a way to use Hy and google's new Grumpy together, and then I dismissed the idea as probably utter madness. For a long time I've been pining for a Clojure-like lisp that can compile to machine code.
- nerdponx 10y ago
  Why wouldn't it work?
  - willismichael 10y ago
    I'm not certain that it wouldn't work, but it would have multiple compile steps: Hy => Python => Go => Native, which would either cause a lot of pain or make nigh impossible many of the advantages of using a lisp, like connecting a repl to a running process and being able to evaluate and rewrite code from there.

gwu78 10y ago

What Lisp did you use?

xaa 10y ago

Also in bioinformatics, and I use Python too, but not for the same reasons. Among languages with mature statistics/ML/numeric libraries, your choices are R and Python (MATLAB/Mathematica if you're willing to consider non-open-source). R is just terrible, so Python is all that's left if you want a sane language with batteries. I used to write a decent amount of Clojure. I would be surprised if you would have problems with reviewers complaining about a nonstandard language -- unless you're publishing a library, then you might rightfully get dinged because the target audience would be small. But if you're just running a routine analysis, I have always found the reviewers don't care what language it's in. Anyway, I dropped Clojure because the numeric support is just terrible. There's Java libraries, of course, but you lose the conciseness and functional aspect. You just can't performantly use arrays or matrices as if they were generic sequences, as appealing as it appears from a distance. Considered Racket, but I consider lazy evaluation a must (a la Python's generators), and I could never figure out how to use Racket's lazy mode. So, Python it is... Maybe choosing a programming language is like marriage: better to have something you can tolerate for a long period with all its warts than something you're infatuated with. > I have a way to justify my usage of Lisp in the future Haha, OK. I found this paper really unconvincing. No mention of the terms "numeric" or "array", and the only mentions of "statistics" and "machine learning" are in the form of "Lisp would be a really great foundation for stats/ML if someone would just write a library for it".

abrax3141 10y ago
Why would one need to mention numeric arrays? Lisp has all the usual capabilities in this area, and with a little effort is as fast as C, sometime faster. See stats comments elsewhere in this thread.