25 ms·

The Machine That Builds Itself: The Strengths of the Lisp Languages (2016)

239 points by kuwze 8y ago

transfire 8y ago
You underestimate the power of the Forth.
agounaris 8y ago
I see a title "The Machine That Builds Itself"....but there is no code! Show me the code or throw the paper away...
0vermorrow 8y ago
The paper itself doesn't focus on Machine Learning / AI or anything too specific for that matter (it reads like a review / praise of LISPs), but I am currently researching Symbolic Artificial Intelligence as well as let's say the history of "Good Old-Fashioned Artificial Intelligence" and the role LISP took in all of that. Any books / papers / articles that one should definitely read on these topics?
- tlavoie 8y ago
  One of the classics, now available on Github: Paradigms of Artificial Intelligence Programminghttps://github.com/norvig/paip-lisp https://github.com/norvig/paip-lisp
- zshrdlu 8y ago
  You may be interested in a little-known older book Computational Semantics: An introduction to Artificial Intelligence and Natural Language Comprehension: https://archive.org/details/computationalsem04char#maincontent https://archive.org/details/computationalsem04char#mainconte...
  - bruthafez 8y ago
    Is there any different link to this book? Not working for me but seems fascinating.
- sahil-kang 8y ago
  If you’re also interested in the history and evolution of lisp please see _The Evolution of Lisp_: http://dreamsongs.com/Files/HOPL2-Uncut.pdf http://dreamsongs.com/Files/HOPL2-Uncut.pdf
- abecedarius 8y ago
  I've only read part, but https://ai.stanford.edu/~nilsson/QAI/qai.pdf https://ai.stanford.edu/~nilsson/QAI/qai.pdf seems pretty comprehensive.
zaarn 8y ago
LISP always makes me sad. It's a language from which all modern languages could learn so much (and currently a lot of features of LISP get cloned). LISP Machines pioneered basically 90% of what modern computers and the internet do today. Yet, it's a very niche language, an almost forgotten artifact of time.
- eafkuor 8y ago
  Clojure for me is the perfect modern LISP, too bad it seems it's dying (happy to stand corrected there)
  - edem 8y ago
    You seem right. Every metric I can check says it is going downhill. Too bad, I used to go to the Clojure meetups in my area, I even had a talk about Clojure + Emacs.
    - joncampbelldev 8y ago
      Can you list some of these metrics?
      - cutler 8y ago
        See my comment above re the Indeed.com API.
    - xixixao 8y ago
      Many reasons. S-expressions would be the least of my worries. Java is dying, not on the whole, but certainly in mindset of people who are picking new languages. It’s integration with OSs, which Clojure depends on, is arcane by today standards. Second, it’s tightly controlled by one person, and they happen to hate types. Similar to CoffeeScript, couple years after the language inception people figure out that they miss types. Adding them back in is super tough. And third, abstractions have costs. There is no wide-spread popular LISP, and people should really ask themselves why. We have lots of code generation, and depend on it more and more. But it’s “simple” (or “primitive”). It’s usually one step. You can read the input, and you can read the output. I’m starting to get a sense that macros are like currying. Elegant, beautiful, powerful. But net negative if your concern is to get lots of people to create software together.
magicmouse 8y ago
Lisp is part of the family of languages that J. Backus referred to as 'transformational languages'. Its ability to self-modify creates programs that are almost impossible for people other than the author to understand. This is why after 50 years it is still a niche language, and will remain so. There is little demand for programming languages that are hard to maintain. A working program lasts decades, and passes through many hands, and LISP is dead-end technology that will forever keep being brought up as its temptations are irresistible to some.
- vindarel 8y ago
  FYI Lisp (CL) is still used and being adopted in some fields: - http://lisp-lang.org/success/ http://lisp-lang.org/success/ (pgloader was rewritten from python) - https://github.com/azzamsa/awesome-lisp-companies https://github.com/azzamsa/awesome-lisp-companies Besides what's call Lisp in the Tiobe index(https://www.tiobe.com/tiobe-index/ https://www.tiobe.com/tiobe-index/) is ranked 27, before D, Clojure, Lua, Erlang, Rust, Julia… And I find Lisp programs great to maintain: the language is stable, the ecosystem too. Python programs on the contrary are a chore to maintain, even after only 2 years.
- TurboHaskal 8y ago
  I am confident that my Lisp code will still run after some decades. What would be the average life expectancy of a JS project?
  - oblio 8y ago
    There is a certain irony to this. Framework heavy Javascript will be hard to rebuild, even, after 1-2 years (NPM churn). Vanilla JS? Decades. I think you can run a JS script from 1998 without any issues. Modern Vanilla JS written in 2018 has the same backwards compatibility guarantees, it's extremely likely that you'll be able to run it in 2038. Once you add Babel, React & co, all bets are off...
- fauigerzigerk 8y ago
  This is one of the eternal debates in software engineering. Knowing that certain powerful features of programming languages can be used to create unmaintanable code, do we ban those features or do we try to harness them by using a disciplined approach to software design? Specifically with regard to meta programming, I think we have seen all that can happen several times over. If you have those features within the programming language they tend to get overused. If you don't have them within the programming language, they get tacked on in some inconsistent, half-assed way as soon as the need for frameworks, DSLs or sophisiticated configuration arises.
arknave 8y ago
> "Lists, which are a generalization of graphs, are extraordinarily well supported by Lisp." I always thought of linked lists as a special kind of graph, where each node except the head and tail had exactly one incoming edge and one outgoing edge. What's a better way to think of this in terms of LISP?
- mirekrusin 8y ago
  An array of bits is generalisation of everything.
- thaumasiotes 8y ago
  Internally a Lisp list is a binary tree in which left children are leaves storing a value and right children aren't leaves (except the terminus) and don't store values. Or rather, there is a structure called a cons cell which consists of two pointers, "left" and "right"; a cons cell in which the right pointer is NIL is a list; and a cons cell in which the right pointer points to a list is also a list. The values in the list are whatever is pointed to by the left pointers. If you diagram this out, you'll find that your mental image is basically accurate already. It seems difficult to describe lists of this form as a "generalization" of graphs.
daniel-cussen 8y ago
It's been so sad to see that while the AI winter ended, the Lisp winter that started around the same time never did. Maybe it's coming back after consolidation on the languages end: SBCL and Clojure are coming ahead as clear winners, and are quite powerful. I'm sure I'm not the only guy who started learning Lisp after reading Paul Graham's essays, and I'm going to keep using Lisp for several reasons, but the momentum just isn't there.
- mbrock 8y ago
  Lisp certainly wasn't in vogue when Paul Graham did his startup and wrote those essays. "Robert and I both knew Lisp well, and we couldn't see any reason not to trust our instincts and go with Lisp. We knew that everyone else was writing their software in C++ or Perl. But we also knew that that didn't mean anything. If you chose technology that way, you'd be running Windows. When you choose technology, you have to ignore what other people are doing, and consider only what will work the best." "This is especially true in a startup. In a big company, you can do what all the other big companies are doing. But a startup can't do what all the other startups do. I don't think a lot of people realize this, even in startups." [...] "If other companies didn't want to use Lisp, so much the better. It might give us a technological edge, and we needed all the help we could get." http://www.paulgraham.com/avg.html http://www.paulgraham.com/avg.html
  - tim333 8y ago
    >But a startup can't do what all the other startups do Seems like a lot of startups do do what the others do tech wise and use Ruby which seems to work out ok.
    - vidarh 8y ago
      It's more that a startup can't do what all the others do in every aspect. You need to do something differently (different niche) or better (if there are others in your niche).
- s-shellfish 8y ago
hevi_jos 8y ago
The guys that wrote this piece have drunk the kool aid of Lisp. To say that library support for bioinformatics is on the early stages is quite a misunderstanding. Lisp was one if not the first language to be used in everything, including bioinformatics. If it is not used anymore is because people have abandoned it. People have abandoned it once its flaws became evident. The main flaw of Lisp is using linear data for everything. It is obsolete, it was designed for linear memory like tapes or punched cards, not random access to memory like we have today. Languages like Clojure has not this limitation but they have others, like forcing you to use the java virtual machine, good if optional, terrible if mandatory. In current systems with huge amount of RAM, when you could have 128GBs on a developer system having to use a system that shoehorns everything to Lists is going backwards. It also maximizes verbosity as everything uses the same thing, pattern differentiation is hard for actual humans to read and understand. This was a serious flaw that even Lisp creators quickly identified and tried to solve, remember Lisp was considered a temporary solution while the good solution was(never) found. Not to talk about the absence of privileged instructions, everybody on the team changing whatever it wants, even basic instructions. Don't get me wrong, I love Lisp, it is one of the more interesting languages in the world. I am actually working on porting some of the features it has(introspection) to other languages. But from my point of view it is not that good for production for most projects. I have drunk the kool Aid myself before and have the scar tissue from that.
- nabla9 8y ago
  > The main flaw of Lisp is using linear data for everything. It This is just utter ignorance. Good example of not knowing that you don't know and interpolating from too little data. The name "Lisp" comes from List Processing, but Lisp programming has not been about list processing since 50's. All the old Lisps like Common Lisp and Scheme had full set of data structures, including vectors and arrays.
- mbrock 8y ago
  The main flaw of Lisp is using linear data for everything. Huh? Which Lisp are you talking about? That seems like a serious misconception! Certainly any Lisp after 1960 has great support for arrays, trees, graphs, hash tables, you name it.
wodenokoto 8y ago
What's a good lisp ide other than emacs? To me, learning lisp and Emacs at the same time seems like a tall order
- kkleindev 8y ago
  Why are you only looking into ides? Any editor might do a solid job. When working with clojure, parinfer[0] was all I really needed. [0] https://shaunlebron.github.io/parinfer/ https://shaunlebron.github.io/parinfer/
  - wodenokoto 8y ago
    Since everything is a function, I really, really want inline function argument hints.
    - quickoats 8y ago
      Emacs Eldoc mode.
  - pjmlp 8y ago
    Maybe because Lisp Machines alongside Smalltalk invented the whole IDE concept?
- heracles 8y ago
  LispWorks offers several version of their free IDE with some limitations (f.x. limited heap size), most of which are irrelevant if you just want to try/learn Common Lisp. I have limited personal experience with it, but it seems good and I have read good things about it, written by free users. * http://www.lispworks.com/products/lispworks.html#personal http://www.lispworks.com/products/lispworks.html#personal
- panzerklein 8y ago
  There are some decent REPL support in vim: slimv for SBCL and vim-fireplace for Clojure
- s-shellfish 8y ago
mark_l_watson 8y ago
Great writeup on specific advantages of Lisp languages! I have used Lisp on two long term medical and bio information consulting projects (one also used semantic web/linked data tech) and I could not agree more that building Lisp up to be a new language for applications, repl based development, and the expressiveness of Lisp languages are all huge wins in building knowledge intensive systems. EDIT: I will add that although I enjoy using Lisp languages, and have written a few Lisp books, that I don’t like to talk anyone into using any particular programming language or technology stack. There is a difference in enjoying our own choices to telling other people what choices they should make.
- marmaduke 8y ago
  What do you do for heavy number crunching and the like?
  - mark_l_watson 8y ago
    In ancient history I used FORTRAN. For machine learning, which is my job, I use TensorFlow using the Keras APIs. There are a few interesting machine learning libraries in Common Lisp but I don’t use them professionally. I am experimenting with saving trained Keras models and converting them to a format with runtime code for Racket Scheme - I might open source that if I can improve the performance.
    - marmaduke 8y ago
      I had previously toyed with numerical prototypes in typed Racket but could never get it fast enough. But, a typed dialect which transpiles to an OpenCL kernel might work.
    - TheRealKing 8y ago
      In modern history, I still use (modern) Fortran (as well as MATLAB, Python, and R). It does what I need to do, efficiently with minimal coding.
AzzieElbab 8y ago
We have reached the point where saying "pick the right tool..." usually means completely opposite. PHP for web servers, python for large applications, go or anything c like for business logic
olavk 8y ago
There is a saying about the band Velvet Underground - that not many people bought their records, but everyone who bought one went out and started their own band. I think Lisp is kind of similar - its influence is much larger than its fame, but ever language designer is inspired by it to some extent. I do think Lisp kind of missed the boat when a wave of dynamic languages (Perl, Python, Ruby) went mainstream. Today the most exciting developments in language design seem to be around static type systems, type inference etc.
- yogthos 8y ago
  I think there is a lot of exciting stuff happening in Racket and Clojure around contract systems right now with things like Racket Contracts and Clojure Spec. This approach provides an appealing alternative to static typing because it focuses on semantic correctness while static typing focuses primarily on self consistency. In my view the former is much more valuable because ultimately we want to know that the code is doing what was intended, and static typing doesn't appear to be an effective tool for encoding intent.
  - hashkb 8y ago
    I think Typed Racket has more buzz than contracts... maybe even seen as a replacement/evolution by many.
    - Avshalom 8y ago
      I can't swear to this but I think they've gone basically hand in hand. I'm pretty sure I read a few Racket posts about how they use contracts to keep a safe interface between typed and untyped code and just in general between languages.
      - samth 8y ago
        Yes, this is correct. The recent explosion of work on gradual typing (including Typed Racket) was enabled by the development of contracts.
  - 8y ago
Areading314 8y ago
As elegant as lisp is, I think the flexibility just kills its usability on teams, while more rigid languages like Java, Go or C see massive adoption. Language expressiveness is rarely a real bottleneck, compared to debugging and understanding program flow and getting new contributors up to speed. Separately I think each lisp has an individual fundamental flaw that makes it unusable for production, eg clojure having terrible error handling, common lisp being case insensitive, scheme having a minimal standard library, racket being fairly slow, etc.
- junke 8y ago
  I tend to find it very easy to debug CL code. In fact, CL is designed to be introspective: "inspect", "trace", "break", "disassemble" and "step" are part of the language. Also, Common Lisp is case sensitive.
  - workinfunk 8y ago
    You can hardly fault newcomers for not understanding the finer nuances when everything they type automatically gets uppercased.
    - junke 8y ago
      That's why I try to correct people when possible and point newcomers to Practical Common Lisp (Seibel).
- jandrese 8y ago
  Is case insensitivity really a flaw? Other than some weak namespace hinting (constants in ALL CAPS for example) what is the flaw of your language being case insensitive for everything but string handling? Would C be ruined if people could type: If ( expression ) { effect }; I'm also becoming unsold on file system case sensitivity as well. Should "Income Report 2018Q1" really be different than "Income report 2018Q1"? How often is this a desirable feature? The only reason I have now for maintaining case sensitivity is that in the Unicode world casefolding is an absolute nightmare and no language or file system built today can afford to not use Unicode.
ectospheno 8y ago
I find the Lisp variants to be lovely right up until I want to package and ship an actual application to a normal human being. Then they are quite clumsy. I find the same to be true of a number of other more recently popular high level languages. I've always assumed it was because graduate students never got to the point where they ship anything.
- lgessler 8y ago
  I haven't tried doing this in other Lisps, but it's really easy if you're using Clojure's most popular build tool, Leiningen. For JVM Clojure you can produce a single .jar ("uberjar") that you can run as you'd expect, and for ClojureScript, Google's Closure Compiler is used to output a single minified and tree-shaken JS file.
  - Oreb 8y ago
    The parent poster was talking about shipping "an actual application to a normal human being". A .jar file is a non-starter, as normal human beings these days don't have Java installed and have no desire to install it. Don't get me wrong: I love Clojure, and it's currently the language that pays my bills. But I'd never consider using it to build an app for end users (except possibly a ClojureScript React Native app for mobile devices).
- vindarel 8y ago
  How so ? In CL you can build an executable. (https://lispcookbook.github.io/cl-cookbook/scripting.html#building-a-self-contained-executable https://lispcookbook.github.io/cl-cookbook/scripting.html#bu...) It's large-ish (because it ships an interpreter, a debugger that we can access from outside, etc), but it has a near-zero startup time, and it works for web apps with their embedded web server.
  - claudiawerner 8y ago
    For ages I couldn't figure out how exactly I would be able to use Quicklisp libraries to create something, and then create an executable from the source code. I think I ended up giving up, after only finding a couple of blogs that were using ancient versions of everything involved. I remember even having trouble loading some of the packages. Overall my experience with writing CL for myself was fun and interesting, but my experience writing it to be used by other people was anything but. There seemed to be too much rubbish to confuse me, as opposed to something either like C (header files + corresponding source) or Python (a directory with an __init__.py is something you can import). The module system has also turned me away from Scheme; the point where different Schemes all implement say R6RS but they have different module systems, or even different ways of converting an integer to a string, varying support for SRFIs (the regex SRFI is nearly four years old and the only Scheme to implement that is Chibi) and the completely variable module system, where nothing at all is consistent, none of the modules work in another Scheme, even if you were to extract the code. You want something from Chicken to use in Guile? Too bad, you have to manually do all the work yourself, despite the fact that both modules are supposedly written in standard Scheme. The module situation eventually pushed me to write my own Lisp, but I realised that unless I were to go ahead and implement all the things I wanted, everything I would miss, I'd be in just the same place, reminded of that xkcd comic in which "there are now 15 competing standards". So I gave up on doing just that, and now I'm literally writing a Lisp compiler and then operating system with a friend, because for me it's a lot less frustrating to say "that doesn't work on my operating system" than it is to say "that doesn't work in this guy's Scheme". It wasn't supposed to make sense.
carapace 8y ago
All these comments about how Lisp missed the boat, and you're all wrong. We missed the boat. Lisp has been going strong among it's own the whole time. It's not their fault we're all so stubborn, punishing ourselves over and over again like some sick masochists. I got mad when I finally learned Lisp and realized how much time and effort has been wasted over the years because we're too lazy, ignorant, or foolish to use it. (As a tangent, it's simply incredible how deliberately ignorant programmers can be. I've worked with guys getting paid to write software who didn't know who Alan Kay is, or had never heard of Prolog. What's worse is they're not ashamed of their ignorance. Can you imagine a physicist who had never heard of Newton?) I don't use Lisp, but I have the grace to admit that that's a personal failing. (I joke that Python's only major problem is that it ruins you for other languages... I just can't quit that sweet syntax. Although Python 3 is a train wreck IMHO. Don't get me started.) I would switch to Lisp, but I found something even better, which brings me to my actual, non-ranty, point: There's a language even better than Lisp. It's called Joy and it combines all the best parts of Lisp with the best parts of Forth to make an enormously simple system for describing software. It's a purely functional, stack-based, "concatinative" language that turns out to be good for "Categorical" programming: programs in Joy can be "instantiated" over different categories to generate different kinds of computations. I only have one good reference for "Categorical" programming right now: a paper and talk by Conal Elliott "Compiling to Categories" February 2017 http://conal.net/papers/compiling-to-categories/ http://conal.net/papers/compiling-to-categories/ He's working in Haskell, not Joy, but he's describing the idea: from one piece of code you can get calculations, dataflow diagrams, circuit descriptions, type signatures (inference), derivatives, etc., by implementations of the categories for each kind.
phlakaton 8y ago
I'm disappointed by this paper. The general review of the language is of the sort that you can find a dime a dozen on LISP enthusiast sites--so this paper does little to effectively recruit existing programmers to the cause. On the other hand, the references to current bioinformatics projects using LISP is helpful as an index, but provide few details to convince a biologist that they're worth checking out--so I'm not convinced on that side either. The "key point" of a forthcoming BioLisp is embarrassingly glossed over. I think the paper would be much stronger if it focused on that BioLisp aspect, talking about: * What is BioLisp? A mere web portal pointing to various resources across various languages? An attempt to create a curated installation of version-compatible libraries for a specific LFL? An attempt to develop a fully-integrated framework for that LFL?? (I think Sage for Python might be a useful model here, but if you were trying to build that, you'd have to start by picking your LFL, which the authors at present seem unwilling to advocate. That in itself demonstrates one of the main liabilities at present in adopting LISP--the community is fractured between Racket, Scheme, Clojure, and various implementations of Common LISP--often to the extent that they argue over whether their respective opponent languages even _deserve_ to be called LISPs. And, of course, in this forum we even have Arc fanatics to add to the lot. ;-) ) * What challenges face biologists attempting to use LISP in their work? How does BioLisp help to address those? ("Your language lacks macros" is not an argument at a sufficiently detailed level, I'm afraid. Rather, actually show them a DSL built in LISP that they will desperately want!) * For that matter, what challenges face biologists attempting to use R/Python/Julia/etc for their work? How does BioLisp help to address those? * What promising pieces could BioLisp be built upon? What pieces are missing? What pieces need modernization? By all means, I encourage scientists and LISP enthusiasts to make the case that biologists should/could be using LISP--but make sure you actually make the case!
oldandtired 8y ago
What are the pros and cons of Lisp (or any other language for that matter)? In my undergraduate days (oh so many decades ago), I was introduced to Lisp. For some of the problems we were solving, it was the perfect tool. For others, a different language could be used. The pros of Lisp (in all its varieties) are in many ways the cons of Lisp. The macro system is both beneficial and antagonistic to the development of sane programs. Used judiciously, macros may life writing code much easier. However, it is not uncommon for Lisp programmers to write macros without consideration for those who come after them. This is, of course, true for any language. There are some major systems out there that are written in Lisp and when trying to change them (in a maintenance way), it is just as painful as in pretty much any other language. The competent and talented Lisp programmers is imbued with the same arrogance that C++ competent and talented programmers have. Tricky programming is fine if it is fully documented and the reasoning for the tricks is fully explained. Lisp and its variations have much to offer the programming community. But the way programs are written quickly diminishes the effectiveness of the language for the larger community. Each language has features that can be of use to each of us. they are tools we can use to solve the problems we are facing. No one language is suitable for all problems. If you have that attitude then you only see hammers and nails.

239 points by kuwze 8y ago

transfire 8y ago

You underestimate the power of the Forth.

agounaris 8y ago

I see a title "The Machine That Builds Itself"....but there is no code! Show me the code or throw the paper away...

0vermorrow 8y ago

The paper itself doesn't focus on Machine Learning / AI or anything too specific for that matter (it reads like a review / praise of LISPs), but I am currently researching Symbolic Artificial Intelligence as well as let's say the history of "Good Old-Fashioned Artificial Intelligence" and the role LISP took in all of that. Any books / papers / articles that one should definitely read on these topics?

tlavoie 8y ago
One of the classics, now available on Github: Paradigms of Artificial Intelligence Programminghttps://github.com/norvig/paip-lisp https://github.com/norvig/paip-lisp
zshrdlu 8y ago
You may be interested in a little-known older book Computational Semantics: An introduction to Artificial Intelligence and Natural Language Comprehension: https://archive.org/details/computationalsem04char#maincontent https://archive.org/details/computationalsem04char#mainconte...
- bruthafez 8y ago
  Is there any different link to this book? Not working for me but seems fascinating.
sahil-kang 8y ago
If you’re also interested in the history and evolution of lisp please see _The Evolution of Lisp_: http://dreamsongs.com/Files/HOPL2-Uncut.pdf http://dreamsongs.com/Files/HOPL2-Uncut.pdf
abecedarius 8y ago
I've only read part, but https://ai.stanford.edu/~nilsson/QAI/qai.pdf https://ai.stanford.edu/~nilsson/QAI/qai.pdf seems pretty comprehensive.

zaarn 8y ago

LISP always makes me sad. It's a language from which all modern languages could learn so much (and currently a lot of features of LISP get cloned). LISP Machines pioneered basically 90% of what modern computers and the internet do today. Yet, it's a very niche language, an almost forgotten artifact of time.

eafkuor 8y ago
Clojure for me is the perfect modern LISP, too bad it seems it's dying (happy to stand corrected there)
- edem 8y ago
  You seem right. Every metric I can check says it is going downhill. Too bad, I used to go to the Clojure meetups in my area, I even had a talk about Clojure + Emacs.
  - joncampbelldev 8y ago
    Can you list some of these metrics?
    - cutler 8y ago
      See my comment above re the Indeed.com API.
  - xixixao 8y ago
    Many reasons. S-expressions would be the least of my worries. Java is dying, not on the whole, but certainly in mindset of people who are picking new languages. It’s integration with OSs, which Clojure depends on, is arcane by today standards. Second, it’s tightly controlled by one person, and they happen to hate types. Similar to CoffeeScript, couple years after the language inception people figure out that they miss types. Adding them back in is super tough. And third, abstractions have costs. There is no wide-spread popular LISP, and people should really ask themselves why. We have lots of code generation, and depend on it more and more. But it’s “simple” (or “primitive”). It’s usually one step. You can read the input, and you can read the output. I’m starting to get a sense that macros are like currying. Elegant, beautiful, powerful. But net negative if your concern is to get lots of people to create software together.

magicmouse 8y ago

Lisp is part of the family of languages that J. Backus referred to as 'transformational languages'. Its ability to self-modify creates programs that are almost impossible for people other than the author to understand. This is why after 50 years it is still a niche language, and will remain so. There is little demand for programming languages that are hard to maintain. A working program lasts decades, and passes through many hands, and LISP is dead-end technology that will forever keep being brought up as its temptations are irresistible to some.

vindarel 8y ago
FYI Lisp (CL) is still used and being adopted in some fields: - http://lisp-lang.org/success/ http://lisp-lang.org/success/ (pgloader was rewritten from python) - https://github.com/azzamsa/awesome-lisp-companies https://github.com/azzamsa/awesome-lisp-companies Besides what's call Lisp in the Tiobe index(https://www.tiobe.com/tiobe-index/ https://www.tiobe.com/tiobe-index/) is ranked 27, before D, Clojure, Lua, Erlang, Rust, Julia… And I find Lisp programs great to maintain: the language is stable, the ecosystem too. Python programs on the contrary are a chore to maintain, even after only 2 years.
TurboHaskal 8y ago
I am confident that my Lisp code will still run after some decades. What would be the average life expectancy of a JS project?
- oblio 8y ago
  There is a certain irony to this. Framework heavy Javascript will be hard to rebuild, even, after 1-2 years (NPM churn). Vanilla JS? Decades. I think you can run a JS script from 1998 without any issues. Modern Vanilla JS written in 2018 has the same backwards compatibility guarantees, it's extremely likely that you'll be able to run it in 2038. Once you add Babel, React & co, all bets are off...
fauigerzigerk 8y ago
This is one of the eternal debates in software engineering. Knowing that certain powerful features of programming languages can be used to create unmaintanable code, do we ban those features or do we try to harness them by using a disciplined approach to software design? Specifically with regard to meta programming, I think we have seen all that can happen several times over. If you have those features within the programming language they tend to get overused. If you don't have them within the programming language, they get tacked on in some inconsistent, half-assed way as soon as the need for frameworks, DSLs or sophisiticated configuration arises.

arknave 8y ago

> "Lists, which are a generalization of graphs, are extraordinarily well supported by Lisp." I always thought of linked lists as a special kind of graph, where each node except the head and tail had exactly one incoming edge and one outgoing edge. What's a better way to think of this in terms of LISP?

mirekrusin 8y ago
An array of bits is generalisation of everything.
thaumasiotes 8y ago
Internally a Lisp list is a binary tree in which left children are leaves storing a value and right children aren't leaves (except the terminus) and don't store values. Or rather, there is a structure called a cons cell which consists of two pointers, "left" and "right"; a cons cell in which the right pointer is NIL is a list; and a cons cell in which the right pointer points to a list is also a list. The values in the list are whatever is pointed to by the left pointers. If you diagram this out, you'll find that your mental image is basically accurate already. It seems difficult to describe lists of this form as a "generalization" of graphs.

daniel-cussen 8y ago

It's been so sad to see that while the AI winter ended, the Lisp winter that started around the same time never did. Maybe it's coming back after consolidation on the languages end: SBCL and Clojure are coming ahead as clear winners, and are quite powerful. I'm sure I'm not the only guy who started learning Lisp after reading Paul Graham's essays, and I'm going to keep using Lisp for several reasons, but the momentum just isn't there.

mbrock 8y ago
Lisp certainly wasn't in vogue when Paul Graham did his startup and wrote those essays. "Robert and I both knew Lisp well, and we couldn't see any reason not to trust our instincts and go with Lisp. We knew that everyone else was writing their software in C++ or Perl. But we also knew that that didn't mean anything. If you chose technology that way, you'd be running Windows. When you choose technology, you have to ignore what other people are doing, and consider only what will work the best." "This is especially true in a startup. In a big company, you can do what all the other big companies are doing. But a startup can't do what all the other startups do. I don't think a lot of people realize this, even in startups." [...] "If other companies didn't want to use Lisp, so much the better. It might give us a technological edge, and we needed all the help we could get." http://www.paulgraham.com/avg.html http://www.paulgraham.com/avg.html
- tim333 8y ago
  >But a startup can't do what all the other startups do Seems like a lot of startups do do what the others do tech wise and use Ruby which seems to work out ok.
  - vidarh 8y ago
    It's more that a startup can't do what all the others do in every aspect. You need to do something differently (different niche) or better (if there are others in your niche).
s-shellfish 8y ago

hevi_jos 8y ago

The guys that wrote this piece have drunk the kool aid of Lisp. To say that library support for bioinformatics is on the early stages is quite a misunderstanding. Lisp was one if not the first language to be used in everything, including bioinformatics. If it is not used anymore is because people have abandoned it. People have abandoned it once its flaws became evident. The main flaw of Lisp is using linear data for everything. It is obsolete, it was designed for linear memory like tapes or punched cards, not random access to memory like we have today. Languages like Clojure has not this limitation but they have others, like forcing you to use the java virtual machine, good if optional, terrible if mandatory. In current systems with huge amount of RAM, when you could have 128GBs on a developer system having to use a system that shoehorns everything to Lists is going backwards. It also maximizes verbosity as everything uses the same thing, pattern differentiation is hard for actual humans to read and understand. This was a serious flaw that even Lisp creators quickly identified and tried to solve, remember Lisp was considered a temporary solution while the good solution was(never) found. Not to talk about the absence of privileged instructions, everybody on the team changing whatever it wants, even basic instructions. Don't get me wrong, I love Lisp, it is one of the more interesting languages in the world. I am actually working on porting some of the features it has(introspection) to other languages. But from my point of view it is not that good for production for most projects. I have drunk the kool Aid myself before and have the scar tissue from that.

nabla9 8y ago
> The main flaw of Lisp is using linear data for everything. It This is just utter ignorance. Good example of not knowing that you don't know and interpolating from too little data. The name "Lisp" comes from List Processing, but Lisp programming has not been about list processing since 50's. All the old Lisps like Common Lisp and Scheme had full set of data structures, including vectors and arrays.
mbrock 8y ago
The main flaw of Lisp is using linear data for everything. Huh? Which Lisp are you talking about? That seems like a serious misconception! Certainly any Lisp after 1960 has great support for arrays, trees, graphs, hash tables, you name it.

wodenokoto 8y ago

What's a good lisp ide other than emacs? To me, learning lisp and Emacs at the same time seems like a tall order

kkleindev 8y ago
Why are you only looking into ides? Any editor might do a solid job. When working with clojure, parinfer[0] was all I really needed. [0] https://shaunlebron.github.io/parinfer/ https://shaunlebron.github.io/parinfer/
- wodenokoto 8y ago
  Since everything is a function, I really, really want inline function argument hints.
  - quickoats 8y ago
    Emacs Eldoc mode.
- pjmlp 8y ago
  Maybe because Lisp Machines alongside Smalltalk invented the whole IDE concept?
heracles 8y ago
LispWorks offers several version of their free IDE with some limitations (f.x. limited heap size), most of which are irrelevant if you just want to try/learn Common Lisp. I have limited personal experience with it, but it seems good and I have read good things about it, written by free users. * http://www.lispworks.com/products/lispworks.html#personal http://www.lispworks.com/products/lispworks.html#personal
panzerklein 8y ago
There are some decent REPL support in vim: slimv for SBCL and vim-fireplace for Clojure
s-shellfish 8y ago

mark_l_watson 8y ago

Great writeup on specific advantages of Lisp languages! I have used Lisp on two long term medical and bio information consulting projects (one also used semantic web/linked data tech) and I could not agree more that building Lisp up to be a new language for applications, repl based development, and the expressiveness of Lisp languages are all huge wins in building knowledge intensive systems. EDIT: I will add that although I enjoy using Lisp languages, and have written a few Lisp books, that I don’t like to talk anyone into using any particular programming language or technology stack. There is a difference in enjoying our own choices to telling other people what choices they should make.

marmaduke 8y ago
What do you do for heavy number crunching and the like?
- mark_l_watson 8y ago
  In ancient history I used FORTRAN. For machine learning, which is my job, I use TensorFlow using the Keras APIs. There are a few interesting machine learning libraries in Common Lisp but I don’t use them professionally. I am experimenting with saving trained Keras models and converting them to a format with runtime code for Racket Scheme - I might open source that if I can improve the performance.
  - marmaduke 8y ago
    I had previously toyed with numerical prototypes in typed Racket but could never get it fast enough. But, a typed dialect which transpiles to an OpenCL kernel might work.
  - TheRealKing 8y ago
    In modern history, I still use (modern) Fortran (as well as MATLAB, Python, and R). It does what I need to do, efficiently with minimal coding.

AzzieElbab 8y ago

We have reached the point where saying "pick the right tool..." usually means completely opposite. PHP for web servers, python for large applications, go or anything c like for business logic

olavk 8y ago

There is a saying about the band Velvet Underground - that not many people bought their records, but everyone who bought one went out and started their own band. I think Lisp is kind of similar - its influence is much larger than its fame, but ever language designer is inspired by it to some extent. I do think Lisp kind of missed the boat when a wave of dynamic languages (Perl, Python, Ruby) went mainstream. Today the most exciting developments in language design seem to be around static type systems, type inference etc.

yogthos 8y ago
I think there is a lot of exciting stuff happening in Racket and Clojure around contract systems right now with things like Racket Contracts and Clojure Spec. This approach provides an appealing alternative to static typing because it focuses on semantic correctness while static typing focuses primarily on self consistency. In my view the former is much more valuable because ultimately we want to know that the code is doing what was intended, and static typing doesn't appear to be an effective tool for encoding intent.
- hashkb 8y ago
  I think Typed Racket has more buzz than contracts... maybe even seen as a replacement/evolution by many.
  - Avshalom 8y ago
    I can't swear to this but I think they've gone basically hand in hand. I'm pretty sure I read a few Racket posts about how they use contracts to keep a safe interface between typed and untyped code and just in general between languages.
    - samth 8y ago
      Yes, this is correct. The recent explosion of work on gradual typing (including Typed Racket) was enabled by the development of contracts.
- 8y ago

Areading314 8y ago

As elegant as lisp is, I think the flexibility just kills its usability on teams, while more rigid languages like Java, Go or C see massive adoption. Language expressiveness is rarely a real bottleneck, compared to debugging and understanding program flow and getting new contributors up to speed. Separately I think each lisp has an individual fundamental flaw that makes it unusable for production, eg clojure having terrible error handling, common lisp being case insensitive, scheme having a minimal standard library, racket being fairly slow, etc.

junke 8y ago
I tend to find it very easy to debug CL code. In fact, CL is designed to be introspective: "inspect", "trace", "break", "disassemble" and "step" are part of the language. Also, Common Lisp is case sensitive.
- workinfunk 8y ago
  You can hardly fault newcomers for not understanding the finer nuances when everything they type automatically gets uppercased.
  - junke 8y ago
    That's why I try to correct people when possible and point newcomers to Practical Common Lisp (Seibel).
jandrese 8y ago
Is case insensitivity really a flaw? Other than some weak namespace hinting (constants in ALL CAPS for example) what is the flaw of your language being case insensitive for everything but string handling? Would C be ruined if people could type: If ( expression ) { effect }; I'm also becoming unsold on file system case sensitivity as well. Should "Income Report 2018Q1" really be different than "Income report 2018Q1"? How often is this a desirable feature? The only reason I have now for maintaining case sensitivity is that in the Unicode world casefolding is an absolute nightmare and no language or file system built today can afford to not use Unicode.

ectospheno 8y ago

I find the Lisp variants to be lovely right up until I want to package and ship an actual application to a normal human being. Then they are quite clumsy. I find the same to be true of a number of other more recently popular high level languages. I've always assumed it was because graduate students never got to the point where they ship anything.

lgessler 8y ago
I haven't tried doing this in other Lisps, but it's really easy if you're using Clojure's most popular build tool, Leiningen. For JVM Clojure you can produce a single .jar ("uberjar") that you can run as you'd expect, and for ClojureScript, Google's Closure Compiler is used to output a single minified and tree-shaken JS file.
- Oreb 8y ago
  The parent poster was talking about shipping "an actual application to a normal human being". A .jar file is a non-starter, as normal human beings these days don't have Java installed and have no desire to install it. Don't get me wrong: I love Clojure, and it's currently the language that pays my bills. But I'd never consider using it to build an app for end users (except possibly a ClojureScript React Native app for mobile devices).
vindarel 8y ago
How so ? In CL you can build an executable. (https://lispcookbook.github.io/cl-cookbook/scripting.html#building-a-self-contained-executable https://lispcookbook.github.io/cl-cookbook/scripting.html#bu...) It's large-ish (because it ships an interpreter, a debugger that we can access from outside, etc), but it has a near-zero startup time, and it works for web apps with their embedded web server.
- claudiawerner 8y ago
  For ages I couldn't figure out how exactly I would be able to use Quicklisp libraries to create something, and then create an executable from the source code. I think I ended up giving up, after only finding a couple of blogs that were using ancient versions of everything involved. I remember even having trouble loading some of the packages. Overall my experience with writing CL for myself was fun and interesting, but my experience writing it to be used by other people was anything but. There seemed to be too much rubbish to confuse me, as opposed to something either like C (header files + corresponding source) or Python (a directory with an __init__.py is something you can import). The module system has also turned me away from Scheme; the point where different Schemes all implement say R6RS but they have different module systems, or even different ways of converting an integer to a string, varying support for SRFIs (the regex SRFI is nearly four years old and the only Scheme to implement that is Chibi) and the completely variable module system, where nothing at all is consistent, none of the modules work in another Scheme, even if you were to extract the code. You want something from Chicken to use in Guile? Too bad, you have to manually do all the work yourself, despite the fact that both modules are supposedly written in standard Scheme. The module situation eventually pushed me to write my own Lisp, but I realised that unless I were to go ahead and implement all the things I wanted, everything I would miss, I'd be in just the same place, reminded of that xkcd comic in which "there are now 15 competing standards". So I gave up on doing just that, and now I'm literally writing a Lisp compiler and then operating system with a friend, because for me it's a lot less frustrating to say "that doesn't work on my operating system" than it is to say "that doesn't work in this guy's Scheme". It wasn't supposed to make sense.

carapace 8y ago

All these comments about how Lisp missed the boat, and you're all wrong. We missed the boat. Lisp has been going strong among it's own the whole time. It's not their fault we're all so stubborn, punishing ourselves over and over again like some sick masochists. I got mad when I finally learned Lisp and realized how much time and effort has been wasted over the years because we're too lazy, ignorant, or foolish to use it. (As a tangent, it's simply incredible how deliberately ignorant programmers can be. I've worked with guys getting paid to write software who didn't know who Alan Kay is, or had never heard of Prolog. What's worse is they're not ashamed of their ignorance. Can you imagine a physicist who had never heard of Newton?) I don't use Lisp, but I have the grace to admit that that's a personal failing. (I joke that Python's only major problem is that it ruins you for other languages... I just can't quit that sweet syntax. Although Python 3 is a train wreck IMHO. Don't get me started.) I would switch to Lisp, but I found something even better, which brings me to my actual, non-ranty, point: There's a language even better than Lisp. It's called Joy and it combines all the best parts of Lisp with the best parts of Forth to make an enormously simple system for describing software. It's a purely functional, stack-based, "concatinative" language that turns out to be good for "Categorical" programming: programs in Joy can be "instantiated" over different categories to generate different kinds of computations. I only have one good reference for "Categorical" programming right now: a paper and talk by Conal Elliott "Compiling to Categories" February 2017 http://conal.net/papers/compiling-to-categories/ http://conal.net/papers/compiling-to-categories/ He's working in Haskell, not Joy, but he's describing the idea: from one piece of code you can get calculations, dataflow diagrams, circuit descriptions, type signatures (inference), derivatives, etc., by implementations of the categories for each kind.

phlakaton 8y ago

I'm disappointed by this paper. The general review of the language is of the sort that you can find a dime a dozen on LISP enthusiast sites--so this paper does little to effectively recruit existing programmers to the cause. On the other hand, the references to current bioinformatics projects using LISP is helpful as an index, but provide few details to convince a biologist that they're worth checking out--so I'm not convinced on that side either. The "key point" of a forthcoming BioLisp is embarrassingly glossed over. I think the paper would be much stronger if it focused on that BioLisp aspect, talking about: * What is BioLisp? A mere web portal pointing to various resources across various languages? An attempt to create a curated installation of version-compatible libraries for a specific LFL? An attempt to develop a fully-integrated framework for that LFL?? (I think Sage for Python might be a useful model here, but if you were trying to build that, you'd have to start by picking your LFL, which the authors at present seem unwilling to advocate. That in itself demonstrates one of the main liabilities at present in adopting LISP--the community is fractured between Racket, Scheme, Clojure, and various implementations of Common LISP--often to the extent that they argue over whether their respective opponent languages even _deserve_ to be called LISPs. And, of course, in this forum we even have Arc fanatics to add to the lot. ;-) ) * What challenges face biologists attempting to use LISP in their work? How does BioLisp help to address those? ("Your language lacks macros" is not an argument at a sufficiently detailed level, I'm afraid. Rather, actually show them a DSL built in LISP that they will desperately want!) * For that matter, what challenges face biologists attempting to use R/Python/Julia/etc for their work? How does BioLisp help to address those? * What promising pieces could BioLisp be built upon? What pieces are missing? What pieces need modernization? By all means, I encourage scientists and LISP enthusiasts to make the case that biologists should/could be using LISP--but make sure you actually make the case!

oldandtired 8y ago

What are the pros and cons of Lisp (or any other language for that matter)? In my undergraduate days (oh so many decades ago), I was introduced to Lisp. For some of the problems we were solving, it was the perfect tool. For others, a different language could be used. The pros of Lisp (in all its varieties) are in many ways the cons of Lisp. The macro system is both beneficial and antagonistic to the development of sane programs. Used judiciously, macros may life writing code much easier. However, it is not uncommon for Lisp programmers to write macros without consideration for those who come after them. This is, of course, true for any language. There are some major systems out there that are written in Lisp and when trying to change them (in a maintenance way), it is just as painful as in pretty much any other language. The competent and talented Lisp programmers is imbued with the same arrogance that C++ competent and talented programmers have. Tricky programming is fine if it is fully documented and the reasoning for the tricks is fully explained. Lisp and its variations have much to offer the programming community. But the way programs are written quickly diminishes the effectiveness of the language for the larger community. Each language has features that can be of use to each of us. they are tools we can use to solve the problems we are facing. No one language is suitable for all problems. If you have that attitude then you only see hammers and nails.