5 ms·

We've also been using Lisp in production. It has gone wonderfully, and has allowed some of the most simultaneously maintainable, resilient, and efficient code I

We've also been using Lisp in production. It has gone wonderfully, and has allowed some of the most simultaneously maintainable, resilient, and efficient code I've seen in my career. It's also attractive differentiator to some programmers who don't want to drone away writing bulk-Java day-in and day-out. A key to running Lisp in production is to step away from thinking of it as a lone-wolf bionic-suit hacker language, and to invest in developing a style guide, code review guidelines, CI/CD, and a culture of training new programmers to write Lisp. I've found that you typically don't have to hire Lisp specialists. As long as you find good programmers, they pick up Lisp just fine in just a few short weeks. The sibling comments are right though: Lisp's reputation is unfortunately affected by so, so many myths that it actually stunts the ecosystem in certain ways. The most rampant myth these days is that (somehow) macros automatically imply doom of a codebase after a certain size or contributor count. Classic myths from decades ago, like "Lisp is interpreted and slow", still echo from time to time too. Lisp's extensive, half-century history is a strength but also clearly a (social) weakness. A lot of myths come from good-intentioned speculation of the supposed consequences of certain features or aspects of Lisp, and not from actual experience using it.

zshrdlu 5y ago
As I like to say, Lisp's reputation precedes it more so than any other $TECH.
peebz 5y ago
That's really interesting. Can I ask what you use it for and if you've come across any limitations or problems using Lisp?
- reikonomusha 5y ago
  For building optimizing compilers for quantum computers. The biggest hurdle has been difficulties with deployment. Open-source Lisp implementations mostly[1] only let you build an executable, and not things like a C ABI-compatible shared library. Commercial Lisp implementations are a lot better in this regard. Even if your OK with building an executable, there are limited options in terms of "enterprise integration", especially in the arena of databases and RPC. Google only very recently released support for gRPC [2], for example, and it's not a first-class citizen in Google's gRPC offering. Most hobbyist Lisp hackers don't exactly care to build (and more importantly maintain) a bunch of open source stuff for enterprise use. [1] There is a Lisp implementation called "ECL" that compiles Lisp to C, but it's lacking in many other dimensions. [2] https://github.com/qitab/grpc https://github.com/qitab/grpc
  - yvdriess 5y ago
    Cool! I used CL for the same purposes in my PhD almost a decade ago. Interesting to see that others independently have been gravitating towards CL. I had a top-level pattern language with a dataflow-like visual representation, using the LispWorks UI lib to make the editor. That pattern language compiled down to a Measurement Calculus representation (one-way QC, so not the usual circuit model) in s-expr form, which I could both execute with my QVM and do optimizing transformations on. I had a CL QVM was initially CL, but it was quickly replaced with a C + OpenMP version using the same s-expr input.
    - reikonomusha 5y ago
      Parallelization and SIMD are pretty easy for numerical code in CL now; I don't think this was the case 10 years ago. LPARALLEL is a nice small library for core-parallelism, and CL-MPI is good for MPI/cluster parallelism. This QVM [1], a pure/density-state simulator, can use either. [1] MPI version: https://github.com/quil-lang/qvm/tree/master/dqvm https://github.com/quil-lang/qvm/tree/master/dqvm
agumonkey 5y ago
Are you using SBCL ? also i'm curious what CI/CD platform you used, or maybe you rolled your own ?
- reikonomusha 5y ago
  SBCL, making use of GitLab, Quicklisp dists, Docker, Conda, and Artifactory.
  - agumonkey 5y ago
    Oh I wish I could see your lab operate :)
ScottBurson 5y ago
> Classic myths from decades ago, like "Lisp is interpreted and slow", still echo from time to time too. Pretty funny, considering how much faster CL is than, say, Python -- never mind Ruby.
- srean 5y ago
  The decline of Lush [0] as a popular data-science language saddens me. One could use it as a dynamically typed interactive language with nice support for nd array manipulation, excellent support for calling C functions. Once satisfied with correctness of the implementation one could then compile it down to a statically typed C for speed. It ran circles around byte compiled Python in speed. Two of Python's relative advantages in the data-science domain were (i) nicer array manipulation syntax and (ii) one could build the non data-sciency part of the application in Python as well thus avoiding a proliferation of tooling and languages necessary. Regarding Ruby, even TCL was faster than non JITed Ruby. That was funny because TCL used to be considered the slowest possible runtime (among professionally used runtimes) [0] http://lush.sourceforge.net/ http://lush.sourceforge.net/
  - phillc73 5y ago
    There are some other active Lisp efforts in the data science space, such as cl-ana.[1] [1] https://github.com/ghollisjr/cl-ana https://github.com/ghollisjr/cl-ana
- Mikeb85 5y ago
  > Python -- never mind Ruby Lol, with the new JIT compilers and TruffleRuby, Ruby is blowing Python out of the water performance-wise. TruffleRuby approaching Nodejs speed for some tasks. But yes, CL is fast. Faster than any other dynamic language, approaching Java speed (and Java is shockingly quick considering it has a GC).
  - DerArzt 5y ago
    It's always cool when we can achieve a speed up in something, but I fear that we may have reached the point of Ruby becoming mostly irrelevant to the larger market. Python, thanks mainly to it's data science libraries, has taken it's place as the go to non-JavaScript scripting language. Unless Ruby comes up with some killer feature like it had with Ruby-on-Rails back in the day, I doubt that the speed up of Ruby will be what revives it for most people. All of that said, I would love for Ruby to usurp Python from it's thrown!
hngpt2 5y ago
I have used C, Java and even Perl extensively for the same "one-off" (like small command-line tools) and "everyday" kind of programming. But it's in those "everyday" kind of programming that I have found most value. The reason is that I often end up doing the same sort of tasks over and over again. As such, I don't want to waste time doing something trivial just because it has to be done so often, but to do something more productive. For some languages, I could not find much of a reason to use them for such tasks. When in Java I used StringBuilder instead of simply concatenating the values. If I had to add something to the existing string in C, I used memcpy and memset. It's easy enough to understand why. So why would I do the same thing again in Lisp? For the same reasons I did in C?
- p_l 5y ago
  Personally, it used to be the biggest issues in tackling scripting tasks in Lisp in my experience was slow startup (mind you, we're talking about running from 4200 rpm spinning rust). There are people successfully using it for such kind of work, and generally it used to be common in environments with more Lisp support. These days my biggest issue is that I kind of lost the need for anything more than Bash for one-off tools, and I do it rarely enough that the incremental benefit faces big hurdle of dealing with development-antagonistic environment (mac)
vbezhenar 5y ago
How do you convince good programmers to spend their life writing lisp? It's very hard to find lisp work after your company, so it kind of wasted years.
- Mikeb85 5y ago
  Programming is more than learning languages...
  - oblio 5y ago
    For many journeymen developers, the choice of programming language is important. There are many, many companies out there that do look for people with N years of experience with language X and not every developer out there is in a place or at a skill level where they can afford to be picky (the average HN developer isn't representative of this population).
    - Mikeb85 5y ago
      Ah yes, the 'ol "need 10 years of experience in a 5 year old technology"...
- pmarreck 5y ago
  imagine telling an electrician what tools they need to use
- reikonomusha 5y ago
  The same way I convince them to learn an employer's large code base that will be useless to them after they leave too: by paying them. That's a real, non-facetious answer. Another facet to the answer is that many people see value in learning a different language to help give them perspective on the existing ones they already know. I generally don't consider it a net loss to ever learn a new language.
dreamcompiler 5y ago
This has been exactly my experience too. It's easy to bring good programmers up to speed in Lisp. It's sometimes harder to bring existing Lisp programmers onto a team. Let me explain: Some Lisp programmers (not the majority, but some) learned it on their own and it became their personal hobby-project language because it's so powerful. When those programmers join a team they bring bad habits with them. Bad habits like using lists for every data structure, overusing reader macros, not using packages properly, writing their own functions for things covered by well-tested cross-platform quicklisp libraries, etc. Programming in Lisp is very different than software engineering in Lisp. Of course that's true in any language, but Lisp has a lot more "shoot your team in the foot" possibilities than C, and you absolutely need to learn to avoid those patterns or formally communicate your use of them in order to make Lisp effective on a software engineering team. Rather than depending on the language itself to enforce good software engineering discipline, with Lisp you must create out-of-band agreements with your team about how to write good software. This is not a bad thing; it's something all SWE teams should do anyway. But Lisp is especially intolerant of neglecting this aspect of SWE.
- xedrac 5y ago
  > Lisp has a lot more "shoot your team in the foot" possibilities than C, and you absolutely need to learn to avoid those patterns or formally communicate your use of them in order to make Lisp effective on a software engineering team. I love Lisp for its adventurous and fun nature, but the above statement is a huge negative for real engineering work on a team. Any time you can rely on your tools or language prevent these types of issues, that's a win. This became especially evident to me when my team started using Rust. Code reviews are now almost purely about logic and design.
  - reikonomusha 5y ago
    You don't shoot yourself in the foot in the same way one does in C. With carelessness, you more... suffocate yourself with your own towers of (probably useless) abstractions. It's like having a power drill when all you knew was a screw driver... and then drilling holes and driving screws every chance you get without regard to a larger structure. Unlike C, the powerful (and safe) language features are a boon to software engineering projects, as long as you're doing was dreamcompiler said: software engineering and not "just programming". I disagree with the thesis that the more restrictive one is, the better it is for software engineering. We've empirically disproved this time and time again with every supposedly "simple" language evolving way, way past its original design goals. Python, Go, and even Scheme are great examples.
deleted 5y ago
[deleted]

zshrdlu 5y ago

As I like to say, Lisp's reputation precedes it more so than any other $TECH.

peebz 5y ago

That's really interesting. Can I ask what you use it for and if you've come across any limitations or problems using Lisp?

reikonomusha 5y ago
For building optimizing compilers for quantum computers. The biggest hurdle has been difficulties with deployment. Open-source Lisp implementations mostly[1] only let you build an executable, and not things like a C ABI-compatible shared library. Commercial Lisp implementations are a lot better in this regard. Even if your OK with building an executable, there are limited options in terms of "enterprise integration", especially in the arena of databases and RPC. Google only very recently released support for gRPC [2], for example, and it's not a first-class citizen in Google's gRPC offering. Most hobbyist Lisp hackers don't exactly care to build (and more importantly maintain) a bunch of open source stuff for enterprise use. [1] There is a Lisp implementation called "ECL" that compiles Lisp to C, but it's lacking in many other dimensions. [2] https://github.com/qitab/grpc https://github.com/qitab/grpc
- yvdriess 5y ago
  Cool! I used CL for the same purposes in my PhD almost a decade ago. Interesting to see that others independently have been gravitating towards CL. I had a top-level pattern language with a dataflow-like visual representation, using the LispWorks UI lib to make the editor. That pattern language compiled down to a Measurement Calculus representation (one-way QC, so not the usual circuit model) in s-expr form, which I could both execute with my QVM and do optimizing transformations on. I had a CL QVM was initially CL, but it was quickly replaced with a C + OpenMP version using the same s-expr input.
  - reikonomusha 5y ago
    Parallelization and SIMD are pretty easy for numerical code in CL now; I don't think this was the case 10 years ago. LPARALLEL is a nice small library for core-parallelism, and CL-MPI is good for MPI/cluster parallelism. This QVM [1], a pure/density-state simulator, can use either. [1] MPI version: https://github.com/quil-lang/qvm/tree/master/dqvm https://github.com/quil-lang/qvm/tree/master/dqvm

agumonkey 5y ago

Are you using SBCL ? also i'm curious what CI/CD platform you used, or maybe you rolled your own ?

reikonomusha 5y ago
SBCL, making use of GitLab, Quicklisp dists, Docker, Conda, and Artifactory.
- agumonkey 5y ago
  Oh I wish I could see your lab operate :)

ScottBurson 5y ago

> Classic myths from decades ago, like "Lisp is interpreted and slow", still echo from time to time too. Pretty funny, considering how much faster CL is than, say, Python -- never mind Ruby.

srean 5y ago
The decline of Lush [0] as a popular data-science language saddens me. One could use it as a dynamically typed interactive language with nice support for nd array manipulation, excellent support for calling C functions. Once satisfied with correctness of the implementation one could then compile it down to a statically typed C for speed. It ran circles around byte compiled Python in speed. Two of Python's relative advantages in the data-science domain were (i) nicer array manipulation syntax and (ii) one could build the non data-sciency part of the application in Python as well thus avoiding a proliferation of tooling and languages necessary. Regarding Ruby, even TCL was faster than non JITed Ruby. That was funny because TCL used to be considered the slowest possible runtime (among professionally used runtimes) [0] http://lush.sourceforge.net/ http://lush.sourceforge.net/
- phillc73 5y ago
  There are some other active Lisp efforts in the data science space, such as cl-ana.[1] [1] https://github.com/ghollisjr/cl-ana https://github.com/ghollisjr/cl-ana
Mikeb85 5y ago
> Python -- never mind Ruby Lol, with the new JIT compilers and TruffleRuby, Ruby is blowing Python out of the water performance-wise. TruffleRuby approaching Nodejs speed for some tasks. But yes, CL is fast. Faster than any other dynamic language, approaching Java speed (and Java is shockingly quick considering it has a GC).
- DerArzt 5y ago
  It's always cool when we can achieve a speed up in something, but I fear that we may have reached the point of Ruby becoming mostly irrelevant to the larger market. Python, thanks mainly to it's data science libraries, has taken it's place as the go to non-JavaScript scripting language. Unless Ruby comes up with some killer feature like it had with Ruby-on-Rails back in the day, I doubt that the speed up of Ruby will be what revives it for most people. All of that said, I would love for Ruby to usurp Python from it's thrown!

hngpt2 5y ago

I have used C, Java and even Perl extensively for the same "one-off" (like small command-line tools) and "everyday" kind of programming. But it's in those "everyday" kind of programming that I have found most value. The reason is that I often end up doing the same sort of tasks over and over again. As such, I don't want to waste time doing something trivial just because it has to be done so often, but to do something more productive. For some languages, I could not find much of a reason to use them for such tasks. When in Java I used StringBuilder instead of simply concatenating the values. If I had to add something to the existing string in C, I used memcpy and memset. It's easy enough to understand why. So why would I do the same thing again in Lisp? For the same reasons I did in C?

p_l 5y ago
Personally, it used to be the biggest issues in tackling scripting tasks in Lisp in my experience was slow startup (mind you, we're talking about running from 4200 rpm spinning rust). There are people successfully using it for such kind of work, and generally it used to be common in environments with more Lisp support. These days my biggest issue is that I kind of lost the need for anything more than Bash for one-off tools, and I do it rarely enough that the incremental benefit faces big hurdle of dealing with development-antagonistic environment (mac)

vbezhenar 5y ago

How do you convince good programmers to spend their life writing lisp? It's very hard to find lisp work after your company, so it kind of wasted years.

Mikeb85 5y ago
Programming is more than learning languages...
- oblio 5y ago
  For many journeymen developers, the choice of programming language is important. There are many, many companies out there that do look for people with N years of experience with language X and not every developer out there is in a place or at a skill level where they can afford to be picky (the average HN developer isn't representative of this population).
  - Mikeb85 5y ago
    Ah yes, the 'ol "need 10 years of experience in a 5 year old technology"...
pmarreck 5y ago
imagine telling an electrician what tools they need to use
reikonomusha 5y ago
The same way I convince them to learn an employer's large code base that will be useless to them after they leave too: by paying them. That's a real, non-facetious answer. Another facet to the answer is that many people see value in learning a different language to help give them perspective on the existing ones they already know. I generally don't consider it a net loss to ever learn a new language.

dreamcompiler 5y ago

This has been exactly my experience too. It's easy to bring good programmers up to speed in Lisp. It's sometimes harder to bring existing Lisp programmers onto a team. Let me explain: Some Lisp programmers (not the majority, but some) learned it on their own and it became their personal hobby-project language because it's so powerful. When those programmers join a team they bring bad habits with them. Bad habits like using lists for every data structure, overusing reader macros, not using packages properly, writing their own functions for things covered by well-tested cross-platform quicklisp libraries, etc. Programming in Lisp is very different than software engineering in Lisp. Of course that's true in any language, but Lisp has a lot more "shoot your team in the foot" possibilities than C, and you absolutely need to learn to avoid those patterns or formally communicate your use of them in order to make Lisp effective on a software engineering team. Rather than depending on the language itself to enforce good software engineering discipline, with Lisp you must create out-of-band agreements with your team about how to write good software. This is not a bad thing; it's something all SWE teams should do anyway. But Lisp is especially intolerant of neglecting this aspect of SWE.

xedrac 5y ago
> Lisp has a lot more "shoot your team in the foot" possibilities than C, and you absolutely need to learn to avoid those patterns or formally communicate your use of them in order to make Lisp effective on a software engineering team. I love Lisp for its adventurous and fun nature, but the above statement is a huge negative for real engineering work on a team. Any time you can rely on your tools or language prevent these types of issues, that's a win. This became especially evident to me when my team started using Rust. Code reviews are now almost purely about logic and design.
- reikonomusha 5y ago
  You don't shoot yourself in the foot in the same way one does in C. With carelessness, you more... suffocate yourself with your own towers of (probably useless) abstractions. It's like having a power drill when all you knew was a screw driver... and then drilling holes and driving screws every chance you get without regard to a larger structure. Unlike C, the powerful (and safe) language features are a boon to software engineering projects, as long as you're doing was dreamcompiler said: software engineering and not "just programming". I disagree with the thesis that the more restrictive one is, the better it is for software engineering. We've empirically disproved this time and time again with every supposedly "simple" language evolving way, way past its original design goals. Python, Go, and even Scheme are great examples.

deleted 5y ago

[deleted]