38 ms·

I'm going to take the opposite position that there is much less special about Lisp than people think. Like I have met so many programmers that travel around li

I'm going to take the opposite position that there is much less special about Lisp than people think. Like I have met so many programmers that travel around like itinerant martial artists looking for true functional programming and they never find it. To be specific, you can work all of the examples in Graham's On Lisp in Python except for one of the last chapters where he implements continuations that really need macros -- but this is basically the async/await facility that Python already has. The other examples use macros for performance but work fine with just plain functional programming. React with hooks is an example of that kind of system at work -- the JSX transformation is a very simple shim you can put in front of the JS compiler and the hooks themselves are the kind of trick that On Lisp teaches you how to do. On Lisp doesn't use the kind of tree-walking macros that really are unique to Lisp. And... the techniques in the Dragon Book for writing compilers are the real magic. If looking to Lisp as an old shiny keeps you from learning how to write compilers, it is holding you back. I think the homoiconic thing leads people astray. There's a real tension that, for performance reasons, mainstream compilers aren't extensible, for instance you might want to write a unless(X) {...} => if(!X) {...} control structure in a new language and for something in Java that is really a production rule in the grammar, maybe a class to represent the unless block, and a rewriting rule that gets applied to the AST. If the compiler was designed to be easily extensible that would be less code than the POM file for the project. But it's not so it isn't. Many things hold us back. The industry has been so traumzatized by slow compiles that trading speed for extensibility doesn't sell to the people who create languages. Also once you have the sophistication to make things like parser generators you know how to get things done with the terribly unergonomic parser generators we have and don't have a lot of empathy for all the programmers out there who might be using parser generators if any of them were built as if usability matters.

pjmlp 3mo ago
Except that CPython still misses on (compile .....) part, indeed holding the industry back.
- PaulHoule 3mo ago
  it compiles to bytecode but the bytecode engine is not that fast. i had a phase when I was using PyPi a lot for branchy "old AI" kinds of workloads and felt it was an easy win but since then it has been either numpy or PIL or pytorch doing the heavy lifting or scripty stuff like uploading files to S3 where performance doesn't matter a lot. I will grant that Common Lisp can be compiled to run amazingly quickly!
  - BoingBoomTschak 3mo ago
    What he probably meant is that compilation is available at runtime to the user, allowing you to JIT stuff at will. This is how https://github.com/marcoheisig/Petalisp#why-is-petalisp-written-in-common-lisp https://github.com/marcoheisig/Petalisp#why-is-petalisp-writ... and https://github.com/numcl/specialized-function https://github.com/numcl/specialized-function can exist.
  - pjmlp 3mo ago
    In Lisp (compile ...) produces machine code, with various optimisation flags available, depending on the implementation.
- guenthert 3mo ago
  Not sure about that. I rather think it's the very dynamic nature of Python (e.g. at run time one can add members to classes) which limits performance gains achievable by compilation. Iirc PyPy got to some five fold performance gain against CPython. Common Lisp, as an example, makes some deliberate flexibility (and elegance) vs. performance trade-offs. I'd think to speed Python up more than PyPy could, one would have to sacrifice some of its features (some of those might not be all that popular or even desirable). Nah, Python has its place. If its performance isn't adequate for the problem at hand, rewrite the bottlenecks in C++ and be done. LLMs should make that easy, no?
  - pjmlp 3mo ago
    The dynamic excuse always misses out fully dynamic languages like Smalltalk, Self, among others have achieved, and how it got further exploited into JavaScript engines. However CPython isn't even half way there to either Papyrus or GraalPy. The community attitude to rewrite into C, C++, Rust, Zig, whatever, is exactly why Python will never be like other dynamic languages, and the dynamism excuse will always be used. Thankfully at least GPU vendors see it otherwise, so if nothing else at least it will get JITs that speak GPU machine code.
iLemming 3mo ago
There's nothing "special" about Lisp and Lisp dialects, yes. Similar features can be or already have been implemented in other languages. Yet, after touching, using and experiencing working with a bunch of different stacks, I cannot simply ignore the enormous pragmatic level of Lisps. Working with Clojure is an absolute delight. It strips down all the dogma and let's you deal with the "business logic" as if you're cooking steak using no BS ingredients - meat is meat, herbs are real, stove is hot. Why would I ever choose bash for writing anything slightly more complex than simple redirection, when I can do things in way better fashion with babashka. Why would I wrestle a YAML CI pipeline that only fails on push, when I can drive the whole thing from a babashka task file, run each step locally in the REPL, and actually debug it? Why would I ever deal with Lua, if I can't even format it for "readability" - no matter how I do it, it just looks darn ugly, and luafmt often makes it worse. Why, if I can just slash down dozen lines of Lua boilerplate compressing it into a three-liner Fennel macro? With Fennel, I can interactively poke through elements of my WM through Hammerspoon on Mac, and that's just bananas. Why would I ever deal with JSON, when EDN is almost twice as compact and far more readable - I can align things and treat data as a literal table. Besides, I can group, sort, filter, slice, dice, salt & pepper that data easily, without ever leaving my trusted editor. Why would I choose to build a web-scraper in Python, when I can use nbb driving Playwright and go through selectors interactively, directly from my editor, as if it is a devtools console. And I don't even have to restart anything, deal with state changes, etc. How can I abandon Emacs where I can just open a scratch buffer, type some Elisp and change the behavior of my editor, my WM, my OS and even things on remote computers. No other text editing environment works the way Emacs does - nothing even comes close. It feels like playing a video game, where my controller in my editor. Why would I write Flutter UIs in Dart, fighting the widget-tree ceremony and endless build() boilerplate, when ClojureDart lets me express the same tree as plain data and hot-reload it interactively? The layout is just nested maps and vectors. Why would I reach for C when I need to embed a small, fast scripting layer. Text parsing alone would be a regex nightmare elsewhere. Why would I bolt a templating engine onto HTML strings, when Hiccup makes markup just vectors - so my views compose, filter, and generate like any other data, no special templating DSL to learn And with all sorts of different runtimes and dissimilar Lisp dialects, it still feels as if you're working with the same language. The mental overhead when switching is so negligible. While switching between just JS and TS - which are supposed to be of the "same family" - feels quite annoying. Despite the fact that I've put years into those - far longer than any Lisp I've ever used. Sure, nothing special about Lisp at all. Except that practicing Lisp can actually make you a polyglot. You'd realize that it isn't syntax that makes a programming language, but runtime and semantics do. After years of dealing with different PLs, I lost a preference for one specific language - I'd choose the runtime best suitable for the task, and then see if I can bolt Lisp on top of it. And these days, it feels like there isn't a platform left where you can't meaningfully do things via Lisp.
BoingBoomTschak 3mo ago
You're really misrepresenting Lisp macros here, being able to rewrite syntax is only one (admittedly important) facet of these. The real thing with CL is that the entire language is available during parsing and macro expansion and that users can hook into these steps to influence them. No artificial limitations like you get in C++'s consteval, you can do everything and anything without having to use a crappy DSL to do it. Also, I've never found SBCL slow to compile. Have you?
- PaulHoule 3mo ago
  (1) SBCL is fine in terms of speed so far as I am concerned. But wherever you have software engineering in the large, especially involving C++, you have 40 minute builds and a split between smart people who think that's a problem and smart people who don't. (2) The trouble with those balls-to-the-walls macros is that systems that use them tend to be "write only", like some guy writes 4000 lines of Lisp that do the work of 80,000 lines of C++ and then he moves on and there is never a version 2.0 but the system gets rewritten in C++, supposedly for performance, but really because they couldn't find anybody who could maintain it. It is really pleasing to see how little code is in Lisp systems from the golden age of AI but... when Yahoo! bought Graham's startup they kept the customers but threw away the code. You are more likely to be sustainable if you get two people to write 12,000 lines in something like typescript or ML that uses Dragon Book technology to the same end.
patterns 3mo ago
While I (think) understand the sentiment (ergonomics and practicality), it is still worth studying Lisp for a number of reasons. In particular, I would recommend to everyone to study Lisp 1.5 [1] to appreciate how an entire universe of programming languages can be bootstrapped with just a few primitives and equations. It is true that, today, we have a vast array of impressive tools at our disposal, from parser combinators and generators, code generators to entire language workbenches with projectional editing capabilities. However, if one were to design a language for any reason, having a deep understanding of the expressiveness of computational models such as the lambda calculus would certainly be an "advantage" (especially for those who end up having to use the language): A Lisp/Scheme is as close to interactive lambda calculus as it gets. From there on, one can learn about implementing different evaluation strategies, scoping rules and continuations (William Byrd's "The Most Beautiful Program Ever Written" [2] also to mind). Now, I am not sure whether homoiconicity tends to lead people astray. It is true that is it not strictly necessary to make a language extensible (e.g., Smalltalk has no macros and is very extensible due to its powerful meta-object protocol and an elegant syntax for closures), but it's still worth studying the concept. For example, writing a metaintpreter in Prolog [3] is surprisingly easy because of its homoiconicity. [1] https://softwarepreservation.computerhistory.org/LISP/book/LISP%201.5%20Programmers%20Manual.pdf https://softwarepreservation.computerhistory.org/LISP/book/L... [2] https://www.youtube.com/watch?v=OyfBQmvr2Hc https://www.youtube.com/watch?v=OyfBQmvr2Hc [3] https://www.metalevel.at/acomip/ https://www.metalevel.at/acomip/
loose-cannon 3mo ago
I thought Scheme was just an educational tool - a minimal language that allows the student to explore language construction & design relatively quickly. I'm not sure I understand your comment about homoiconicity leading people astray. Isn't that the characteristic feature of lisp? Doesn't that dramatically simplify generating code (in principle)?
- Jtsummers 3mo ago
  > I thought Scheme was just an educational tool That's one use. It's also used in commercial and open source systems. Guile, for instance, has been used as an extension language (like lua and others) in many projects.
- jrapdx3 3mo ago
  I find it interesting that a number of PLs were initially developed as educational or research tools but later expanded for general use. Some examples are Pascal, Scheme, Prolog. AFAIK many development languages are used in education so the distinction is rather fuzzy. Several Scheme implementations, e.g., Chicken, Chez, Gambit, are capable of producing almost any type of program. I'm most familiar with Chicken Scheme. Comes with a large "standard" library plus a broad range of available extensions. Its easy to use FFI adds to productivity. My experience anyway. Of course, mileage varies widely among users of any language.
  - deleted 3mo ago
    [deleted]
  - loose-cannon 3mo ago
    Yea sorry. I didn't mean to criticize or diminish its value by saying it's just an educational tool. It's a powerful tool that lets you do some advanced things quickly. That's totally valuable outside of an educational setting. And I didn't mean to imply that there's any limitation to its use. I just don't understand the criticism in the post I was replying to.
kazinator 3mo ago
> the techniques in the Dragon Book for writing compilers are the real magic. Can confirm; got a few hints from the 1988 edition when working on the TXR Lisp compiler. Oh, and also when implementing regexes, not to forget! However, the Lisp compiler has its own magics that are not exactly covered in the Dragon Book. Some really nice way of doing things.
hajile 3mo ago
I might buy this if those languages were actually faster. Instead, implementations of scheme with no commercial backers run circles around scripting languages with huge budgets. Common Lisp can match and exceed languages like Java without the backing of a megacorp. Carp Lisp is basically identical to C in performance and old PS games were shipped in a proprietary dialect (GOAL) and had very good performance. Lisp has proven itself capable great performance at every abstraction level.

pjmlp 3mo ago

Except that CPython still misses on (compile .....) part, indeed holding the industry back.

PaulHoule 3mo ago
it compiles to bytecode but the bytecode engine is not that fast. i had a phase when I was using PyPi a lot for branchy "old AI" kinds of workloads and felt it was an easy win but since then it has been either numpy or PIL or pytorch doing the heavy lifting or scripty stuff like uploading files to S3 where performance doesn't matter a lot. I will grant that Common Lisp can be compiled to run amazingly quickly!
- BoingBoomTschak 3mo ago
  What he probably meant is that compilation is available at runtime to the user, allowing you to JIT stuff at will. This is how https://github.com/marcoheisig/Petalisp#why-is-petalisp-written-in-common-lisp https://github.com/marcoheisig/Petalisp#why-is-petalisp-writ... and https://github.com/numcl/specialized-function https://github.com/numcl/specialized-function can exist.
- pjmlp 3mo ago
  In Lisp (compile ...) produces machine code, with various optimisation flags available, depending on the implementation.
guenthert 3mo ago
Not sure about that. I rather think it's the very dynamic nature of Python (e.g. at run time one can add members to classes) which limits performance gains achievable by compilation. Iirc PyPy got to some five fold performance gain against CPython. Common Lisp, as an example, makes some deliberate flexibility (and elegance) vs. performance trade-offs. I'd think to speed Python up more than PyPy could, one would have to sacrifice some of its features (some of those might not be all that popular or even desirable). Nah, Python has its place. If its performance isn't adequate for the problem at hand, rewrite the bottlenecks in C++ and be done. LLMs should make that easy, no?
- pjmlp 3mo ago
  The dynamic excuse always misses out fully dynamic languages like Smalltalk, Self, among others have achieved, and how it got further exploited into JavaScript engines. However CPython isn't even half way there to either Papyrus or GraalPy. The community attitude to rewrite into C, C++, Rust, Zig, whatever, is exactly why Python will never be like other dynamic languages, and the dynamism excuse will always be used. Thankfully at least GPU vendors see it otherwise, so if nothing else at least it will get JITs that speak GPU machine code.

iLemming 3mo ago

There's nothing "special" about Lisp and Lisp dialects, yes. Similar features can be or already have been implemented in other languages. Yet, after touching, using and experiencing working with a bunch of different stacks, I cannot simply ignore the enormous pragmatic level of Lisps. Working with Clojure is an absolute delight. It strips down all the dogma and let's you deal with the "business logic" as if you're cooking steak using no BS ingredients - meat is meat, herbs are real, stove is hot. Why would I ever choose bash for writing anything slightly more complex than simple redirection, when I can do things in way better fashion with babashka. Why would I wrestle a YAML CI pipeline that only fails on push, when I can drive the whole thing from a babashka task file, run each step locally in the REPL, and actually debug it? Why would I ever deal with Lua, if I can't even format it for "readability" - no matter how I do it, it just looks darn ugly, and luafmt often makes it worse. Why, if I can just slash down dozen lines of Lua boilerplate compressing it into a three-liner Fennel macro? With Fennel, I can interactively poke through elements of my WM through Hammerspoon on Mac, and that's just bananas. Why would I ever deal with JSON, when EDN is almost twice as compact and far more readable - I can align things and treat data as a literal table. Besides, I can group, sort, filter, slice, dice, salt & pepper that data easily, without ever leaving my trusted editor. Why would I choose to build a web-scraper in Python, when I can use nbb driving Playwright and go through selectors interactively, directly from my editor, as if it is a devtools console. And I don't even have to restart anything, deal with state changes, etc. How can I abandon Emacs where I can just open a scratch buffer, type some Elisp and change the behavior of my editor, my WM, my OS and even things on remote computers. No other text editing environment works the way Emacs does - nothing even comes close. It feels like playing a video game, where my controller in my editor. Why would I write Flutter UIs in Dart, fighting the widget-tree ceremony and endless build() boilerplate, when ClojureDart lets me express the same tree as plain data and hot-reload it interactively? The layout is just nested maps and vectors. Why would I reach for C when I need to embed a small, fast scripting layer. Text parsing alone would be a regex nightmare elsewhere. Why would I bolt a templating engine onto HTML strings, when Hiccup makes markup just vectors - so my views compose, filter, and generate like any other data, no special templating DSL to learn And with all sorts of different runtimes and dissimilar Lisp dialects, it still feels as if you're working with the same language. The mental overhead when switching is so negligible. While switching between just JS and TS - which are supposed to be of the "same family" - feels quite annoying. Despite the fact that I've put years into those - far longer than any Lisp I've ever used. Sure, nothing special about Lisp at all. Except that practicing Lisp can actually make you a polyglot. You'd realize that it isn't syntax that makes a programming language, but runtime and semantics do. After years of dealing with different PLs, I lost a preference for one specific language - I'd choose the runtime best suitable for the task, and then see if I can bolt Lisp on top of it. And these days, it feels like there isn't a platform left where you can't meaningfully do things via Lisp.

BoingBoomTschak 3mo ago

You're really misrepresenting Lisp macros here, being able to rewrite syntax is only one (admittedly important) facet of these. The real thing with CL is that the entire language is available during parsing and macro expansion and that users can hook into these steps to influence them. No artificial limitations like you get in C++'s consteval, you can do everything and anything without having to use a crappy DSL to do it. Also, I've never found SBCL slow to compile. Have you?

PaulHoule 3mo ago
(1) SBCL is fine in terms of speed so far as I am concerned. But wherever you have software engineering in the large, especially involving C++, you have 40 minute builds and a split between smart people who think that's a problem and smart people who don't. (2) The trouble with those balls-to-the-walls macros is that systems that use them tend to be "write only", like some guy writes 4000 lines of Lisp that do the work of 80,000 lines of C++ and then he moves on and there is never a version 2.0 but the system gets rewritten in C++, supposedly for performance, but really because they couldn't find anybody who could maintain it. It is really pleasing to see how little code is in Lisp systems from the golden age of AI but... when Yahoo! bought Graham's startup they kept the customers but threw away the code. You are more likely to be sustainable if you get two people to write 12,000 lines in something like typescript or ML that uses Dragon Book technology to the same end.

patterns 3mo ago

While I (think) understand the sentiment (ergonomics and practicality), it is still worth studying Lisp for a number of reasons. In particular, I would recommend to everyone to study Lisp 1.5 [1] to appreciate how an entire universe of programming languages can be bootstrapped with just a few primitives and equations. It is true that, today, we have a vast array of impressive tools at our disposal, from parser combinators and generators, code generators to entire language workbenches with projectional editing capabilities. However, if one were to design a language for any reason, having a deep understanding of the expressiveness of computational models such as the lambda calculus would certainly be an "advantage" (especially for those who end up having to use the language): A Lisp/Scheme is as close to interactive lambda calculus as it gets. From there on, one can learn about implementing different evaluation strategies, scoping rules and continuations (William Byrd's "The Most Beautiful Program Ever Written" [2] also to mind). Now, I am not sure whether homoiconicity tends to lead people astray. It is true that is it not strictly necessary to make a language extensible (e.g., Smalltalk has no macros and is very extensible due to its powerful meta-object protocol and an elegant syntax for closures), but it's still worth studying the concept. For example, writing a metaintpreter in Prolog [3] is surprisingly easy because of its homoiconicity. [1] https://softwarepreservation.computerhistory.org/LISP/book/LISP%201.5%20Programmers%20Manual.pdf https://softwarepreservation.computerhistory.org/LISP/book/L... [2] https://www.youtube.com/watch?v=OyfBQmvr2Hc https://www.youtube.com/watch?v=OyfBQmvr2Hc [3] https://www.metalevel.at/acomip/ https://www.metalevel.at/acomip/

loose-cannon 3mo ago

I thought Scheme was just an educational tool - a minimal language that allows the student to explore language construction & design relatively quickly. I'm not sure I understand your comment about homoiconicity leading people astray. Isn't that the characteristic feature of lisp? Doesn't that dramatically simplify generating code (in principle)?

Jtsummers 3mo ago
> I thought Scheme was just an educational tool That's one use. It's also used in commercial and open source systems. Guile, for instance, has been used as an extension language (like lua and others) in many projects.
jrapdx3 3mo ago
I find it interesting that a number of PLs were initially developed as educational or research tools but later expanded for general use. Some examples are Pascal, Scheme, Prolog. AFAIK many development languages are used in education so the distinction is rather fuzzy. Several Scheme implementations, e.g., Chicken, Chez, Gambit, are capable of producing almost any type of program. I'm most familiar with Chicken Scheme. Comes with a large "standard" library plus a broad range of available extensions. Its easy to use FFI adds to productivity. My experience anyway. Of course, mileage varies widely among users of any language.
- deleted 3mo ago
  [deleted]
- loose-cannon 3mo ago
  Yea sorry. I didn't mean to criticize or diminish its value by saying it's just an educational tool. It's a powerful tool that lets you do some advanced things quickly. That's totally valuable outside of an educational setting. And I didn't mean to imply that there's any limitation to its use. I just don't understand the criticism in the post I was replying to.

kazinator 3mo ago

> the techniques in the Dragon Book for writing compilers are the real magic. Can confirm; got a few hints from the 1988 edition when working on the TXR Lisp compiler. Oh, and also when implementing regexes, not to forget! However, the Lisp compiler has its own magics that are not exactly covered in the Dragon Book. Some really nice way of doing things.

hajile 3mo ago

I might buy this if those languages were actually faster. Instead, implementations of scheme with no commercial backers run circles around scripting languages with huge budgets. Common Lisp can match and exceed languages like Java without the backing of a megacorp. Carp Lisp is basically identical to C in performance and old PS games were shipped in a proprietary dialect (GOAL) and had very good performance. Lisp has proven itself capable great performance at every abstraction level.