5 ms·

Does the Common Lisp standard need to be revamped? I remember seeing stuff about CL21 a while back... I keep my eyes peeled for anything Scheme on GitHub, I lo

Does the Common Lisp standard need to be revamped? I remember seeing stuff about CL21 a while back... I keep my eyes peeled for anything Scheme on GitHub, I love reading Scheme code, and seeing R7RS march eternally forward has brought me a lot of joy over the past few years. But try as I might, and I HAVE tried, I just don't see Common Lisp as dealing with modern computing that well. It looks crufty and tired where Scheme continues to refine and sharpen. And then there's Clojure, which has become what I always kinda thought Common Lisp should've tried to become. This isn't a very good comment, I guess it just always amazes me how much innovation is still happening from simple, unfancy s-expressions. Thank God.

lisper 5y ago
> Does the Common Lisp standard need to be revamped? Yes. Will it happen? Almost certainly not, though I would like nothing better than to be proven wrong about this. > it just always amazes me how much innovation is still happening from simple, unfancy s-expressions Turns out S-expressions are just a Really Good Idea (tm). That's why people keep reinventing them over and over.
- tgbugs 5y ago
  > Yes. Will it happen? Almost certainly not, though I would like nothing better than to be proven wrong about this. I have a sense that any new standards process would have to work hard to defend against a number of distractions, such as forcing character encoding into the standard, the lure of JVM style compile once run anywhere (which has destroyed the ability for Clojure to bootstrap without maven), or attempts to reduce or remove certain warts that some users find annoying. Many who don't know the history don't realize that the standard as it exists encoded the separate practices of many disparate groups into a single standard. This can be confusing for newcomers because they may encounter ways of doing things that are archaic but still in the language. Part of the challenge thus is not getting rid of things, but somehow figuring out how to communicate what have come to be understood as the better ways of doing things --- all without changing those parts of the standard. I'm sure that the old lispers have fairly extensive lists of fundamental issues and improvements. The short list that I have been able to compile from chance encounters with mentions of shortcomings includes implementation details, such as lexical O(1) jump tables, delimited continuations, proper tail call elimination, threads, extending dispatch macro syntax, and default precision for reading floats, and issues that almost certainly need more research and exploration, such as finding a solution to the CLOS action at a distance issue, CLOS dispatch on parametric types, something about static types, ways to alloc without gc, and many more things I am completely unaware of. The Racket ecosystem has done quite a bit of research and implementation in many of these areas, but Racket lies at the far end of the lisp family from common lisp. Finding a way to take the more static and text based ideas from the ML/Racket end of the spectrum and make them accessible at the the dynamic image based Smalltalk/Interlisp end seems like something that could happen within the common lisp space more readily than elsewhere. There seem to be some fundamental questions about how late binding interacts with many other language features that do not have satisfactory answers. All of this being said, I don't see the motivation for anyone in the space to spend the time on another standards process while so many ideas haven't been explored.
  - sedachv 5y ago
    > I'm sure that the old lispers have fairly extensive lists of fundamental issues and improvements. There are two public lists where specific proposals are collected: https://cliki.net/Proposed%20ANSI%20Changes https://cliki.net/Proposed%20ANSI%20Changes https://cliki.net/Proposed%20Extensions%20to%20ANSI https://cliki.net/Proposed%20Extensions%20to%20ANSI And a page for more speculative proposals: https://cliki.net/Lisp%20-%20Next%20Generation https://cliki.net/Lisp%20-%20Next%20Generation Please feel free to add your ideas there. > ways to alloc without gc Do you mean something between DYNAMIC-EXTENT and https://gitlab.common-lisp.net/vsedach/Thinlisp-1.1 https://gitlab.common-lisp.net/vsedach/Thinlisp-1.1 ?
- jhoechtl 5y ago
  > Turns out S-expressions are just a Really Good Idea (tm). That's why people keep reinventing them over and over. In the age of multicore, the perception that everything is a list which lends to sequential processing it makes lists actually less appealing.
  - sedachv 5y ago
    http://dspace.mit.edu/handle/1721.1/14719 http://dspace.mit.edu/handle/1721.1/14719
  - pjmlp 5y ago
    Except most supercomputers have had Lisp and ML dialects experiments, because the hardware abstractions allows for easier scalabilitiy across cluster nodes, either on the same hardware or across the network.
  - reikonomusha 5y ago
    Lisp represents source code as lists. It’s a huge myth that modern Lisp just uses lists as the only data structure.
    - protomyth 5y ago
      Even older lisps had interesting data structures. Star Lisp (*Lisp) on the Connection Machine had vectors. https://en.m.wikipedia.org/wiki/*Lisp https://en.m.wikipedia.org/wiki/*Lisp
      - lispm 5y ago
        Every Common Lisp already has vectors. STARLISP is an extension of Common Lisp. Pvars in STARLISP are basically similar to vectors with elements spread over different processors.
- sedachv 5y ago
  > Will it happen? Almost certainly not, though I would like nothing better than to be proven wrong about this. At this point it is about consensus among implementations. phoe got package local nicknames into all implementations, and that is a very major revision to the standard. So in practice it is happening.
  - jackdaniel 5y ago
    Ecl and abcl (and of course sbcl) adopted pln before that, so this statement is not technically correct.
  - jorams 5y ago
    > phoe got package local nicknames into all implementations Unfortunately it's not yet in Clisp. I submitted a merge request[1] a year ago, but it's been silent since then. [1]: https://gitlab.com/gnu-clisp/clisp/-/merge_requests/3 https://gitlab.com/gnu-clisp/clisp/-/merge_requests/3
- Blikkentrekker 5y ago
  > Yes. Will it happen? Almost certainly not, though I would like nothing better than to be proven wrong about this. Of course it did, was Clojure not released? — “And then there's Clojure, which has become what I always kinda thought Common Lisp should've tried to become.”. I can't help but think that this is purely a game of names, not of existence and features.
tmm84 5y ago
I too love Scheme. I prefer Racket Scheme but to each his/her own. I think it may be whole running VM kind of model that Common Lisp requires the programmer to keep in mind. What is defined/expanded at run-time VS the compiled nature of Scheme/Clojure. I may be wrong about my wording here but there have been times when playing around with real Common Lisp that I just couldn't understand why something wouldn't work when it did in Scheme (I don't remember the specifics).
Siira 5y ago
What’s a Scheme that 1) has fast startup binaries (so no Racket) 2) has a good emacs experience ala SLIME/SLY, CIDER?
- rgrmrts 5y ago
  Chicken Scheme compiles to C, and you can build binaries as you would with any C compiler. Geiser is the Scheme equivalent in emacs, but I don’t know how well it competes with SLIME/SLY/CIDER.
- lispm 5y ago
  I would guess that the new Racket implementation based on Chez has 'fast startup binaries'. Doesn't it have that?
  - wtetzner 5y ago
    And Chez itself seems like a good candidate.
- adenozine 5y ago
  I've been messing with Cyclone Scheme lately, but I don't use emacs. Might be worth a look? Guile is what I mainly use for hobby projects, it's pretty dang reliable and fast, Andy Wingo is a compiler genius and there's a lot of good internals walkthroughs on his blog. Other than Racket, have you tried a lot of implementations?
lukego 5y ago
I consider Serapeum to be a revamp of the Common Lisp standard: https://github.com/ruricolist/serapeum/blob/master/REFERENCE.md https://github.com/ruricolist/serapeum/blob/master/REFERENCE.... This provides a bunch of new features and idioms including ideas borrowed from newer languages like Clojure. Great example of "growing a language" as a long-term evolutionary process that doesn't require changing earlier specifications in incompatible ways.
rauhl 5y ago
> Does the Common Lisp standard need to be revamped? Probably. It has been nearly twenty years after all! > I remember seeing stuff about CL21 a while back... That was mostly one guy’s thoughts, and didn’t meet with universal approval. Here are this guy’s thoughts: - Retain case sensitivity. Remove case-folding. Use lower-case for all symbols in the common-lisp package. Basically, Allegro’s modern mode is the right mode. - Improve, don’t eliminate, pathnames. Standardise their semantics for Unix and Unix-like systems. Bonus points to integrate URLs and pathnames, although … that way may lie madness. - Expand the standard. In 1994 Common Lisp was derided for being a huge language, but compared to languages like Go or Python the library portion of the standard is pretty small. It is probably time to add some TCP & UDP primitives. It is time to settle on some concurrency primitives. And then there are the higher-level things 21st-century programmers take for granted: HTTP clients and servers, Base64, that sort of thing. Maybe specify them in separate optional packages for smaller environments? - Consider CLOSing all the things. Caution: this could be a terrifically bad idea, but … - Consider breaking backward compatibility and settling on one argument order to rule them all. (nth 1 list) and (elt list 1) is a wart. Switching to lower-case raises the possibility of retaining the entire COMMON-LISP package for backwards compatibility here …
eadmund 5y ago
> I just don't see Common Lisp as dealing with modern computing that well. It looks crufty and tired where Scheme continues to refine and sharpen. I don't think that they are aimed at the same thing. Lisp is stable, reliable and suitable for large, durable systems; Scheme is conceptually pure, but its refinement and sharpness lead to fragility. You applaud the success of R7RS, but it is a retreat from specifying a general-purpose programming language (something R6Rs tried to do and was castigated for), choosing simply to provide a toolkit for building a general-purpose programming language. That's not bad, but it means Scheme is not suited for the things Lisp is suited for. I would hate for the RTOS monitoring my car to be written in Scheme, but I think it could in principle be written in Lisp. Racket, arguably the most successful and influential Scheme out there, isn't even a Scheme anymore! There's nothing wrong with that, of course, but it says something when the best Scheme had to evolve until it was no longer Scheme. Meanwhile, systems written in Common Lisp decades ago keep running, and new systems keep being written. It has some warts, some big ones, but it has some real virtues, too.
- adenozine 5y ago
  You've made some very good points, all of which I agree with. I suppose I somewhat despise the Common Lisp "way" of things so much so that it distorts my perception of it. A lot of old JPL stuff was written in Lisp, and what you said about resilience strikes a chord with me. I think the things that really stimulate me the most are things that a small, refined, core language can provide. I've always considered it a benefit that R7RS has become smaller and also that history will not look favorably to R6RS, I thought it was the wrong move. The projects that matter the most to me and to my sensibilities for scheme are abstract and academic. Check out namin's staged-minikanren repo on GitHub. (On mobile, else I'd link you) That is one of the most fascinating pieces of technology ever produced by humankind, imo, and it's written in a simple, common scheme. I think it actually evaluates on Racket, but you can see most of the code tastes like regular Scheme. If I said that Common Lisp wasn't good, I surely regret it and can admit my wrong. I wouldn't call Racket the best scheme, but certainly the most well known, as you say. It evolved in parallel to it's vision of meta-languages, and that's something I would argue entirely catalyzed by having a small, simple, functional core of Scheme. Racket could just as well have been Lisp, and it'd look and feel different, but the same magic would be lurking beneath the surface. It'd probably execute a lot faster these days too! Anyhow, thanks a lot for your insights.
pjmlp 5y ago
It doesn´t help that most people that try Common Lisp via the FOSS offerings, are completly unaware of what LispWorks and Allegro Common Lisp are capable of.
- vindarel 5y ago
  Agreed. I just wrote this LispWorks IDE review and will add it to the CL Cookbook (when slightly enhanced): https://lisp-journey.gitlab.io/blog/discovering-the-lispworks-ide/ https://lisp-journey.gitlab.io/blog/discovering-the-lispwork... It should shed some light on their offerings and help newcomers get a bigger idea of what CL is.
  - pjmlp 5y ago
    Uau, very nicely written.

lisper 5y ago

> Does the Common Lisp standard need to be revamped? Yes. Will it happen? Almost certainly not, though I would like nothing better than to be proven wrong about this. > it just always amazes me how much innovation is still happening from simple, unfancy s-expressions Turns out S-expressions are just a Really Good Idea (tm). That's why people keep reinventing them over and over.

tgbugs 5y ago
> Yes. Will it happen? Almost certainly not, though I would like nothing better than to be proven wrong about this. I have a sense that any new standards process would have to work hard to defend against a number of distractions, such as forcing character encoding into the standard, the lure of JVM style compile once run anywhere (which has destroyed the ability for Clojure to bootstrap without maven), or attempts to reduce or remove certain warts that some users find annoying. Many who don't know the history don't realize that the standard as it exists encoded the separate practices of many disparate groups into a single standard. This can be confusing for newcomers because they may encounter ways of doing things that are archaic but still in the language. Part of the challenge thus is not getting rid of things, but somehow figuring out how to communicate what have come to be understood as the better ways of doing things --- all without changing those parts of the standard. I'm sure that the old lispers have fairly extensive lists of fundamental issues and improvements. The short list that I have been able to compile from chance encounters with mentions of shortcomings includes implementation details, such as lexical O(1) jump tables, delimited continuations, proper tail call elimination, threads, extending dispatch macro syntax, and default precision for reading floats, and issues that almost certainly need more research and exploration, such as finding a solution to the CLOS action at a distance issue, CLOS dispatch on parametric types, something about static types, ways to alloc without gc, and many more things I am completely unaware of. The Racket ecosystem has done quite a bit of research and implementation in many of these areas, but Racket lies at the far end of the lisp family from common lisp. Finding a way to take the more static and text based ideas from the ML/Racket end of the spectrum and make them accessible at the the dynamic image based Smalltalk/Interlisp end seems like something that could happen within the common lisp space more readily than elsewhere. There seem to be some fundamental questions about how late binding interacts with many other language features that do not have satisfactory answers. All of this being said, I don't see the motivation for anyone in the space to spend the time on another standards process while so many ideas haven't been explored.
- sedachv 5y ago
  > I'm sure that the old lispers have fairly extensive lists of fundamental issues and improvements. There are two public lists where specific proposals are collected: https://cliki.net/Proposed%20ANSI%20Changes https://cliki.net/Proposed%20ANSI%20Changes https://cliki.net/Proposed%20Extensions%20to%20ANSI https://cliki.net/Proposed%20Extensions%20to%20ANSI And a page for more speculative proposals: https://cliki.net/Lisp%20-%20Next%20Generation https://cliki.net/Lisp%20-%20Next%20Generation Please feel free to add your ideas there. > ways to alloc without gc Do you mean something between DYNAMIC-EXTENT and https://gitlab.common-lisp.net/vsedach/Thinlisp-1.1 https://gitlab.common-lisp.net/vsedach/Thinlisp-1.1 ?
jhoechtl 5y ago
> Turns out S-expressions are just a Really Good Idea (tm). That's why people keep reinventing them over and over. In the age of multicore, the perception that everything is a list which lends to sequential processing it makes lists actually less appealing.
- sedachv 5y ago
  http://dspace.mit.edu/handle/1721.1/14719 http://dspace.mit.edu/handle/1721.1/14719
- pjmlp 5y ago
  Except most supercomputers have had Lisp and ML dialects experiments, because the hardware abstractions allows for easier scalabilitiy across cluster nodes, either on the same hardware or across the network.
- reikonomusha 5y ago
  Lisp represents source code as lists. It’s a huge myth that modern Lisp just uses lists as the only data structure.
  - protomyth 5y ago
    Even older lisps had interesting data structures. Star Lisp (*Lisp) on the Connection Machine had vectors. https://en.m.wikipedia.org/wiki/*Lisp https://en.m.wikipedia.org/wiki/*Lisp
    - lispm 5y ago
      Every Common Lisp already has vectors. STARLISP is an extension of Common Lisp. Pvars in STARLISP are basically similar to vectors with elements spread over different processors.
sedachv 5y ago
> Will it happen? Almost certainly not, though I would like nothing better than to be proven wrong about this. At this point it is about consensus among implementations. phoe got package local nicknames into all implementations, and that is a very major revision to the standard. So in practice it is happening.
- jackdaniel 5y ago
  Ecl and abcl (and of course sbcl) adopted pln before that, so this statement is not technically correct.
- jorams 5y ago
  > phoe got package local nicknames into all implementations Unfortunately it's not yet in Clisp. I submitted a merge request[1] a year ago, but it's been silent since then. [1]: https://gitlab.com/gnu-clisp/clisp/-/merge_requests/3 https://gitlab.com/gnu-clisp/clisp/-/merge_requests/3
Blikkentrekker 5y ago
> Yes. Will it happen? Almost certainly not, though I would like nothing better than to be proven wrong about this. Of course it did, was Clojure not released? — “And then there's Clojure, which has become what I always kinda thought Common Lisp should've tried to become.”. I can't help but think that this is purely a game of names, not of existence and features.

tmm84 5y ago

I too love Scheme. I prefer Racket Scheme but to each his/her own. I think it may be whole running VM kind of model that Common Lisp requires the programmer to keep in mind. What is defined/expanded at run-time VS the compiled nature of Scheme/Clojure. I may be wrong about my wording here but there have been times when playing around with real Common Lisp that I just couldn't understand why something wouldn't work when it did in Scheme (I don't remember the specifics).

Siira 5y ago

What’s a Scheme that 1) has fast startup binaries (so no Racket) 2) has a good emacs experience ala SLIME/SLY, CIDER?

rgrmrts 5y ago
Chicken Scheme compiles to C, and you can build binaries as you would with any C compiler. Geiser is the Scheme equivalent in emacs, but I don’t know how well it competes with SLIME/SLY/CIDER.
lispm 5y ago
I would guess that the new Racket implementation based on Chez has 'fast startup binaries'. Doesn't it have that?
- wtetzner 5y ago
  And Chez itself seems like a good candidate.
adenozine 5y ago
I've been messing with Cyclone Scheme lately, but I don't use emacs. Might be worth a look? Guile is what I mainly use for hobby projects, it's pretty dang reliable and fast, Andy Wingo is a compiler genius and there's a lot of good internals walkthroughs on his blog. Other than Racket, have you tried a lot of implementations?

lukego 5y ago

I consider Serapeum to be a revamp of the Common Lisp standard: https://github.com/ruricolist/serapeum/blob/master/REFERENCE.md https://github.com/ruricolist/serapeum/blob/master/REFERENCE.... This provides a bunch of new features and idioms including ideas borrowed from newer languages like Clojure. Great example of "growing a language" as a long-term evolutionary process that doesn't require changing earlier specifications in incompatible ways.

rauhl 5y ago

> Does the Common Lisp standard need to be revamped? Probably. It has been nearly twenty years after all! > I remember seeing stuff about CL21 a while back... That was mostly one guy’s thoughts, and didn’t meet with universal approval. Here are this guy’s thoughts: - Retain case sensitivity. Remove case-folding. Use lower-case for all symbols in the common-lisp package. Basically, Allegro’s modern mode is the right mode. - Improve, don’t eliminate, pathnames. Standardise their semantics for Unix and Unix-like systems. Bonus points to integrate URLs and pathnames, although … that way may lie madness. - Expand the standard. In 1994 Common Lisp was derided for being a huge language, but compared to languages like Go or Python the library portion of the standard is pretty small. It is probably time to add some TCP & UDP primitives. It is time to settle on some concurrency primitives. And then there are the higher-level things 21st-century programmers take for granted: HTTP clients and servers, Base64, that sort of thing. Maybe specify them in separate optional packages for smaller environments? - Consider CLOSing all the things. Caution: this could be a terrifically bad idea, but … - Consider breaking backward compatibility and settling on one argument order to rule them all. (nth 1 list) and (elt list 1) is a wart. Switching to lower-case raises the possibility of retaining the entire COMMON-LISP package for backwards compatibility here …

eadmund 5y ago

> I just don't see Common Lisp as dealing with modern computing that well. It looks crufty and tired where Scheme continues to refine and sharpen. I don't think that they are aimed at the same thing. Lisp is stable, reliable and suitable for large, durable systems; Scheme is conceptually pure, but its refinement and sharpness lead to fragility. You applaud the success of R7RS, but it is a retreat from specifying a general-purpose programming language (something R6Rs tried to do and was castigated for), choosing simply to provide a toolkit for building a general-purpose programming language. That's not bad, but it means Scheme is not suited for the things Lisp is suited for. I would hate for the RTOS monitoring my car to be written in Scheme, but I think it could in principle be written in Lisp. Racket, arguably the most successful and influential Scheme out there, isn't even a Scheme anymore! There's nothing wrong with that, of course, but it says something when the best Scheme had to evolve until it was no longer Scheme. Meanwhile, systems written in Common Lisp decades ago keep running, and new systems keep being written. It has some warts, some big ones, but it has some real virtues, too.

adenozine 5y ago
You've made some very good points, all of which I agree with. I suppose I somewhat despise the Common Lisp "way" of things so much so that it distorts my perception of it. A lot of old JPL stuff was written in Lisp, and what you said about resilience strikes a chord with me. I think the things that really stimulate me the most are things that a small, refined, core language can provide. I've always considered it a benefit that R7RS has become smaller and also that history will not look favorably to R6RS, I thought it was the wrong move. The projects that matter the most to me and to my sensibilities for scheme are abstract and academic. Check out namin's staged-minikanren repo on GitHub. (On mobile, else I'd link you) That is one of the most fascinating pieces of technology ever produced by humankind, imo, and it's written in a simple, common scheme. I think it actually evaluates on Racket, but you can see most of the code tastes like regular Scheme. If I said that Common Lisp wasn't good, I surely regret it and can admit my wrong. I wouldn't call Racket the best scheme, but certainly the most well known, as you say. It evolved in parallel to it's vision of meta-languages, and that's something I would argue entirely catalyzed by having a small, simple, functional core of Scheme. Racket could just as well have been Lisp, and it'd look and feel different, but the same magic would be lurking beneath the surface. It'd probably execute a lot faster these days too! Anyhow, thanks a lot for your insights.

pjmlp 5y ago

It doesn´t help that most people that try Common Lisp via the FOSS offerings, are completly unaware of what LispWorks and Allegro Common Lisp are capable of.

vindarel 5y ago
Agreed. I just wrote this LispWorks IDE review and will add it to the CL Cookbook (when slightly enhanced): https://lisp-journey.gitlab.io/blog/discovering-the-lispworks-ide/ https://lisp-journey.gitlab.io/blog/discovering-the-lispwork... It should shed some light on their offerings and help newcomers get a bigger idea of what CL is.
- pjmlp 5y ago
  Uau, very nicely written.