16 ms·

Technical Issues of Separation in Lisp Function Cells and Value Cells (1988)

85 points by adgasf 10y ago

taeric 10y ago
My favorite quote: We feel that the time for such radical changes to Common Lisp passed, and it would be the job of future Lisp designers to take lessons from Common Lisp and Scheme to produce an improved Lisp. I don't know if it is truly indicative of a general outlook, but the implication I read is that future Lisps are expected and encouraged. An open question I have, and if this is covered somewhere already, I'd love to see it. It is interesting to me that while Lisp was looking to condense namespaces dramatically, it seems many other languages went the opposite route. Is that a trend I just don't understand, or is there something inherent in Lisp that favors fewer namespaces?
- hellofunk 10y ago
  Well, many have argued that Clojure is that future lisp. (It's a lisp-1, incidentally)
  - taeric 10y ago
    Yeah, Clojure was what I would expect to be the popular frontrunner here. Evidently, a few years ago arc would have been considered. My point was more that it intrigued me that the view given was not that one or the other was correct and everyone should move to it. More that both sides had lessons learned that could be taken to the next iteration. This seems different from most other language discussions which are about building on more and more to the given language. Maybe I'm just reading the tea leaves, as it were, incorrectly.
    - DonaldFisk 10y ago
      Arc is also a Lisp 1. There's also Mark Tarver's Shen (http://www.shenlanguage.org/ http://www.shenlanguage.org/). I don't know whether it's a Lisp 1 or 2, but I wouldn't be surprised if it's also a Lisp 1. I have read about Shen but not programmed in it, but I see it's added quite a few features absent from Common Lisp and the newer Lisps, which I think are worth having.
      - hellofunk 10y ago
        I decided to read up on Shen, and followed some reddit threads about it, and the consensus seems to be that it is a very inefficient language and many of its abstractions are not actually portable between platforms, i.e. the language changes a bit depending on where you use it.
  - deleted 10y ago
    [deleted]
- netsettler 10y ago
  I dunno about expected and encouraged. Neither really, nor discouraged though. Really the point was that this was written in the context of a particular standardization effort and was to say "this is out of scope". As it happens, RPG and I disagreed completely on whether Lisp1 or Lisp2 was a good idea. Most readers read this like a Rorschach, assuming that the article confirms their own beliefs, not stopping to think that the article just presents two sides of an issue. In the context of the design process, Lisp2 won out (and I happen to be happy about that). But it is reasonable and appropriate for both to thrive if there are users who like those sorts of things. I created the Lisp1/Lisp2 terminology as a dodge because we started out writing this paper using terms something akin to "Scheme-like" and "CL-like", and Scheme was winning for reasons unrelated to the issue at hand. I wanted people to separate their warm fuzzies for Scheme from the particular design choice, as I really think there are very strong and reasonable reasons to have a Lisp2. The net result was that in the context of CL, those of us liking Lisp2 successfully argued that it was an unnecessary change to the language from a stability standpoint. This paper is called "Technical Issues..." because the committee document (titled "Issues...") was longer and went into other issues that RPG didn't think were pertinent to formal publication. I frankly don't understand (or perhaps don't agree) with any claim that Lisp was looking to condense namespaces. By convention and John McCarthy's official request, Lisp is the name of a family of languages, not of a particular language. Lisp has no preference. It spans languages with broadly varying points of view. Ironically, CL is really at least a Lisp4, by the way. block tags and go tags are legitimately different namespaces. They just messed up the discussion so we left those out. Whether you also count the type/class homes as a namespace is something that doesn't fit neatly into the terminology so is the reason I say "at least a Lisp4". Left to personal subjectivity. -kmp
  - taeric 10y ago
    My claim for the Lisp community wanting to condense is strictly colored by my (possibly incorrect) view that every Lisp since has been a Lisp1. I have not personally taken too deep of a dive on Lisp1/Lisp2. Reading your response, I get the impression that this was actually a somewhat contested choice. I have to admit, I'm not clear on why I would personally care. Are there hard reasons to prefer one over the other? (In particular, I have not seen any advantages for either way explored from a user perspective.) I'll have to read this particular paper in more depth, so apologies if the answers I'm asking for are in this article.
- lisper 10y ago
  > future Lisps are expected and encouraged That's right. There are two reasons that Common Lisp never got revised: 1. AI winter hit and the money dried up 2. Common Lisp as it stood when it was first released turned to be a very (very!) good design, in particular because it's almost infinitely flexible. You can implement almost anything within CL because you have control over every part of the code-processing process. You can modify the lexer by making new readtables, and you can change the way the AST is processed by writing macros. There is almost nothing you can't do in CL. (There are a very few things you really can't do without changing the standard, but even those are generally possible using vendor-specific extensions, or by making small tweaks to an implementation.) FWIW, here's my take on bringing CL into the 21st century: https://github.com/rongarret/ergolib https://github.com/rongarret/ergolib and the beginning of an associated book: https://github.com/rongarret/BWFP https://github.com/rongarret/BWFP > is there something inherent in Lisp that favors fewer namespaces Not really. It's pretty easy to make a Lisp2 act like a Lisp1 when you want it to (https://news.ycombinator.com/item?id=12801372 https://news.ycombinator.com/item?id=12801372). Much harder to go the other way.
  - nabla9 10y ago
    >You can implement almost anything within CL because you have control over every part of the code-processing process. Almost. There is one big missing piece. Standard code walker. If CL had standard code walker that could correctly walk trough any piece of standard CL code, it would implement fully the promise of Lisp and you could build almost anything on top of it with ease.
    - lisper 10y ago
      True, but 1) there are open-source code-walkers that do a pretty good job, 2) many implementations expose their internal code walker in one way or another and 3) you can do somewhere between 90 and 99% of what you'd want to do with a code walker using MACROLET. My pet peeve is that ((...) ...) is a syntax error, so you can't implement Scheme in Common Lisp without violating the standard. But see http://www.flownet.com/ron/lisp/combination-hook.lisp http://www.flownet.com/ron/lisp/combination-hook.lisp for an example of how easy it is to hack at least one implementation to support this.
chriswarbo 10y ago
For those who are more used to non-lisp scripting languages, I've found the awkwardness of functions in a lisp-2 (Emacs Lisp) compared to a lisp-1 (Scheme) to be similar to the awkwardness of functions in PHP compared to e.g. Javascript or Python.
- aerique 10y ago
  Why would functions in a lisp-2 be more awkward than in a lisp-1? PHP has been too long ago for me to understand your comparison. Would you please explain it differently?
  - Jtsummers 10y ago
    In a lisp-1: (define inc (lambda (n) (+ n 1))) When referring to `inc` (assuming no shadowing in a different scope), `inc` refers to a function object, we can do: (define g (lambda (f) (f 3)) (g inc) ;; yields 4 In a lisp2, it won't work the same way: * (defun inc (n) (+ 1 n)) * (defun g (f) (f 3)) ; in: DEFUN G ; (SB-INT:NAMED-LAMBDA G ; (F) ; (BLOCK G (F 3))) ; ; caught STYLE-WARNING: ; The variable F is defined but never used. ; in: DEFUN G ; (F 3) ; ; caught STYLE-WARNING: ; undefined function: F ; ; compilation unit finished ; Undefined function: ; F ; caught 2 STYLE-WARNING conditions Instead we can define it like: * (defun g (f) (apply f '(3))) ;; specifying the parameters as a list * (g 'inc) ;; get the symbol for inc, as a variable it has no binding 4 * (g inc) debugger invoked on a UNBOUND-VARIABLE in thread #<THREAD "main thread" RUNNING {10028F6AF3}>: The variable INC is unbound. There are some benefits, though it's been so long since I've used either Scheme or Common Lisp that I can't clearly articulate why, but I recall at the time preferring lisp2 (or lispN, really).
    - zeveb 10y ago
      I'm pretty sure that the lispy way to write the example would be: (defun g (f) (funcall f 3)) (g #'inc) Which I think is actually prettier than the Scheme: (define g (lambda (f) (f 3)) (g inc)
DonaldFisk 10y ago
I read the article some time ago. I have my own Lisp, Emblem, which I released well over ten years ago. (There were few takers. I have improved it considerably since then, but not made a re-release.) I decided that it shouldn't be gratuitously different from Common Lisp. However, it is a Lisp 1. The reasons are: (1) my programming style was never inconvenienced by the lack of separate function and value cells when programming in Scheme instead of Common Lisp; and (2) Emblem uses a world (i.e. an image) to store its compiled code in, and an extra, rarely used, cell on every symbol increases the image size.
kazinator 10y ago
In TXR Lisp, I implemented a design which achieves a harmony between the separation of function/value bindings, and their union. You can program in both styles in the same scope. http://www.nongnu.org/txr/txr-manpage.html#N-02DC9E04 http://www.nongnu.org/txr/txr-manpage.html#N-02DC9E04 TL;DR: The underlying Lisp dialect is a Lisp-2. However, when a form is denoted by square brackets, Lisp-1-style evaluation applies to each of its positions: every position is evaluated as a form, and any form which is a symbol is evaluated in a conflated single namespace. This rule does not recurse into the forms: any nested forms also have to use square brackets to work the same way. Thus we can do: [mapcar cons '(1 2 3) '(a b c)] ;; no (fun cons) [f arg] ;; f is a variable And so on. This works by making [...] a syntactic sugar for (dwim ...) where dwim is a special operator. This operator is deeply integrated into the language (both macro-expansion semantics and evaluation), which is why it is able to change the semantics of name lookup. A rule I imposed is that macros are not allowed in this notation: [let ((a 3)) ...] ;; nonsense: let not recognized as operator here so it is actually implements the Lips-1 mantra literally: "evaluate all positions of the form equally". There is no exception for recognizing a function-like macro in the leftmost position of a form, but not doing so in other positions.
- kazinator 10y ago
  It gets tricky. Bracket/dwim forms can designate access to objects, because sequences are funcallable. So we can do: (set [a b] c) What does that look like? 1> (sys:expand '(set [a b] c)) (let ((#:g0138 (sys:lisp1-value b)) (#:g0137 c)) (sys:lisp1-setq a (sys:dwim-set (sys:lisp1-value a) #:g0138 #:g0137)) #:g0137) The a and b forms turned into (sys:lisp1-value a) and (sys:lisp1-value b). These secret operators bring in the name lookup semantics anywhere it is required, disembodied from the dwim operator. Also sys:lisp1-setq is used to store the updated sequence back into a. sys:setq can't be used because a could be a function binding in the given scope. There are hacks you can do in Common Lisp to achieve a bit of a Lisp-1 style, but not to such detail. See this piece of documentation under labels and flet: http://www.nongnu.org/txr/txr-manpage.html#N-0209307D http://www.nongnu.org/txr/txr-manpage.html#N-0209307D "Furthermore, function bindings introduced by labels and flet also shadow symbol macros defined by symacrolet, when those symbol macros occur as arguments of a dwim form." A test case for this is the following, which must produce (1 1 1): (let ((g (lambda (x) 0))) (symacrolet ((f g)) (flet ((f (x) 1)) [mapcar f '(a a a)]))) ;; Lisp-1 f: flet shadows symbol macro However, this must produce (0 0 0): (let ((g (lambda (x) 0))) (symacrolet ((f g)) (flet ((f (x) 1)) (mapcar f '(a a a))))) ;; now Lisp-2 var reference
deleted 10y ago
[deleted]

85 points by adgasf 10y ago

My favorite quote: We feel that the time for such radical changes to Common Lisp passed, and it would be the job of future Lisp designers to take lessons from Common Lisp and Scheme to produce an improved Lisp. I don't know if it is truly indicative of a general outlook, but the implication I read is that future Lisps are expected and encouraged. An open question I have, and if this is covered somewhere already, I'd love to see it. It is interesting to me that while Lisp was looking to condense namespaces dramatically, it seems many other languages went the opposite route. Is that a trend I just don't understand, or is there something inherent in Lisp that favors fewer namespaces?

hellofunk 10y ago
Well, many have argued that Clojure is that future lisp. (It's a lisp-1, incidentally)
- taeric 10y ago
  Yeah, Clojure was what I would expect to be the popular frontrunner here. Evidently, a few years ago arc would have been considered. My point was more that it intrigued me that the view given was not that one or the other was correct and everyone should move to it. More that both sides had lessons learned that could be taken to the next iteration. This seems different from most other language discussions which are about building on more and more to the given language. Maybe I'm just reading the tea leaves, as it were, incorrectly.
  - DonaldFisk 10y ago
    Arc is also a Lisp 1. There's also Mark Tarver's Shen (http://www.shenlanguage.org/ http://www.shenlanguage.org/). I don't know whether it's a Lisp 1 or 2, but I wouldn't be surprised if it's also a Lisp 1. I have read about Shen but not programmed in it, but I see it's added quite a few features absent from Common Lisp and the newer Lisps, which I think are worth having.
    - hellofunk 10y ago
      I decided to read up on Shen, and followed some reddit threads about it, and the consensus seems to be that it is a very inefficient language and many of its abstractions are not actually portable between platforms, i.e. the language changes a bit depending on where you use it.
- deleted 10y ago
  [deleted]
netsettler 10y ago
I dunno about expected and encouraged. Neither really, nor discouraged though. Really the point was that this was written in the context of a particular standardization effort and was to say "this is out of scope". As it happens, RPG and I disagreed completely on whether Lisp1 or Lisp2 was a good idea. Most readers read this like a Rorschach, assuming that the article confirms their own beliefs, not stopping to think that the article just presents two sides of an issue. In the context of the design process, Lisp2 won out (and I happen to be happy about that). But it is reasonable and appropriate for both to thrive if there are users who like those sorts of things. I created the Lisp1/Lisp2 terminology as a dodge because we started out writing this paper using terms something akin to "Scheme-like" and "CL-like", and Scheme was winning for reasons unrelated to the issue at hand. I wanted people to separate their warm fuzzies for Scheme from the particular design choice, as I really think there are very strong and reasonable reasons to have a Lisp2. The net result was that in the context of CL, those of us liking Lisp2 successfully argued that it was an unnecessary change to the language from a stability standpoint. This paper is called "Technical Issues..." because the committee document (titled "Issues...") was longer and went into other issues that RPG didn't think were pertinent to formal publication. I frankly don't understand (or perhaps don't agree) with any claim that Lisp was looking to condense namespaces. By convention and John McCarthy's official request, Lisp is the name of a family of languages, not of a particular language. Lisp has no preference. It spans languages with broadly varying points of view. Ironically, CL is really at least a Lisp4, by the way. block tags and go tags are legitimately different namespaces. They just messed up the discussion so we left those out. Whether you also count the type/class homes as a namespace is something that doesn't fit neatly into the terminology so is the reason I say "at least a Lisp4". Left to personal subjectivity. -kmp
- taeric 10y ago
  My claim for the Lisp community wanting to condense is strictly colored by my (possibly incorrect) view that every Lisp since has been a Lisp1. I have not personally taken too deep of a dive on Lisp1/Lisp2. Reading your response, I get the impression that this was actually a somewhat contested choice. I have to admit, I'm not clear on why I would personally care. Are there hard reasons to prefer one over the other? (In particular, I have not seen any advantages for either way explored from a user perspective.) I'll have to read this particular paper in more depth, so apologies if the answers I'm asking for are in this article.
lisper 10y ago
> future Lisps are expected and encouraged That's right. There are two reasons that Common Lisp never got revised: 1. AI winter hit and the money dried up 2. Common Lisp as it stood when it was first released turned to be a very (very!) good design, in particular because it's almost infinitely flexible. You can implement almost anything within CL because you have control over every part of the code-processing process. You can modify the lexer by making new readtables, and you can change the way the AST is processed by writing macros. There is almost nothing you can't do in CL. (There are a very few things you really can't do without changing the standard, but even those are generally possible using vendor-specific extensions, or by making small tweaks to an implementation.) FWIW, here's my take on bringing CL into the 21st century: https://github.com/rongarret/ergolib https://github.com/rongarret/ergolib and the beginning of an associated book: https://github.com/rongarret/BWFP https://github.com/rongarret/BWFP > is there something inherent in Lisp that favors fewer namespaces Not really. It's pretty easy to make a Lisp2 act like a Lisp1 when you want it to (https://news.ycombinator.com/item?id=12801372 https://news.ycombinator.com/item?id=12801372). Much harder to go the other way.
- nabla9 10y ago
  >You can implement almost anything within CL because you have control over every part of the code-processing process. Almost. There is one big missing piece. Standard code walker. If CL had standard code walker that could correctly walk trough any piece of standard CL code, it would implement fully the promise of Lisp and you could build almost anything on top of it with ease.
  - lisper 10y ago
    True, but 1) there are open-source code-walkers that do a pretty good job, 2) many implementations expose their internal code walker in one way or another and 3) you can do somewhere between 90 and 99% of what you'd want to do with a code walker using MACROLET. My pet peeve is that ((...) ...) is a syntax error, so you can't implement Scheme in Common Lisp without violating the standard. But see http://www.flownet.com/ron/lisp/combination-hook.lisp http://www.flownet.com/ron/lisp/combination-hook.lisp for an example of how easy it is to hack at least one implementation to support this.

chriswarbo 10y ago

For those who are more used to non-lisp scripting languages, I've found the awkwardness of functions in a lisp-2 (Emacs Lisp) compared to a lisp-1 (Scheme) to be similar to the awkwardness of functions in PHP compared to e.g. Javascript or Python.

aerique 10y ago
Why would functions in a lisp-2 be more awkward than in a lisp-1? PHP has been too long ago for me to understand your comparison. Would you please explain it differently?
- Jtsummers 10y ago
  In a lisp-1: (define inc (lambda (n) (+ n 1))) When referring to `inc` (assuming no shadowing in a different scope), `inc` refers to a function object, we can do: (define g (lambda (f) (f 3)) (g inc) ;; yields 4 In a lisp2, it won't work the same way: * (defun inc (n) (+ 1 n)) * (defun g (f) (f 3)) ; in: DEFUN G ; (SB-INT:NAMED-LAMBDA G ; (F) ; (BLOCK G (F 3))) ; ; caught STYLE-WARNING: ; The variable F is defined but never used. ; in: DEFUN G ; (F 3) ; ; caught STYLE-WARNING: ; undefined function: F ; ; compilation unit finished ; Undefined function: ; F ; caught 2 STYLE-WARNING conditions Instead we can define it like: * (defun g (f) (apply f '(3))) ;; specifying the parameters as a list * (g 'inc) ;; get the symbol for inc, as a variable it has no binding 4 * (g inc) debugger invoked on a UNBOUND-VARIABLE in thread #<THREAD "main thread" RUNNING {10028F6AF3}>: The variable INC is unbound. There are some benefits, though it's been so long since I've used either Scheme or Common Lisp that I can't clearly articulate why, but I recall at the time preferring lisp2 (or lispN, really).
  - zeveb 10y ago
    I'm pretty sure that the lispy way to write the example would be: (defun g (f) (funcall f 3)) (g #'inc) Which I think is actually prettier than the Scheme: (define g (lambda (f) (f 3)) (g inc)

DonaldFisk 10y ago

I read the article some time ago. I have my own Lisp, Emblem, which I released well over ten years ago. (There were few takers. I have improved it considerably since then, but not made a re-release.) I decided that it shouldn't be gratuitously different from Common Lisp. However, it is a Lisp 1. The reasons are: (1) my programming style was never inconvenienced by the lack of separate function and value cells when programming in Scheme instead of Common Lisp; and (2) Emblem uses a world (i.e. an image) to store its compiled code in, and an extra, rarely used, cell on every symbol increases the image size.

kazinator 10y ago

In TXR Lisp, I implemented a design which achieves a harmony between the separation of function/value bindings, and their union. You can program in both styles in the same scope. http://www.nongnu.org/txr/txr-manpage.html#N-02DC9E04 http://www.nongnu.org/txr/txr-manpage.html#N-02DC9E04 TL;DR: The underlying Lisp dialect is a Lisp-2. However, when a form is denoted by square brackets, Lisp-1-style evaluation applies to each of its positions: every position is evaluated as a form, and any form which is a symbol is evaluated in a conflated single namespace. This rule does not recurse into the forms: any nested forms also have to use square brackets to work the same way. Thus we can do: [mapcar cons '(1 2 3) '(a b c)] ;; no (fun cons) [f arg] ;; f is a variable And so on. This works by making [...] a syntactic sugar for (dwim ...) where dwim is a special operator. This operator is deeply integrated into the language (both macro-expansion semantics and evaluation), which is why it is able to change the semantics of name lookup. A rule I imposed is that macros are not allowed in this notation: [let ((a 3)) ...] ;; nonsense: let not recognized as operator here so it is actually implements the Lips-1 mantra literally: "evaluate all positions of the form equally". There is no exception for recognizing a function-like macro in the leftmost position of a form, but not doing so in other positions.

kazinator 10y ago
It gets tricky. Bracket/dwim forms can designate access to objects, because sequences are funcallable. So we can do: (set [a b] c) What does that look like? 1> (sys:expand '(set [a b] c)) (let ((#:g0138 (sys:lisp1-value b)) (#:g0137 c)) (sys:lisp1-setq a (sys:dwim-set (sys:lisp1-value a) #:g0138 #:g0137)) #:g0137) The a and b forms turned into (sys:lisp1-value a) and (sys:lisp1-value b). These secret operators bring in the name lookup semantics anywhere it is required, disembodied from the dwim operator. Also sys:lisp1-setq is used to store the updated sequence back into a. sys:setq can't be used because a could be a function binding in the given scope. There are hacks you can do in Common Lisp to achieve a bit of a Lisp-1 style, but not to such detail. See this piece of documentation under labels and flet: http://www.nongnu.org/txr/txr-manpage.html#N-0209307D http://www.nongnu.org/txr/txr-manpage.html#N-0209307D "Furthermore, function bindings introduced by labels and flet also shadow symbol macros defined by symacrolet, when those symbol macros occur as arguments of a dwim form." A test case for this is the following, which must produce (1 1 1): (let ((g (lambda (x) 0))) (symacrolet ((f g)) (flet ((f (x) 1)) [mapcar f '(a a a)]))) ;; Lisp-1 f: flet shadows symbol macro However, this must produce (0 0 0): (let ((g (lambda (x) 0))) (symacrolet ((f g)) (flet ((f (x) 1)) (mapcar f '(a a a))))) ;; now Lisp-2 var reference

deleted 10y ago

[deleted]