3 ms·

As a Common Lisp developer, the intro about Lisp strikes me as sort of mostly(?) true. I would say in my experience we do actually "compile and execute" code,

by dieggsy 1mo ago

As a Common Lisp developer, the intro about Lisp strikes me as sort of mostly(?) true. I would say in my experience we do actually "compile and execute" code, we can just do this incrementally and with less context switching because of the described workflow. But quite often there are in fact separate compile and run steps, it's just not at the whole program level (say, compile a function and run it or a calling function). And despite how often it's shown off as a strength, redefining code during actual execution isn't quite that common, I think. Yes, it is done sometimes, but lots of programs don't really fit that paradigm in the first place. It can work well for e.g. some games or long running servers. As for working in an image without source code: this seems like a terrible idea. You might do this for trivial, one off experiments, but in general you benefit from writing out and organizing your code early. I'm not even aware of a particularly easy or clean way to write out source code from an image as described. It could be done, but all the ways I can think of sound like more of a pain or mess than anything. Could just be my experience or I'm missing some neat feature of a commercial Lisp or something.

jerf 1mo ago
I haven't used Lisp but I've used another environment that was heavily image based (Frontier). The runtime ability to modify things sounds super neat but in my opinion it leads to disaster. It is not a weakness that we have a lot of tech built on a concrete specification of the initial state of the system and very careful control over where data and the modified versions of that data live. It is hard-won experience that it is a good way to build robust systems. "Reboot & pray" isn't just an accident, it's a legitimately good way to set up a system. Some of you may object to the idea that we are super careful with data and modifications to it. I know where you are coming from... however, compared to an image-based system, the modern world is in fact super careful! To the extent that you think we are still not careful enough, that leads you even farther from wanting an image-based system. If you conceive of a system as the full state space of everywhere it can go, the vast, vast, vast majority of systems have pathological places in it. It is very hard to get them all out. It is advantageous to have a button that says "restart this system from the initial state", and to push it often to make sure it continues to work, even if you're not thinking of it this way. By contrast, image-based systems have the tendency to either 1. get into a bad place in the state space and then the user has no way back out or 2. accidentally create a scenario in which there is no way to bring the system up from scratch anymore, which includes things like "giving the system to someone else to start their own work up". Sure, it's a good idea to minimize the need to hit the reset button. But you don't really want to give it up entirely. There are a variety of ways of improving the image-based system. Most, perhaps all of the practical ones, involve basically removing the image-based nature of it and moving closer to the systems we all work with today.
- dieggsy 1mo ago
  I don't object to any of this. "Restart this system from initial state" is quite a bit more common than people imagine in CL. I was hinting at it perhaps being even more common than image-based development, though don't quote me on that. I personally don't even really create Lisp images in dev (save for creating executables that I don't then further modify, which are technically images in SBCL). Incremental runtime modification is great for development, but I very frequently restart (from sources) every so often, and certainly at least before deployment to verify everything is working as expected. Sometimes because it's completely necessary, sometimes for my own sanity. For production, I think it's a very "neat" thing to be able to do and I've heard some great stories (no personal experience), but it's definitely a foot gun if not used very carefully.
  - sroerick 1mo ago
    Can you explain how you sync back to code, in practice? In other words - if you modify the run time, are you then able to mirror those changes in your codebase and then reboot?
    - dieggsy 1mo ago
      tl;dr you just don't really do that. 99% of the time, you simply write the code or change in the sources first, then compile/run it in the runtime. If you're modifying the runtime directly at the REPL without having sources, good luck? or you don't really care about persisting those changes. CL has pretty good introspection, so you could look at the source expression of a function defined at the REPL for example, if you really wanted to recover something (and happened to put the change in a function first), but this isn't always reliable.
Zak 1mo ago
I also found the description of Lisp development weird, as if it's a second-hand repetition of someone else describing watching a Lisp developer in action. The workflow I always use is to write code in my editor and use a keyboard shortcut to evaluate it. Typing directly into the REPL is for one-off state updates, inspection, experiments, and similar throwaway code.
meindnoch 1mo ago
That intro would make much more sense if you replaced "Lisp" with "Smalltalk".
rjsw 1mo ago
Even the original MIT Lisp Machine was designed around editing source files, there were "compile the current function" keybindings and menu options in the version of Emacs for it. You could also type an individual function into the repl then pretty-print it to a file but that typically expands out any read macros that you may have used.
- sroerick 1mo ago
  Awesome. Got any more info? Were any of the Lisp Machines image based?
  - rjsw 1mo ago
    Interlisp-D was more image based. You could type something in to the repl then add it to a file later. There was a structure editor that worked on the s-expression of a function, including any comments as they were represented as a special form within that structure. (* This is a comment) See section 7 of the introductory guide [1]. [1] https://www.mirrorservice.org/sites/www.bitsavers.org/pdf/xerox/interlisp-d/199202_Medley_2.0/An_Introduction_to_Medley_Release_2.0_Feb92.pdf https://www.mirrorservice.org/sites/www.bitsavers.org/pdf/xe...
sroerick 1mo ago
I am not a Lisp developer by trade, but I accidentally made an interpreted Lisp as a habitat for LLMs. My research has mirrored exactly what you are saying. I think conceptually, Lisp was an interpreted runtime. However, the combination of chasing efficiency with compilation and perhaps "everything is a file" consuming mindshare, interpreted Lisp seems a largely academic pursuit. It's really great for running agents, though.
- Jtsummers 1mo ago
  > I think conceptually, Lisp was an interpreted runtime. This is a strange statement. The manual for Lisp 1.5 describes a compiled language. Lisps have been compiled for over 64 years now using just that version as a baseline. Why do you think Lisp was meant to be interpreted instead?
  - yubblegum 1mo ago
    You are ignoring his "conceptually" bit and that is a reference to R.E.P.Loop which is what interpreters do: they read, they evaluate, they output, and loop.
    - Jtsummers 1mo ago
      Conceptually is doing a lot of heavy lifting there, then. The evaluation step of many (I won't say most) mainstream (not academic, and not toys people create in class) lisps will do a compilation step. It is not being interpreted any more than gcc interprets your C code. Describing it as interpretation is often wrong, and leads people to the extremely common (even here on HN) mistaken belief that Lisps are strictly interpreted and never compiled. No reason to leave a mistake (if it was one) or a confusing statement (if it was meant in your, hah, interpretation) unaddressed.
      - yubblegum 1mo ago
        Disagree. Concepually is lifting according to its weight class here. It is entirely irrelevant whether all or most implementations compile. The germinal fact is that a "naive" interpreter implementation of LISP does not in anyway alter the semantics of the language or the code being executed. > No reason to leave a mistake (if it was one) or a confusing statement (if it was meant in your, hah, interpretation) unaddressed. Look buddy, I'm not here for snarking others. I am here to learn and exchange ideas.

by dieggsy 1mo ago

jerf 1mo ago

I haven't used Lisp but I've used another environment that was heavily image based (Frontier). The runtime ability to modify things sounds super neat but in my opinion it leads to disaster. It is not a weakness that we have a lot of tech built on a concrete specification of the initial state of the system and very careful control over where data and the modified versions of that data live. It is hard-won experience that it is a good way to build robust systems. "Reboot & pray" isn't just an accident, it's a legitimately good way to set up a system. Some of you may object to the idea that we are super careful with data and modifications to it. I know where you are coming from... however, compared to an image-based system, the modern world is in fact super careful! To the extent that you think we are still not careful enough, that leads you even farther from wanting an image-based system. If you conceive of a system as the full state space of everywhere it can go, the vast, vast, vast majority of systems have pathological places in it. It is very hard to get them all out. It is advantageous to have a button that says "restart this system from the initial state", and to push it often to make sure it continues to work, even if you're not thinking of it this way. By contrast, image-based systems have the tendency to either 1. get into a bad place in the state space and then the user has no way back out or 2. accidentally create a scenario in which there is no way to bring the system up from scratch anymore, which includes things like "giving the system to someone else to start their own work up". Sure, it's a good idea to minimize the need to hit the reset button. But you don't really want to give it up entirely. There are a variety of ways of improving the image-based system. Most, perhaps all of the practical ones, involve basically removing the image-based nature of it and moving closer to the systems we all work with today.

dieggsy 1mo ago
I don't object to any of this. "Restart this system from initial state" is quite a bit more common than people imagine in CL. I was hinting at it perhaps being even more common than image-based development, though don't quote me on that. I personally don't even really create Lisp images in dev (save for creating executables that I don't then further modify, which are technically images in SBCL). Incremental runtime modification is great for development, but I very frequently restart (from sources) every so often, and certainly at least before deployment to verify everything is working as expected. Sometimes because it's completely necessary, sometimes for my own sanity. For production, I think it's a very "neat" thing to be able to do and I've heard some great stories (no personal experience), but it's definitely a foot gun if not used very carefully.
- sroerick 1mo ago
  Can you explain how you sync back to code, in practice? In other words - if you modify the run time, are you then able to mirror those changes in your codebase and then reboot?
  - dieggsy 1mo ago
    tl;dr you just don't really do that. 99% of the time, you simply write the code or change in the sources first, then compile/run it in the runtime. If you're modifying the runtime directly at the REPL without having sources, good luck? or you don't really care about persisting those changes. CL has pretty good introspection, so you could look at the source expression of a function defined at the REPL for example, if you really wanted to recover something (and happened to put the change in a function first), but this isn't always reliable.

Zak 1mo ago

I also found the description of Lisp development weird, as if it's a second-hand repetition of someone else describing watching a Lisp developer in action. The workflow I always use is to write code in my editor and use a keyboard shortcut to evaluate it. Typing directly into the REPL is for one-off state updates, inspection, experiments, and similar throwaway code.

meindnoch 1mo ago

That intro would make much more sense if you replaced "Lisp" with "Smalltalk".

rjsw 1mo ago

Even the original MIT Lisp Machine was designed around editing source files, there were "compile the current function" keybindings and menu options in the version of Emacs for it. You could also type an individual function into the repl then pretty-print it to a file but that typically expands out any read macros that you may have used.

sroerick 1mo ago
Awesome. Got any more info? Were any of the Lisp Machines image based?
- rjsw 1mo ago
  Interlisp-D was more image based. You could type something in to the repl then add it to a file later. There was a structure editor that worked on the s-expression of a function, including any comments as they were represented as a special form within that structure. (* This is a comment) See section 7 of the introductory guide [1]. [1] https://www.mirrorservice.org/sites/www.bitsavers.org/pdf/xerox/interlisp-d/199202_Medley_2.0/An_Introduction_to_Medley_Release_2.0_Feb92.pdf https://www.mirrorservice.org/sites/www.bitsavers.org/pdf/xe...

sroerick 1mo ago

I am not a Lisp developer by trade, but I accidentally made an interpreted Lisp as a habitat for LLMs. My research has mirrored exactly what you are saying. I think conceptually, Lisp was an interpreted runtime. However, the combination of chasing efficiency with compilation and perhaps "everything is a file" consuming mindshare, interpreted Lisp seems a largely academic pursuit. It's really great for running agents, though.

Jtsummers 1mo ago
> I think conceptually, Lisp was an interpreted runtime. This is a strange statement. The manual for Lisp 1.5 describes a compiled language. Lisps have been compiled for over 64 years now using just that version as a baseline. Why do you think Lisp was meant to be interpreted instead?
- yubblegum 1mo ago
  You are ignoring his "conceptually" bit and that is a reference to R.E.P.Loop which is what interpreters do: they read, they evaluate, they output, and loop.
  - Jtsummers 1mo ago
    Conceptually is doing a lot of heavy lifting there, then. The evaluation step of many (I won't say most) mainstream (not academic, and not toys people create in class) lisps will do a compilation step. It is not being interpreted any more than gcc interprets your C code. Describing it as interpretation is often wrong, and leads people to the extremely common (even here on HN) mistaken belief that Lisps are strictly interpreted and never compiled. No reason to leave a mistake (if it was one) or a confusing statement (if it was meant in your, hah, interpretation) unaddressed.
    - yubblegum 1mo ago
      Disagree. Concepually is lifting according to its weight class here. It is entirely irrelevant whether all or most implementations compile. The germinal fact is that a "naive" interpreter implementation of LISP does not in anyway alter the semantics of the language or the code being executed. > No reason to leave a mistake (if it was one) or a confusing statement (if it was meant in your, hah, interpretation) unaddressed. Look buddy, I'm not here for snarking others. I am here to learn and exchange ideas.