6 ms·

On Repl-Driven Programming (2020)

100 points by ivanvas 4y ago

Jtsummers 4y ago
Part of why this [EDIT: to clarify - the debug/break loop process paired with the REPL] works in Smalltalk and Lisp is their dynamic nature, and also to the fact that they are systems as much as languages. In principle, Ruby and Python should be able to have as capable dev environments. With more static languages like C, Java, C#, and others you run into the problem of needing more "skeleton". You can't write: void foo() { bar(); } in those languages and expect the compiler to be thrilled about it. But you could get closer. Java runs on the JVM, C# on the CLR, those provide systems on which the languages (or programs written in those languages) are executed and the kind of access needed to approach the debugger-driven development approach. I haven't done this in a long time but with C# you can do: void foo() { throw new NotImplementedException(); } Now when your program execution hits `foo` you can catch it in the debugger (versus just crashing or otherwise terminating, or needing to set a breakpoint). With VS (and probably others, I did this with VS) you can change this code and recompile it and resume from that point. Now you decide to make the call to `bar`, oops, that method doesn't exist yet so you also have to create `bar`, and you can stub it out the exact same way. So it's not as easy and pleasant as in Lisp or Smalltalk where you can forgo the skeletal framework of the program and jump straight into it, building each function, method, class, etc. only as needed (truly JIT development), but you can get close. I believe Java has some similar facilities with some dev environments. There is more work, but it's not totally impossible. All that said, I really enjoy the way you can do this in Lisp (and Smalltalk). I've been debating re-solving the Synacor Challenge [0] and recording how I did it in Lisp the first time. I used the debugger in precisely this way, writing a basic version which could read in the binary blob and did minimal parsing, but then entered an ecase (case expression that errors if there is no matching case). For every instruction that I hadn't implemented (none of them at the start) it brought me to the debugger, I modified the program, resumed it and it continued until the next unimplemented instruction. By the time I'd finished, I had the interpreter for the byte code up and running and was immediately getting to work on the rest of the challenge. This also worked as a nice goal-directed approach, implementing the whole interpreter was a "big" (not really) task, how to break it down and which part to do first? Well, interpreting the binary blob acted as my (very complex) test case and gave me immediate feedback (I also added some basic debugger capabilities to the interpreter) on whether I had done it correctly. No need to make the whole thing before testing it, I got to incrementally test each small change. That approach could also be made to work with C# as I described before, write a basic skeleton and parse the instructions but throw a `NotImplementedException` whenever you get to an instruction you haven't, well, implemented yet. Then using VS change the throw to a method call (or do it locally) that implements the handler for that instruction. Repeat until all of them are filled out. It's not as smooth as the Lisp or Smalltalk cases would be, but it's doable. [0] https://challenge.synacor.com https://challenge.synacor.com - programming challenge by the same guy that does Advent of Code. ---------- Also, this has come up on here a few times in the past since it was written (2020). Two past submissions with comments: https://news.ycombinator.com/item?id=25620256 https://news.ycombinator.com/item?id=25620256 - Jan 3, 2021 (208 comments) https://news.ycombinator.com/item?id=25618457 https://news.ycombinator.com/item?id=25618457 - Jan 3, 2021 (5 comments, 4 + 1 dead, don't know why it's dead)
- tester756 4y ago
  I do often use this approach in C#, so I started calling it debugger driven development Seems like I should try Smalltalk
- waf 4y ago
  For what it's worth, modern C# can evaluate top-level statements with no other ceremony. The following is a valid standalone C# program: Console.WriteLine("Hello"); For REPLs, there are options like (my own) https://github.com/waf/CSharpRepl https://github.com/waf/CSharpRepl which stand on top of the Roslyn compiler infrastructure, which is quite extensive and can easily evaluate standalone functions and statements. It's still nowhere close to the REPLs of lisp and smalltalk, but it's a step in a more flexible direction.
  - Jtsummers 4y ago
    True, I'd forgotten about that. That was in the last major release, right?
dang 4y ago
Related: On Repl-Driven Programming - https://news.ycombinator.com/item?id=25620256 https://news.ycombinator.com/item?id=25620256 - Jan 2021 (206 comments)
ngcc_hk 4y ago
Objective c could be. You can have undeveloped method … if you can make any unbanked message invoke debugger and continue ? Not so sure about swift playground. One surprise may be mvs … it is not a Repl. But the idea is that the system cannot be stopped. The sysplex cannot be down because you have a sub-system or just a program error. You have to look at any core dump and fix the system whilst it is running. Obviously not development-Repl. Waterfall and punch card, thank you. But production-Repl. For production-Repl, basically all current systems needed it. There is no stop of your game world. Or may be yours but not the underlying world. It continues to run even on function upgrade or bugs etc. The real question is this production-Repl kind (not exactly Repl but the feature of keep the image running, debug or change one part of it and propagate the change without breaking the system) is really hard. Changing a s/370 control block or jes2 exit lately? Good luck and thanks you for all the fishes, that is why I go away.
revscat 4y ago
tl;dr: LISP and Smalltalk have the only True REPLs because only in those can you rebuild the interpreter and redefine classes without restarting. Everything else is just so cute with how hard they try.
- mikelevins 4y ago
  Its an amusing parody and I appreciate it. I don't actually think that Lisp and Smalltalk systems are the only possible examples of comprehensively interactive development systems. I'd say FORTH systems qualify (though they lack a lot of creature comforts) and so does Factor. Probably some others I'm not thinking of, as well. I also don't think it's impossible to build a proper repl-driven development environment with other languages. I just think that the needed work hasn't been done for very many other languages, and for perfectly understandable reasons. One reason is that it's a lot of extra work to build a proper repl-driven environment. I mean, you have to do all the work that would be needed for a non-repl-driven environment, and then you have a bunch more work to do on top of that. A second reason is that in order to build such an environment you have to be sufficiently familiar with them to know why you would want one, and then you also need to be the sort of programmer who actually prefers to work that way (or else why would you bother with all that extra work?). And, finally, you have to have the time, ambition, and other resources needed to actually build the thing. So it should be no surprise that proper repl-driven environments are the exception rather than the rule, but I still prefer them, so I want more people to know about them, in hopes that some fraction of people will find them preferable, as I do, and the demand for them will grow, and that demand will help ensure that they don't disappear. They're how I prefer to work. I don't want to lose them.
  - jrvarela56 4y ago
    Right now I'm in this conundrum. Should I setup a REPL env (and build the missing pieces) for web dev (JS/TS) or just adopt Clojure? Once you've worked that way, the regular way of programming seems archaic and a pita.
    - medo-bear 4y ago
mbil 4y ago
Used Clojure for ~4 years, doing REPL-driven development in Emacs. Have used Ruby and Java since. Maybe it's a rosy retrospection, but in terms of program production I think that was the most productive I've ever been. Falling into a flow state was much easier. Translating complex ideas into code was much easier. Building constructs up piece by piece felt natural and smooth. Sometimes I worry I'll never work in a LISP again.
- wawjgreen 4y ago
  Do you mean "falling into flow state was much easier" when you switched to java/ruby?
  - mbil 4y ago
    No, sorry, I meant it was easier back when I was using Clojure. Even invoking Java unit tests in the IDE or referencing a Ruby REPL in another window, while fast, are enough to break my concentration. Those seconds of context switching can derail my train of thought. At least in my case, building a piece of logic incrementally with near instant and continuous feedback from a REPL-integrated editor in Clojure just felt like an extension of my internal thought process.
- sorry_i_lisp 4y ago
  Not to make you feel too bad, but based on your sentence you may appreciate a recent addition to my personal REPL toolkit in Clojure which I very much enjoy. You may remember the rich comment forms in Clojure development as a communication tool to your future self and others. Now someone made them testable https://github.com/hyperfiddle/rcf#readme https://github.com/hyperfiddle/rcf#readme I feel this is even closer to the platonic ideal Clojure way of serializing your thoughts to file.
- thom 4y ago
  I also love Clojure and find it a very productive language. But surely the article makes it clear it falls very short of ‘real’ Lisp environments? There’s not much that Emacs and CIDER do that is so different to modes for other languages mentioned in the article with REPLs. If you’ve ever had stale protocol/record definitions in a Clojure REPL you know it doesn’t provide the features described. The whole ‘reloaded’ workflow is built around these limitations. All that said I’ve never really liked this idea of editing as you go with extreme late binding in the debugger. Just seems a recipe for non-reproducible code. I love being able to explore and experiment in the REPL, but ultimately everything coalesces into code and tests in files. These give me far more confidence to move forward than a long lived REPL session. There’s stuff like Reveal[0] which is interesting but I’ve only ever found it a marginal upgrade on pretty printing, at the cost that it sits outside Emacs so slows me down anyway. 0: https://vlaaad.github.io/reveal/ https://vlaaad.github.io/reveal/
cobaltoxide 4y ago
A peculiar workflow of REPL-driven programming environments is that the running program is eventually "saved" by writing the state of the virtual machine to disk. There may be no source code to rebuild from, if the program was built interactively. It has its appeal, but it's kind of a nightmare from a maintainability standpoint.
- ratzkewatzke 4y ago
  I write Lisp for a living (in a couple of different forms), and I've never seen this. You do occasionally get problems where the state drifts a bit and you end up with order dependencies in files, but everybody I know works on source code and pushes it back and forth to the REPL via their editor.
- Jtsummers 4y ago
  > There may be no source code to rebuild from, if the program was built interactively. If someone did that, they'd be a fool. Common Lisp and Smalltalk, the quintessential REPL-driven languages, have supported storing and loading source from files since their standards were published, and before then since the standards weren't created out of thin air.
- gumby 4y ago
  That was true of InterLisp-D and the SmallTalk environments, but not the case of the systems that begat CommonLisp (MACLISP, the lispms, franzlisp, etc). In other words it was a cultural thing, not a property of the languages. And even with the systems for which that was possible it was also possible to start with a fresh band and load code into it.
  - mikelevins 4y ago
    Moreover, Smalltalk and Interlisp interactions are recorded in files automatically--the changes file, the sources file, and the working image (which also contains tools--e.g. Interlisp's Masterscope--for keeping track of changes). The notion that you'll lose your work if you use the normal means of interacting with the repl is a misunderstanding of how such systems work.
ffhhj 4y ago
What happens if there is a data structure and I make some changes to it that break different parts of the program? Does it warn about them, or do I need to place some tests? Also, is it possible to revert the program state back and forth to try some idea?
- mikelevins 4y ago
  Lisp will tell you if it can, by dropping you in a breakloop. A breakloop is a repl that includes all of the normal Lisp repl features plus the full stack and dynamic environment of the broken code (which you can inspect and edit), plus a set of restarts you can select to resume execution, abandon the broken function, or transfer control in some other way defined by the context of the breakloop (which you can customize to your heart's content). Smalltalk has equivalent features. Lisp won't catch every possible error you could possibly make, of course, so testing is good defensive programming. On the other hand, when I'm working in Lisp, testing happens almost by default. It's the majority of what I do: I write experiments in files and hit a key to send them to the repl to test what happens. Tests generally start as little experiments like this. Some of them are useful over and over. Those graduate from informal experiments to formal tests.
- dunefox 4y ago
  This might be interesting for you: https://malisper.me/category/debugging-common-lisp/ https://malisper.me/category/debugging-common-lisp/
- lokedhs 4y ago
  This depends on on the language of course. In Common Lisp, a generics function called REINITIALIZE-INSTANCE is called. It's possibly to implement a method for this that updates the instance based on the new definition. By default, any slots that have been added will be unbound. Attempting to access them will raise an error which will drop you in the debugger. From there you can then fix the instance and continue running.
KSPAtlas 4y ago
I am yet to get truly repl-pilled, currently I just use files for bigger pieces of code, if there's a way to move functions and stuff into a file via SLIME I'm interested
- LesZedCB 4y ago
  clojure with Calva in VSCode is pretty amazing!
- mikelevins 4y ago
  The way I work (and have worked for years) is I just always work in a file buffer and use SLIME keystrokes to send expressions to Lisp for evaluation. Actually typing text in the repl buffer is very much the exception, not the rule.
jnash 4y ago
How do you use this approach on a team with shared code?
- Jtsummers 4y ago
  With source files and git. They aren't incompatible with using a REPL to drive your work.
wnolens 4y ago
> A breakloop is a full-featured repl, complete with all of the tools of the main repl, but it exists inside the dynamic environment of the broken function Sweeeet. Well damn.. here I am in my python repl for hours per day, feeling smug about how efficient I am pulling logs and analyzing data. But is there an AWS lisp sdk? Edit: there is but not officially. Sounds fun but lisp seems so much less interoperable (out of the box) than python, and more of a departure from the style of everyday engineering. I love my Python repl tools.
- sorry_i_lisp 4y ago
  Clojure is the ultimate interop language to me. ClojureScript (shadow-cljs, nbb) for JavaScript Clojure for JVM babashka for scripts libpython-clj for python interop. ClojureCLR for .NET (beta state IMO) ClojureDart for Dart/Flutter (early alpha state, one big shipped prod app on iOS/android) --- Edit: I don't quite know how to describe it, but having the _exact_ same files in a full-stack web application being read in both the Clojure backend and ClojureScript frontend (API contract) feels pretty neat. As an example having the frontend accurately give the user feedback why his input form isn't going to be accepted without a server round-trip or duplicating code just feels elegant.
  - BeFlatXIII 4y ago
    I wonder if there is Clojure for the BEAM
    - fredrikholm 4y ago
      Why yes! https://github.com/clojerl/clojerl https://github.com/clojerl/clojerl
  - wnolens 4y ago
    Ok interesting thanks.
sieste 4y ago
What are good resources to learn "proper" repl-driven development in lisp?
- medo-bear 4y ago
  not all lisps have the same level of repl-driven development. clojure seems to be good, but it has nothing on common lisp. common lisp has this designed into it anyway, check out https://www.youtube.com/user/CBaggers/videos https://www.youtube.com/user/CBaggers/videos https://github.com/rabbibotton/clog/blob/main/LEARN.md https://github.com/rabbibotton/clog/blob/main/LEARN.md edit: https://malisper.me/category/debugging-common-lisp/ https://malisper.me/category/debugging-common-lisp/ (from another poster)
vegai_ 4y ago
The mistake that a lot of "technique x" aficiandos make seems to often be that the technique has to be 100% complete or it's completely worthless. In this case, there's a claim that python isn't actually REPL-based because you cannot change state while in an exception handler. Yes, that's kinda true, but there are debuggers for python that if you really need to can do a lot of that. Other notable examples of this are type system maximalists. E.g. Go: no sum types or generics (until recently). Rust: no HKTs. Python: no runtime checks (I actually sometimes think this :)).
abrax3141 4y ago
I've been programming in lisp pretty much daily since the 1970s. I've also programmed in probably 50 other languages under as many different development methodologies. The methodology described in the posting does exist, but it is never used for development; it is, however, very commonly used in three settings: (1) one-shot short programming [1SP], (2) exploratory programming [XP], and (3) interactive debugging [ID]. I just last night did an examples of 1SP: I was trying to work out the equation for a combinatorial probability, and wanted to simulate it so that I could tell what the right answer should be. I wrote the simulation expression right into the REPL -- not even in EMACS (my usual mode of programming is emacs+shell(usually lisp) -- I'm so old-timey that I never even bothered to install SLIME!) It took a couple copy-paste fixes to get it to work, but in a couple minutes, I had my result. I do XP all the time, but usually on a single function, or small set of them, often when I'm trying to make a complex recursion work. This is like 1SP, but I'm not going to throw away the code; to the contrary, I'm going to copy the final, hopefully working, code into the body of my program. Therefore, I'll often do XP in an emacs+shell(lisp). ID needs no further explanation; this is well described in the posting, and is, as mentioned, quite a wonderful process. (Sometimes ID and XP are mixed, as suggested in the posting.) What never even happens is complete software development in lisp "core" -- what's sometimes called "workspace" development. This can be done, but it's a terrible software engineering methodology. (APL, which I've also done quite a lot of, although not so much anymore, has had a REPL since around the same time as Lisp, and has always worked in the workspace parading, and it was always a terrible mess for anything very complex.) [BASIC also had a workspace paradigm and REPL since around the same time as Lisp, but no one did anything big in the early BASICs, so this was fine. They were mostly used by student on early timesharing systems.] One very important and under-appreciated property of Lisp that really makes interactive programming work so well in Lisp, and not so well in most other languages, is that line turns and white space make no difference at all to Lisp; it's all from the first open paren to the last close, so you can copy-paste whole expressions into the place of variables, if you like (more commonly the other way around, I guess), and there's never any question about it being syntactically correct. For example, I take this expression: (foo (bar (baz ...)) (bar (baz ...))) where, say, (bar (baz ...)) is a heavy computation, so I want to refactor it. All I need to do is pull the expression (bar (baz ...)) out using paren-matching, wrap the thing in a let, and bob's your uncle, as: (let ((x (bar (baz ...)))) (foo x x)) But the thing to note it that the expression (bar (baz ...)) could be 50 lines long! As I said above, line turns and white space (aside from enough white space to break up obvious things, like the names of separate variables), are mere decoration to Lisp, so you can decorate, or not, all you want. This facility to not care at all about the lines and spacing, which most other languages care about in one or another way; make interactive programming a real joy.

100 points by ivanvas 4y ago

Part of why this [EDIT: to clarify - the debug/break loop process paired with the REPL] works in Smalltalk and Lisp is their dynamic nature, and also to the fact that they are systems as much as languages. In principle, Ruby and Python should be able to have as capable dev environments. With more static languages like C, Java, C#, and others you run into the problem of needing more "skeleton". You can't write: void foo() { bar(); } in those languages and expect the compiler to be thrilled about it. But you could get closer. Java runs on the JVM, C# on the CLR, those provide systems on which the languages (or programs written in those languages) are executed and the kind of access needed to approach the debugger-driven development approach. I haven't done this in a long time but with C# you can do: void foo() { throw new NotImplementedException(); } Now when your program execution hits `foo` you can catch it in the debugger (versus just crashing or otherwise terminating, or needing to set a breakpoint). With VS (and probably others, I did this with VS) you can change this code and recompile it and resume from that point. Now you decide to make the call to `bar`, oops, that method doesn't exist yet so you also have to create `bar`, and you can stub it out the exact same way. So it's not as easy and pleasant as in Lisp or Smalltalk where you can forgo the skeletal framework of the program and jump straight into it, building each function, method, class, etc. only as needed (truly JIT development), but you can get close. I believe Java has some similar facilities with some dev environments. There is more work, but it's not totally impossible. All that said, I really enjoy the way you can do this in Lisp (and Smalltalk). I've been debating re-solving the Synacor Challenge [0] and recording how I did it in Lisp the first time. I used the debugger in precisely this way, writing a basic version which could read in the binary blob and did minimal parsing, but then entered an ecase (case expression that errors if there is no matching case). For every instruction that I hadn't implemented (none of them at the start) it brought me to the debugger, I modified the program, resumed it and it continued until the next unimplemented instruction. By the time I'd finished, I had the interpreter for the byte code up and running and was immediately getting to work on the rest of the challenge. This also worked as a nice goal-directed approach, implementing the whole interpreter was a "big" (not really) task, how to break it down and which part to do first? Well, interpreting the binary blob acted as my (very complex) test case and gave me immediate feedback (I also added some basic debugger capabilities to the interpreter) on whether I had done it correctly. No need to make the whole thing before testing it, I got to incrementally test each small change. That approach could also be made to work with C# as I described before, write a basic skeleton and parse the instructions but throw a `NotImplementedException` whenever you get to an instruction you haven't, well, implemented yet. Then using VS change the throw to a method call (or do it locally) that implements the handler for that instruction. Repeat until all of them are filled out. It's not as smooth as the Lisp or Smalltalk cases would be, but it's doable. [0] https://challenge.synacor.com https://challenge.synacor.com - programming challenge by the same guy that does Advent of Code. ---------- Also, this has come up on here a few times in the past since it was written (2020). Two past submissions with comments: https://news.ycombinator.com/item?id=25620256 https://news.ycombinator.com/item?id=25620256 - Jan 3, 2021 (208 comments) https://news.ycombinator.com/item?id=25618457 https://news.ycombinator.com/item?id=25618457 - Jan 3, 2021 (5 comments, 4 + 1 dead, don't know why it's dead)

tester756 4y ago
I do often use this approach in C#, so I started calling it debugger driven development Seems like I should try Smalltalk
waf 4y ago
For what it's worth, modern C# can evaluate top-level statements with no other ceremony. The following is a valid standalone C# program: Console.WriteLine("Hello"); For REPLs, there are options like (my own) https://github.com/waf/CSharpRepl https://github.com/waf/CSharpRepl which stand on top of the Roslyn compiler infrastructure, which is quite extensive and can easily evaluate standalone functions and statements. It's still nowhere close to the REPLs of lisp and smalltalk, but it's a step in a more flexible direction.
- Jtsummers 4y ago
  True, I'd forgotten about that. That was in the last major release, right?

dang 4y ago

Related: On Repl-Driven Programming - https://news.ycombinator.com/item?id=25620256 https://news.ycombinator.com/item?id=25620256 - Jan 2021 (206 comments)

ngcc_hk 4y ago

Objective c could be. You can have undeveloped method … if you can make any unbanked message invoke debugger and continue ? Not so sure about swift playground. One surprise may be mvs … it is not a Repl. But the idea is that the system cannot be stopped. The sysplex cannot be down because you have a sub-system or just a program error. You have to look at any core dump and fix the system whilst it is running. Obviously not development-Repl. Waterfall and punch card, thank you. But production-Repl. For production-Repl, basically all current systems needed it. There is no stop of your game world. Or may be yours but not the underlying world. It continues to run even on function upgrade or bugs etc. The real question is this production-Repl kind (not exactly Repl but the feature of keep the image running, debug or change one part of it and propagate the change without breaking the system) is really hard. Changing a s/370 control block or jes2 exit lately? Good luck and thanks you for all the fishes, that is why I go away.

revscat 4y ago

tl;dr: LISP and Smalltalk have the only True REPLs because only in those can you rebuild the interpreter and redefine classes without restarting. Everything else is just so cute with how hard they try.

mikelevins 4y ago
Its an amusing parody and I appreciate it. I don't actually think that Lisp and Smalltalk systems are the only possible examples of comprehensively interactive development systems. I'd say FORTH systems qualify (though they lack a lot of creature comforts) and so does Factor. Probably some others I'm not thinking of, as well. I also don't think it's impossible to build a proper repl-driven development environment with other languages. I just think that the needed work hasn't been done for very many other languages, and for perfectly understandable reasons. One reason is that it's a lot of extra work to build a proper repl-driven environment. I mean, you have to do all the work that would be needed for a non-repl-driven environment, and then you have a bunch more work to do on top of that. A second reason is that in order to build such an environment you have to be sufficiently familiar with them to know why you would want one, and then you also need to be the sort of programmer who actually prefers to work that way (or else why would you bother with all that extra work?). And, finally, you have to have the time, ambition, and other resources needed to actually build the thing. So it should be no surprise that proper repl-driven environments are the exception rather than the rule, but I still prefer them, so I want more people to know about them, in hopes that some fraction of people will find them preferable, as I do, and the demand for them will grow, and that demand will help ensure that they don't disappear. They're how I prefer to work. I don't want to lose them.
- jrvarela56 4y ago
  Right now I'm in this conundrum. Should I setup a REPL env (and build the missing pieces) for web dev (JS/TS) or just adopt Clojure? Once you've worked that way, the regular way of programming seems archaic and a pita.
  - medo-bear 4y ago

mbil 4y ago

Used Clojure for ~4 years, doing REPL-driven development in Emacs. Have used Ruby and Java since. Maybe it's a rosy retrospection, but in terms of program production I think that was the most productive I've ever been. Falling into a flow state was much easier. Translating complex ideas into code was much easier. Building constructs up piece by piece felt natural and smooth. Sometimes I worry I'll never work in a LISP again.

wawjgreen 4y ago
Do you mean "falling into flow state was much easier" when you switched to java/ruby?
- mbil 4y ago
  No, sorry, I meant it was easier back when I was using Clojure. Even invoking Java unit tests in the IDE or referencing a Ruby REPL in another window, while fast, are enough to break my concentration. Those seconds of context switching can derail my train of thought. At least in my case, building a piece of logic incrementally with near instant and continuous feedback from a REPL-integrated editor in Clojure just felt like an extension of my internal thought process.
sorry_i_lisp 4y ago
Not to make you feel too bad, but based on your sentence you may appreciate a recent addition to my personal REPL toolkit in Clojure which I very much enjoy. You may remember the rich comment forms in Clojure development as a communication tool to your future self and others. Now someone made them testable https://github.com/hyperfiddle/rcf#readme https://github.com/hyperfiddle/rcf#readme I feel this is even closer to the platonic ideal Clojure way of serializing your thoughts to file.
thom 4y ago
I also love Clojure and find it a very productive language. But surely the article makes it clear it falls very short of ‘real’ Lisp environments? There’s not much that Emacs and CIDER do that is so different to modes for other languages mentioned in the article with REPLs. If you’ve ever had stale protocol/record definitions in a Clojure REPL you know it doesn’t provide the features described. The whole ‘reloaded’ workflow is built around these limitations. All that said I’ve never really liked this idea of editing as you go with extreme late binding in the debugger. Just seems a recipe for non-reproducible code. I love being able to explore and experiment in the REPL, but ultimately everything coalesces into code and tests in files. These give me far more confidence to move forward than a long lived REPL session. There’s stuff like Reveal[0] which is interesting but I’ve only ever found it a marginal upgrade on pretty printing, at the cost that it sits outside Emacs so slows me down anyway. 0: https://vlaaad.github.io/reveal/ https://vlaaad.github.io/reveal/

cobaltoxide 4y ago

A peculiar workflow of REPL-driven programming environments is that the running program is eventually "saved" by writing the state of the virtual machine to disk. There may be no source code to rebuild from, if the program was built interactively. It has its appeal, but it's kind of a nightmare from a maintainability standpoint.

ratzkewatzke 4y ago
I write Lisp for a living (in a couple of different forms), and I've never seen this. You do occasionally get problems where the state drifts a bit and you end up with order dependencies in files, but everybody I know works on source code and pushes it back and forth to the REPL via their editor.
Jtsummers 4y ago
> There may be no source code to rebuild from, if the program was built interactively. If someone did that, they'd be a fool. Common Lisp and Smalltalk, the quintessential REPL-driven languages, have supported storing and loading source from files since their standards were published, and before then since the standards weren't created out of thin air.
gumby 4y ago
That was true of InterLisp-D and the SmallTalk environments, but not the case of the systems that begat CommonLisp (MACLISP, the lispms, franzlisp, etc). In other words it was a cultural thing, not a property of the languages. And even with the systems for which that was possible it was also possible to start with a fresh band and load code into it.
- mikelevins 4y ago
  Moreover, Smalltalk and Interlisp interactions are recorded in files automatically--the changes file, the sources file, and the working image (which also contains tools--e.g. Interlisp's Masterscope--for keeping track of changes). The notion that you'll lose your work if you use the normal means of interacting with the repl is a misunderstanding of how such systems work.

ffhhj 4y ago

What happens if there is a data structure and I make some changes to it that break different parts of the program? Does it warn about them, or do I need to place some tests? Also, is it possible to revert the program state back and forth to try some idea?

mikelevins 4y ago
Lisp will tell you if it can, by dropping you in a breakloop. A breakloop is a repl that includes all of the normal Lisp repl features plus the full stack and dynamic environment of the broken code (which you can inspect and edit), plus a set of restarts you can select to resume execution, abandon the broken function, or transfer control in some other way defined by the context of the breakloop (which you can customize to your heart's content). Smalltalk has equivalent features. Lisp won't catch every possible error you could possibly make, of course, so testing is good defensive programming. On the other hand, when I'm working in Lisp, testing happens almost by default. It's the majority of what I do: I write experiments in files and hit a key to send them to the repl to test what happens. Tests generally start as little experiments like this. Some of them are useful over and over. Those graduate from informal experiments to formal tests.
dunefox 4y ago
This might be interesting for you: https://malisper.me/category/debugging-common-lisp/ https://malisper.me/category/debugging-common-lisp/
lokedhs 4y ago
This depends on on the language of course. In Common Lisp, a generics function called REINITIALIZE-INSTANCE is called. It's possibly to implement a method for this that updates the instance based on the new definition. By default, any slots that have been added will be unbound. Attempting to access them will raise an error which will drop you in the debugger. From there you can then fix the instance and continue running.

KSPAtlas 4y ago

I am yet to get truly repl-pilled, currently I just use files for bigger pieces of code, if there's a way to move functions and stuff into a file via SLIME I'm interested

LesZedCB 4y ago
clojure with Calva in VSCode is pretty amazing!
mikelevins 4y ago
The way I work (and have worked for years) is I just always work in a file buffer and use SLIME keystrokes to send expressions to Lisp for evaluation. Actually typing text in the repl buffer is very much the exception, not the rule.

jnash 4y ago

How do you use this approach on a team with shared code?

Jtsummers 4y ago
With source files and git. They aren't incompatible with using a REPL to drive your work.

wnolens 4y ago

> A breakloop is a full-featured repl, complete with all of the tools of the main repl, but it exists inside the dynamic environment of the broken function Sweeeet. Well damn.. here I am in my python repl for hours per day, feeling smug about how efficient I am pulling logs and analyzing data. But is there an AWS lisp sdk? Edit: there is but not officially. Sounds fun but lisp seems so much less interoperable (out of the box) than python, and more of a departure from the style of everyday engineering. I love my Python repl tools.

sorry_i_lisp 4y ago
Clojure is the ultimate interop language to me. ClojureScript (shadow-cljs, nbb) for JavaScript Clojure for JVM babashka for scripts libpython-clj for python interop. ClojureCLR for .NET (beta state IMO) ClojureDart for Dart/Flutter (early alpha state, one big shipped prod app on iOS/android) --- Edit: I don't quite know how to describe it, but having the _exact_ same files in a full-stack web application being read in both the Clojure backend and ClojureScript frontend (API contract) feels pretty neat. As an example having the frontend accurately give the user feedback why his input form isn't going to be accepted without a server round-trip or duplicating code just feels elegant.
- BeFlatXIII 4y ago
  I wonder if there is Clojure for the BEAM
  - fredrikholm 4y ago
    Why yes! https://github.com/clojerl/clojerl https://github.com/clojerl/clojerl
- wnolens 4y ago
  Ok interesting thanks.

sieste 4y ago

What are good resources to learn "proper" repl-driven development in lisp?

medo-bear 4y ago
not all lisps have the same level of repl-driven development. clojure seems to be good, but it has nothing on common lisp. common lisp has this designed into it anyway, check out https://www.youtube.com/user/CBaggers/videos https://www.youtube.com/user/CBaggers/videos https://github.com/rabbibotton/clog/blob/main/LEARN.md https://github.com/rabbibotton/clog/blob/main/LEARN.md edit: https://malisper.me/category/debugging-common-lisp/ https://malisper.me/category/debugging-common-lisp/ (from another poster)

vegai_ 4y ago

The mistake that a lot of "technique x" aficiandos make seems to often be that the technique has to be 100% complete or it's completely worthless. In this case, there's a claim that python isn't actually REPL-based because you cannot change state while in an exception handler. Yes, that's kinda true, but there are debuggers for python that if you really need to can do a lot of that. Other notable examples of this are type system maximalists. E.g. Go: no sum types or generics (until recently). Rust: no HKTs. Python: no runtime checks (I actually sometimes think this :)).

abrax3141 4y ago

I've been programming in lisp pretty much daily since the 1970s. I've also programmed in probably 50 other languages under as many different development methodologies. The methodology described in the posting does exist, but it is never used for development; it is, however, very commonly used in three settings: (1) one-shot short programming [1SP], (2) exploratory programming [XP], and (3) interactive debugging [ID]. I just last night did an examples of 1SP: I was trying to work out the equation for a combinatorial probability, and wanted to simulate it so that I could tell what the right answer should be. I wrote the simulation expression right into the REPL -- not even in EMACS (my usual mode of programming is emacs+shell(usually lisp) -- I'm so old-timey that I never even bothered to install SLIME!) It took a couple copy-paste fixes to get it to work, but in a couple minutes, I had my result. I do XP all the time, but usually on a single function, or small set of them, often when I'm trying to make a complex recursion work. This is like 1SP, but I'm not going to throw away the code; to the contrary, I'm going to copy the final, hopefully working, code into the body of my program. Therefore, I'll often do XP in an emacs+shell(lisp). ID needs no further explanation; this is well described in the posting, and is, as mentioned, quite a wonderful process. (Sometimes ID and XP are mixed, as suggested in the posting.) What never even happens is complete software development in lisp "core" -- what's sometimes called "workspace" development. This can be done, but it's a terrible software engineering methodology. (APL, which I've also done quite a lot of, although not so much anymore, has had a REPL since around the same time as Lisp, and has always worked in the workspace parading, and it was always a terrible mess for anything very complex.) [BASIC also had a workspace paradigm and REPL since around the same time as Lisp, but no one did anything big in the early BASICs, so this was fine. They were mostly used by student on early timesharing systems.] One very important and under-appreciated property of Lisp that really makes interactive programming work so well in Lisp, and not so well in most other languages, is that line turns and white space make no difference at all to Lisp; it's all from the first open paren to the last close, so you can copy-paste whole expressions into the place of variables, if you like (more commonly the other way around, I guess), and there's never any question about it being syntactically correct. For example, I take this expression: (foo (bar (baz ...)) (bar (baz ...))) where, say, (bar (baz ...)) is a heavy computation, so I want to refactor it. All I need to do is pull the expression (bar (baz ...)) out using paren-matching, wrap the thing in a let, and bob's your uncle, as: (let ((x (bar (baz ...)))) (foo x x)) But the thing to note it that the expression (bar (baz ...)) could be 50 lines long! As I said above, line turns and white space (aside from enough white space to break up obvious things, like the names of separate variables), are mere decoration to Lisp, so you can decorate, or not, all you want. This facility to not care at all about the lines and spacing, which most other languages care about in one or another way; make interactive programming a real joy.