4 ms·

Which CL "interactive development" feature can't you get with either a jupyter notebook or ipython + autoreload enables? With a smart editor like VSCode or Spyd

Which CL "interactive development" feature can't you get with either a jupyter notebook or ipython + autoreload enables? With a smart editor like VSCode or Spyder you also get visual variable inspections, history of executed code in one way or another etc. Only that can't easily save an "image" auto-magically, but data science and ML people wouldn't want to do this other than explicitly (eg. using fast loading formats like feather to reload semi-processed vars etc.). I don't think you are aware how much the Python workflows of AI / ML / datasci people resemble you "common lisp ideal" ...but with a different language and better libraries :) And if Julia succeeded, it will also bring decent performance to such workflows. Sure, the way backend devs or devops people use Python is very static and different, but any experienced ML practitioners are at home with "interactive development".

reikonomusha 4y ago
Re-define a class and have all objects automatically get updated to reflect the changes. Evaluate arbitrary expressions mid-stack trace. Run into a bug, get caught in an error, re-define the offensive function interactively, and re-run the computation before that point, all without stopping and restarting. (Example real-world use-case: You re-define one of your REST API endpoints without having to restart your server or interrupt the request-flow, be it in test or—if you so desire—in prod.) Change the class of an object at run-time. Re-compile and re-load individual functions from your editor with your production, git-committed code. This one is huge. You use interactive and incremental development to, well, develop! Not side-by-side with your code, but in your code as well, at a fine-enough grained level. Common Lisp was designed to be both interactive and incremental. It's not a bolt-on feature. Jupyter notebooks and auto-reloading have a very different feel, and (dis)incentivize different qualities and processes of development.
- catchclose8919 4y ago
  Basically all you mentioned above can be done in modern Python. Unfortunately I've seen it done... and God, the results are horrible, in Python. "Wait did calling this method just change the class of the object the method was on??!!!! Fuuuu..." But simple stuff like incrementally building classes method by method in eg. jupyter notebooks can be done and it's even quite clean, it's just that nobody is teaching people the techniques for doing it... (And the jupyter autoreload magic doesn't seem to like it so you'd have to "do it by hand") And sure, since Python got scoping and concurrency quite wrong, if you go full-interactive on things you'll shoot yourself in the foot faaast!
vindarel 4y ago
This set of jupyter + autoreloads + editor features might get you close, but you won't be on par to CL's level. I'm thinking about CL's interactive debugger, and about language features that were thought with interactivity in mind. On an error, you get the interactive debugger, you can explore the stack frames and get to the erroneous function, change it, recompile it, come back to the debugger and ask it to resume progress from this point, and see execution pass. You didn't have to re-run the whole operation. If we can do that in Python, I've been in my cave for too long^^ As such, this is how debugging a complex program looks like in Python vs in CL: https://lisp-journey.gitlab.io/images/debugging-python-VS-lisp.png https://lisp-journey.gitlab.io/images/debugging-python-VS-li... (Ravenpack [1] is a company that does big data analysis and that uses CL, they gave some talks, maybe you'll find more practical reasons from them) for the second point: https://mikelevins.github.io/posts/2020-12-18-repl-driven/ https://mikelevins.github.io/posts/2020-12-18-repl-driven/ There might be more CL libraries than you think: https://github.com/CodyReichert/awesome-cl https://github.com/CodyReichert/awesome-cl but I'm not that a zealot to say you can ignore Python's ecosystem, especially in certain areas… [1]: https://www.ravenpack.com/ https://www.ravenpack.com/
- catchclose8919 4y ago
  > If we can do that in Python, I've been in my cave for too long^^ ...practically speaking you're kind of right, you (almost) can't. In theory one can use `sys.excepthook` to start the python interactive debugger on error and/or %debug jupyter magic, but the "change it, recompile it, come back to the debugger" is not really possible unless you the yuuuuucky thing of changing it from the interactive debugger itself (think change def in code, paster changed def in interactive debugger). As a python expert you can do it. Even modify classes at runtime etc. Tried at various times doing the various things mentioned above. But tbh you learn to rarely do it because you easily get in a twilight zone hell where it's impossible to reproduce anything, no idea what state of code is "active" and what your functions and variables actually are :) ...but basically what Python lacks to enable proper implementations of such feature is decent concurrency - while a thread is paused at the breakloop, the interpreter should allow another to eg. execute code that updates functions and classes definitions - all Jupyter's magic can't do this bc concurrency in the Python interpreter is not doable :) ...in theory the Julia jupyter kernel for example could actually implement all the features mentioned by people describing CL loops - just that someone would need to put the tons of work in, and unfortunately Julia folks seem focused on the automagic-reactive approach to recreating excell-but-as-a-notebook (I don't like this magic-reactive approach tbh... try doing that and have a random variable change "accidentally" triggering recomputing something with a 10gb dataframe you had in var or the retraining of a big deep learning model).
omaranto 4y ago
I don't think it's fair that you got down-voted for not knowing what Lisp-style interactive development offers beyond what ipython or jupyter offers. Many people don't know either and the proper response would be to explain, not to down-vote. EDIT: I was made aware of the guidelines to not comment on voting, I apologize.
- throw10920 4y ago
  > Please don't comment about the voting on comments. It never does any good, and it makes boring reading. From the HN guidelines at https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
  - deleted 4y ago
    [deleted]
throw10920 4y ago
In addition to the comprehensive answers of reikonomusha and vindarel, I'll give you some specific things you can't do: Jupyter notebook: you can't run it as normal code - that is, you can't import your notebook from another file. You also can't evaluate arbitrary regions of the notebook (it's cells, the whole thing, or nothing), or nested sub-expressions. You can't evaluate code in parallel - if you have a cell calling a function foo() in a loop, and you try to redefine foo() in another cell - impossible in Juypter, trivial in CL. iPython + autoreload: you also can't redefine a foo() in a file when it's being called in a loop, docs state "reloads modules automatically before entering the execution of code typed at the IPython prompt". You don't get any sort of interactive debug-and-fix-up when you encounter an error. And so on.
- breuleux 4y ago
  For the record, I have written a reloading library for Python called jurigged[0] that works on normal Python code and can indeed replace foo() when it's being called in a loop. There is also a way to do debug-and-fix-up, but only if you specify restart points in advance, unfortunately. It is probably still a good deal more limited to what can be done with CL, there are no miracles here, just saying the gap may not be that big. [0] https://github.com/breuleux/jurigged https://github.com/breuleux/jurigged

reikonomusha 4y ago

Re-define a class and have all objects automatically get updated to reflect the changes. Evaluate arbitrary expressions mid-stack trace. Run into a bug, get caught in an error, re-define the offensive function interactively, and re-run the computation before that point, all without stopping and restarting. (Example real-world use-case: You re-define one of your REST API endpoints without having to restart your server or interrupt the request-flow, be it in test or—if you so desire—in prod.) Change the class of an object at run-time. Re-compile and re-load individual functions from your editor with your production, git-committed code. This one is huge. You use interactive and incremental development to, well, develop! Not side-by-side with your code, but in your code as well, at a fine-enough grained level. Common Lisp was designed to be both interactive and incremental. It's not a bolt-on feature. Jupyter notebooks and auto-reloading have a very different feel, and (dis)incentivize different qualities and processes of development.

catchclose8919 4y ago
Basically all you mentioned above can be done in modern Python. Unfortunately I've seen it done... and God, the results are horrible, in Python. "Wait did calling this method just change the class of the object the method was on??!!!! Fuuuu..." But simple stuff like incrementally building classes method by method in eg. jupyter notebooks can be done and it's even quite clean, it's just that nobody is teaching people the techniques for doing it... (And the jupyter autoreload magic doesn't seem to like it so you'd have to "do it by hand") And sure, since Python got scoping and concurrency quite wrong, if you go full-interactive on things you'll shoot yourself in the foot faaast!

vindarel 4y ago

This set of jupyter + autoreloads + editor features might get you close, but you won't be on par to CL's level. I'm thinking about CL's interactive debugger, and about language features that were thought with interactivity in mind. On an error, you get the interactive debugger, you can explore the stack frames and get to the erroneous function, change it, recompile it, come back to the debugger and ask it to resume progress from this point, and see execution pass. You didn't have to re-run the whole operation. If we can do that in Python, I've been in my cave for too long^^ As such, this is how debugging a complex program looks like in Python vs in CL: https://lisp-journey.gitlab.io/images/debugging-python-VS-lisp.png https://lisp-journey.gitlab.io/images/debugging-python-VS-li... (Ravenpack [1] is a company that does big data analysis and that uses CL, they gave some talks, maybe you'll find more practical reasons from them) for the second point: https://mikelevins.github.io/posts/2020-12-18-repl-driven/ https://mikelevins.github.io/posts/2020-12-18-repl-driven/ There might be more CL libraries than you think: https://github.com/CodyReichert/awesome-cl https://github.com/CodyReichert/awesome-cl but I'm not that a zealot to say you can ignore Python's ecosystem, especially in certain areas… [1]: https://www.ravenpack.com/ https://www.ravenpack.com/

catchclose8919 4y ago
> If we can do that in Python, I've been in my cave for too long^^ ...practically speaking you're kind of right, you (almost) can't. In theory one can use `sys.excepthook` to start the python interactive debugger on error and/or %debug jupyter magic, but the "change it, recompile it, come back to the debugger" is not really possible unless you the yuuuuucky thing of changing it from the interactive debugger itself (think change def in code, paster changed def in interactive debugger). As a python expert you can do it. Even modify classes at runtime etc. Tried at various times doing the various things mentioned above. But tbh you learn to rarely do it because you easily get in a twilight zone hell where it's impossible to reproduce anything, no idea what state of code is "active" and what your functions and variables actually are :) ...but basically what Python lacks to enable proper implementations of such feature is decent concurrency - while a thread is paused at the breakloop, the interpreter should allow another to eg. execute code that updates functions and classes definitions - all Jupyter's magic can't do this bc concurrency in the Python interpreter is not doable :) ...in theory the Julia jupyter kernel for example could actually implement all the features mentioned by people describing CL loops - just that someone would need to put the tons of work in, and unfortunately Julia folks seem focused on the automagic-reactive approach to recreating excell-but-as-a-notebook (I don't like this magic-reactive approach tbh... try doing that and have a random variable change "accidentally" triggering recomputing something with a 10gb dataframe you had in var or the retraining of a big deep learning model).

omaranto 4y ago

I don't think it's fair that you got down-voted for not knowing what Lisp-style interactive development offers beyond what ipython or jupyter offers. Many people don't know either and the proper response would be to explain, not to down-vote. EDIT: I was made aware of the guidelines to not comment on voting, I apologize.

throw10920 4y ago
> Please don't comment about the voting on comments. It never does any good, and it makes boring reading. From the HN guidelines at https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- deleted 4y ago
  [deleted]

throw10920 4y ago

In addition to the comprehensive answers of reikonomusha and vindarel, I'll give you some specific things you can't do: Jupyter notebook: you can't run it as normal code - that is, you can't import your notebook from another file. You also can't evaluate arbitrary regions of the notebook (it's cells, the whole thing, or nothing), or nested sub-expressions. You can't evaluate code in parallel - if you have a cell calling a function foo() in a loop, and you try to redefine foo() in another cell - impossible in Juypter, trivial in CL. iPython + autoreload: you also can't redefine a foo() in a file when it's being called in a loop, docs state "reloads modules automatically before entering the execution of code typed at the IPython prompt". You don't get any sort of interactive debug-and-fix-up when you encounter an error. And so on.

breuleux 4y ago
For the record, I have written a reloading library for Python called jurigged[0] that works on normal Python code and can indeed replace foo() when it's being called in a loop. There is also a way to do debug-and-fix-up, but only if you specify restart points in advance, unfortunately. It is probably still a good deal more limited to what can be done with CL, there are no miracles here, just saying the gap may not be that big. [0] https://github.com/breuleux/jurigged https://github.com/breuleux/jurigged