5 ms·

For those unaware, Common Lisp exception handling is heads and shoulders above almost any other programming language: Imagine having your program crash in the m

by drcode 6y ago

For those unaware, Common Lisp exception handling is heads and shoulders above almost any other programming language: Imagine having your program crash in the middle of a function, being able to inspect the call stack to find the bug, fixing it, then completing the function call from the middle of the function, at the exact point where the exception happened, with the fix in place.

tester34 6y ago
During debugging, yea?
- phoe-krk 6y ago
  Yes, all that without leaving the debugger. The debugger is left only when we want to continue execution of the program.
  - protomyth 6y ago
    I've noticed quite a few Smalltalk programmers who actually coded in the debugger. They would put a call in the program, hit it as an error, and then write new code. Does this happen with Common Lisp programmers?
    - phoe-krk 6y ago
      Smalltalk and Common Lisp are both image-based programming languages where the compiler and debugger are always available. I think debugger-oriented programming is much more common in Smalltalk than in Lisp (at least I prefer to leave the debugger back to REPL pretty often), but I think it's certainly possible to adopt. The Slime/Sly environments for emacs might accept a few patches to facilitate that programming style better.
    - p_l 6y ago
      I know of several lispers that swear by top-down "programming inside debugger", where they continuously fill in the gaps starting from top level function :-D
      - phoe-krk 6y ago
        Do you have any videos of such programming style performed in Lisp, or could you try to convince some of those programmers to record some of their programming sessions?
        rags2riches 6y ago
        You could look up Baggers on Youtube for plenty of fun Lisp coding sessions.
        phoe-krk 6y ago
        I know Baggers and his videos but I wasn't aware that he does debugger-oriented programming. Thanks, I'll pay closer attention to them!
- aidenn0 6y ago
  For this example, it would be during debugging. You could intentionally write a bug, and then write a condition handler that would fix the bug without unwinding the stack further than necessary, but that would be rather pointless. However think of all the things that can go wrong with your environment that don't involve a bug in your program. A network connection goes down, the disk gets full, &c. All of those can be handled without unwinding the stack to the point where the code for handling them was defined. For a really trivial example, consider a performance watchdog. If you want to completely abort the attempt and log a call-stack, you can do that it just about any modern, dynamic language because the exception object includes the call-stack. But what if you want to log the call-stack and continue for the first N times the watchdog rolls-over? This is, of course, doable in other languages with passing callbacks and such, but in Lisp this can be done by signalling a condition in the exact same manner that an exceptional condition would be signaled. I'm excited about this book, because that particular feature is underutilized in Lisp (by myself included), probably because it's one of the last dynamic things that Lisp does that more well-known dynamic languages do not, so programmers coming from other languages just aren't going to use it.
  - phoe-krk 6y ago
    > All of those can be handled without unwinding the stack to the point where the code for handling them was defined. Nitpick: the only thing that cannot be handled by winding the stack is when you run out of stack to wind. But, luckily, if your program overflows the stack, you usually have bigger issues to deal with and/or you usually have all the stack information you need to diagnose the problem.
phoe-krk 6y ago
Additionally, one of the most tasty facts about it is that the Common Lisp condition system is written in Common Lisp itself. It's literally a Lisp plugin that only needs to be integrated with the rest of the system at its end points: where the conditions are signaled and where the debugger is entered.
- lukashrb 6y ago
  Does anyone know if it is possible to integrate such a system, as plugin, in clojure? Is there already a project like that?
  - phoe-krk 6y ago
    I haven't explored Clojure. If it allows one to customize the means in which errors are signaled (read: force the language to not immediately throw a Java exception on an error, and instead execute some code), then one should be able to implement both signaling and a debugger. It might not be easy to tell Clojure or the JVM to not be trigger happy with throwing exceptions and therefore destroying the stack. Perhaps one will find a way through!
  - divs1210 6y ago
    Yes, there are multiple libraries, for example: https://github.com/clojureman/special https://github.com/clojureman/special
    - phoe-krk 6y ago
      Thank you! I will include this in the book. How should I credit you?
      - divs1210 6y ago
        I am Divyansh Prakash - github.com/divs1210 The credit goes to the author of that library, though. :) All the best for your book!
kqr 6y ago
Something else that sets it apart is that the decision on how to handle something is decoupled from the implementation of that thing. With traditional exceptions, the stack unwind all the way to the "catch" block which has full responsibility of handling the exception. With CL conditions, you can implement multiple handlers which do not catch the exception, and then a handler way up in the stack can decide which of the low-level handlers to run. It sounds bonkers, but when you've tried it you'll see that "normal" exceptions are a very crippled version of it.
- phoe-krk 6y ago
  > With traditional exceptions, the stack unwind all the way to the "catch" block which has full responsibility of handling the exception. With CL conditions, you can implement multiple handlers which do not catch the exception, and then a handler way up in the stack can decide which of the low-level handlers to run. I'd say that this is the main difference that sets the CL condition system apart from mainstream programming languages. In these other languages, when an error happens, the stack is automatically unwound; in Lisp, when an error happens, the stack is automatically wound further, and the code executed in such way has the full choice of whether to unwind the stack and where exactly to unwind it to; this information is provided in the dynamic environment in which a given piece of code is running.
  - Akronymus 6y ago
    Is this a consequence of the "mythical" treating code as data that lisp is often praised for?
    - phoe-krk 6y ago
      Not at all. It is a consequence of language design of other languages. When you divide by zero in Java, then there is nothing set in stone that prevents the language from winding the stack further and executing some code that will analyze the dynamic environment in which the error happened, calling all error handlers found in the dynamic environment, and only then - if no handler transferred control outside the error side - either entering an interactive debugger of some sorts or giving up and crashing. But instead Java goes another way and immediately destroys the stack by throwing an exception. That's a design choice made by Java creators. Lisp's homoiconicity has nothing to do with this. You can also analyze the Python implementation of the condition system posted elsewhere in the comments - that's an example of a condition system in a non-homoiconic language.
jwr 6y ago
Expanding on that: the most impressive feature of CL's condition handling is that it doesn't unwind the stack for you automatically. In languages with "exceptions", when an exception gets thrown, you have a chance to catch it, but at that point the stack has already been unwound and there is no way to resume execution at the point where exception was thrown. In CL, you can let someone else (the caller) make the decision on what to do in case of an exception, providing options. For example, if a value is a NaN, the caller could request to continue the computation using the caller-supplied value instead. In practical terms, this system, while very impressive, it isn't widely understood or used in CL. It's one of the very few things I miss after I moved from CL to Clojure. Rich Hickey wrote that he did consider it, but it was a lot of complexity for fairly little gain (all platforms where Clojure is hosted support thrown exceptions only), which I would agree with.
- phoe-krk 6y ago
  > In practical terms, this system, while very impressive, it isn't widely understood or used in CL. I agree and wholeheartedly hope that my book mends that issue, even if just partially.
dmux 6y ago
How would this work for something like an http server? Would a single response block until the issue was fixed?
- phoe-krk 6y ago
  If the server decides to escalate the issue to the programmer, e.g. by invoking the debugger, then yes, it would most likely hang until the programmer interactively resolves the problem (e.g. by issuing a proper restart), and only then execution would proceed. However, that doesn't need to be the case; it is still possible to e.g. do the industry standard, which is to log all available information about the request and program state to some sort of log file or crash dumps, return a HTTP 500, and start waiting for another request. The possibility of interactive resolution of exceptional situations is a possibility, not a requirement. It's also possible to block and invoke the debugger only in some cases, e.g. when some global debug variable is set to true. In all other cases, the system can behave automatically.
  - dmux 6y ago
    Thanks for the reply. >log all available information about the request and program state to some sort of log file I've only dabbled with some Schemes, but given Lisp's homoiconic nature, I'm guessing it would also be possible to generate and persist a unit test based on the failing call too, right?
    - phoe-krk 6y ago
      I think that'll be possible. As long as you can persist all the state (which is easy if you are programming in a functional manner), then you will know exactly what sort of input caused a crash in your program. And when you have the data, you can write it in a manner that is later readable by the Lisp reaer.
hardeeparty 6y ago
This seems really valuable for development, but is there a way to use this in production systems? Say a user hits a bug in a CL web app. Could I write an exception handler that (1) immediately returns a 500 error page and (2) persists an image somewhere so that I could fire it up later and have the debugger ready at the offending point?
- phoe-krk 6y ago
  I think that it's possible. Immediately returning a HTTP 500 is easily doable, and you should be able to fork the Lisp process, continue serving requests in the parent, and save stack information (such as function parameters, etc..) and dump the Lisp core for further analysis in the child. (Do note, however, that this process breaks all FDs, which includes open files and network sockets.)
  - zeveb 6y ago
    Depending on how the HTTP library works, one might be able to return an error to the user, but preserve all the other call state in an open thread, and later on join it from an interactive debugger … perhaps even a Lisp debugger running in a Web browser!
    - phoe-krk 6y ago
      That's doable. Common Lisp has a programmable debugger hook which allows for implementing one's own debugger - also, possibly, as a web application. And for situations where the standard debugger hook is not enough (e.g. BREAK or INVOKE-DEBUGGER), one can use https://github.com/phoe/trivial-custom-debugger https://github.com/phoe/trivial-custom-debugger to completely override the system debugger with a custom one.
- p_l 6y ago
  Yes, that's actually possible. Biggest issue might be forking to dump the image and dealing with OS. It also means you don't have to write blinks of try catch nested in while loop just to be able to retry the operation...
plafl 6y ago
Thank you for the explanation. I cannot count how many times I have wished for something similar in python.
- phoe-krk 6y ago
  Christmas comes early for you this year. :D See https://github.com/svetlyak40wt/python-cl-conditions https://github.com/svetlyak40wt/python-cl-conditions

by drcode 6y ago

tester34 6y ago

During debugging, yea?

phoe-krk 6y ago
Yes, all that without leaving the debugger. The debugger is left only when we want to continue execution of the program.
- protomyth 6y ago
  I've noticed quite a few Smalltalk programmers who actually coded in the debugger. They would put a call in the program, hit it as an error, and then write new code. Does this happen with Common Lisp programmers?
  - phoe-krk 6y ago
    Smalltalk and Common Lisp are both image-based programming languages where the compiler and debugger are always available. I think debugger-oriented programming is much more common in Smalltalk than in Lisp (at least I prefer to leave the debugger back to REPL pretty often), but I think it's certainly possible to adopt. The Slime/Sly environments for emacs might accept a few patches to facilitate that programming style better.
  - p_l 6y ago
    I know of several lispers that swear by top-down "programming inside debugger", where they continuously fill in the gaps starting from top level function :-D
    - phoe-krk 6y ago
      Do you have any videos of such programming style performed in Lisp, or could you try to convince some of those programmers to record some of their programming sessions?
      - rags2riches 6y ago
        You could look up Baggers on Youtube for plenty of fun Lisp coding sessions.
        phoe-krk 6y ago
        I know Baggers and his videos but I wasn't aware that he does debugger-oriented programming. Thanks, I'll pay closer attention to them!
aidenn0 6y ago
For this example, it would be during debugging. You could intentionally write a bug, and then write a condition handler that would fix the bug without unwinding the stack further than necessary, but that would be rather pointless. However think of all the things that can go wrong with your environment that don't involve a bug in your program. A network connection goes down, the disk gets full, &c. All of those can be handled without unwinding the stack to the point where the code for handling them was defined. For a really trivial example, consider a performance watchdog. If you want to completely abort the attempt and log a call-stack, you can do that it just about any modern, dynamic language because the exception object includes the call-stack. But what if you want to log the call-stack and continue for the first N times the watchdog rolls-over? This is, of course, doable in other languages with passing callbacks and such, but in Lisp this can be done by signalling a condition in the exact same manner that an exceptional condition would be signaled. I'm excited about this book, because that particular feature is underutilized in Lisp (by myself included), probably because it's one of the last dynamic things that Lisp does that more well-known dynamic languages do not, so programmers coming from other languages just aren't going to use it.
- phoe-krk 6y ago
  > All of those can be handled without unwinding the stack to the point where the code for handling them was defined. Nitpick: the only thing that cannot be handled by winding the stack is when you run out of stack to wind. But, luckily, if your program overflows the stack, you usually have bigger issues to deal with and/or you usually have all the stack information you need to diagnose the problem.

phoe-krk 6y ago

Additionally, one of the most tasty facts about it is that the Common Lisp condition system is written in Common Lisp itself. It's literally a Lisp plugin that only needs to be integrated with the rest of the system at its end points: where the conditions are signaled and where the debugger is entered.

lukashrb 6y ago
Does anyone know if it is possible to integrate such a system, as plugin, in clojure? Is there already a project like that?
- phoe-krk 6y ago
  I haven't explored Clojure. If it allows one to customize the means in which errors are signaled (read: force the language to not immediately throw a Java exception on an error, and instead execute some code), then one should be able to implement both signaling and a debugger. It might not be easy to tell Clojure or the JVM to not be trigger happy with throwing exceptions and therefore destroying the stack. Perhaps one will find a way through!
- divs1210 6y ago
  Yes, there are multiple libraries, for example: https://github.com/clojureman/special https://github.com/clojureman/special
  - phoe-krk 6y ago
    Thank you! I will include this in the book. How should I credit you?
    - divs1210 6y ago
      I am Divyansh Prakash - github.com/divs1210 The credit goes to the author of that library, though. :) All the best for your book!

kqr 6y ago

Something else that sets it apart is that the decision on how to handle something is decoupled from the implementation of that thing. With traditional exceptions, the stack unwind all the way to the "catch" block which has full responsibility of handling the exception. With CL conditions, you can implement multiple handlers which do not catch the exception, and then a handler way up in the stack can decide which of the low-level handlers to run. It sounds bonkers, but when you've tried it you'll see that "normal" exceptions are a very crippled version of it.

phoe-krk 6y ago
> With traditional exceptions, the stack unwind all the way to the "catch" block which has full responsibility of handling the exception. With CL conditions, you can implement multiple handlers which do not catch the exception, and then a handler way up in the stack can decide which of the low-level handlers to run. I'd say that this is the main difference that sets the CL condition system apart from mainstream programming languages. In these other languages, when an error happens, the stack is automatically unwound; in Lisp, when an error happens, the stack is automatically wound further, and the code executed in such way has the full choice of whether to unwind the stack and where exactly to unwind it to; this information is provided in the dynamic environment in which a given piece of code is running.
- Akronymus 6y ago
  Is this a consequence of the "mythical" treating code as data that lisp is often praised for?
  - phoe-krk 6y ago
    Not at all. It is a consequence of language design of other languages. When you divide by zero in Java, then there is nothing set in stone that prevents the language from winding the stack further and executing some code that will analyze the dynamic environment in which the error happened, calling all error handlers found in the dynamic environment, and only then - if no handler transferred control outside the error side - either entering an interactive debugger of some sorts or giving up and crashing. But instead Java goes another way and immediately destroys the stack by throwing an exception. That's a design choice made by Java creators. Lisp's homoiconicity has nothing to do with this. You can also analyze the Python implementation of the condition system posted elsewhere in the comments - that's an example of a condition system in a non-homoiconic language.

jwr 6y ago

Expanding on that: the most impressive feature of CL's condition handling is that it doesn't unwind the stack for you automatically. In languages with "exceptions", when an exception gets thrown, you have a chance to catch it, but at that point the stack has already been unwound and there is no way to resume execution at the point where exception was thrown. In CL, you can let someone else (the caller) make the decision on what to do in case of an exception, providing options. For example, if a value is a NaN, the caller could request to continue the computation using the caller-supplied value instead. In practical terms, this system, while very impressive, it isn't widely understood or used in CL. It's one of the very few things I miss after I moved from CL to Clojure. Rich Hickey wrote that he did consider it, but it was a lot of complexity for fairly little gain (all platforms where Clojure is hosted support thrown exceptions only), which I would agree with.

phoe-krk 6y ago
> In practical terms, this system, while very impressive, it isn't widely understood or used in CL. I agree and wholeheartedly hope that my book mends that issue, even if just partially.

dmux 6y ago

How would this work for something like an http server? Would a single response block until the issue was fixed?

phoe-krk 6y ago
If the server decides to escalate the issue to the programmer, e.g. by invoking the debugger, then yes, it would most likely hang until the programmer interactively resolves the problem (e.g. by issuing a proper restart), and only then execution would proceed. However, that doesn't need to be the case; it is still possible to e.g. do the industry standard, which is to log all available information about the request and program state to some sort of log file or crash dumps, return a HTTP 500, and start waiting for another request. The possibility of interactive resolution of exceptional situations is a possibility, not a requirement. It's also possible to block and invoke the debugger only in some cases, e.g. when some global debug variable is set to true. In all other cases, the system can behave automatically.
- dmux 6y ago
  Thanks for the reply. >log all available information about the request and program state to some sort of log file I've only dabbled with some Schemes, but given Lisp's homoiconic nature, I'm guessing it would also be possible to generate and persist a unit test based on the failing call too, right?
  - phoe-krk 6y ago
    I think that'll be possible. As long as you can persist all the state (which is easy if you are programming in a functional manner), then you will know exactly what sort of input caused a crash in your program. And when you have the data, you can write it in a manner that is later readable by the Lisp reaer.

hardeeparty 6y ago

This seems really valuable for development, but is there a way to use this in production systems? Say a user hits a bug in a CL web app. Could I write an exception handler that (1) immediately returns a 500 error page and (2) persists an image somewhere so that I could fire it up later and have the debugger ready at the offending point?

phoe-krk 6y ago
I think that it's possible. Immediately returning a HTTP 500 is easily doable, and you should be able to fork the Lisp process, continue serving requests in the parent, and save stack information (such as function parameters, etc..) and dump the Lisp core for further analysis in the child. (Do note, however, that this process breaks all FDs, which includes open files and network sockets.)
- zeveb 6y ago
  Depending on how the HTTP library works, one might be able to return an error to the user, but preserve all the other call state in an open thread, and later on join it from an interactive debugger … perhaps even a Lisp debugger running in a Web browser!
  - phoe-krk 6y ago
    That's doable. Common Lisp has a programmable debugger hook which allows for implementing one's own debugger - also, possibly, as a web application. And for situations where the standard debugger hook is not enough (e.g. BREAK or INVOKE-DEBUGGER), one can use https://github.com/phoe/trivial-custom-debugger https://github.com/phoe/trivial-custom-debugger to completely override the system debugger with a custom one.
p_l 6y ago
Yes, that's actually possible. Biggest issue might be forking to dump the image and dealing with OS. It also means you don't have to write blinks of try catch nested in while loop just to be able to retry the operation...

plafl 6y ago

Thank you for the explanation. I cannot count how many times I have wished for something similar in python.

phoe-krk 6y ago
Christmas comes early for you this year. :D See https://github.com/svetlyak40wt/python-cl-conditions https://github.com/svetlyak40wt/python-cl-conditions