4 ms·

I've read the same, almost word-for-word claims repeatedly, which is annoying given how misleading it is. Concretely... To know the context of the conditions,

I've read the same, almost word-for-word claims repeatedly, which is annoying given how misleading it is. Concretely... To know the context of the conditions, the conditions must give the information. Otherwise, the handler would need to know intimately the implementation details to be able to retrieve the filename, line number, etc. If you can provide the information to the condition you call fill a throw exception with the exact same data. The exception can carry the filename, line numbers, token being parsed... Second, the example itself is ludicrous. The caller of a file parser providing replacement data for a malformed file? In what world does that ever happens? How could it handle every possible ways a file might be malformed? Third, in every language, the same can be implemented with a callback. In C++, the standard is now to use std::function for this, which supported free functions, members, lambdas... pretty much everything. The only advantage of List is that the declaration and registration of the callback is a language feature.

thequux 3y ago
Yes, you can do this in ways other than conditions. However, I disagree with basically every objection you raise. First, the decision made by the handler case doesn't necessarily care which file the error was in, what the line number is, etc. All I've ever needed to decide what to do (details below) was the text content of the malformed record. From the perspective of my program, there were only a few possible cases: 1. The record is malformed in a way that I know how to recover from. In that case, I can just do so and invoke the `use-instead` recovery path. 2. The record is damaged in a new and exciting way. I can log it and try to muddle on, in hopes of catching all the new error cases while I'm off doing more interesting things than waiting on a 6-hour job. 3. The record is damaged irrecoverably and future records depend on it. (e.g., the file structure itself is damaged and this can't be recovered from). This is the rarest case I've come across, but also the only one that's convenient to handle with exceptions. Further, if it was just a filename and byte offset that was needed to resume where I left off, you may have a point. However, suppose that there was an additional decompression step involved. You can't, with most decompression libraries, pick up decompression in the middle of a stream, at least not without littering knowledge of the decompression through the entire process. Further, bundling everything necessary to pick up the computation where it left off forces you to structure your code in a certain way. For example, packing the state of the computation into a class with member functions doing each part, so that the file, current list of results, etc, are essentially scoped globals. I estimate that this would have been at least 5x more code than what was essentially wrapping a stream with a couple of transformers and iterating over it. To your third point, I could have structured the parser to call a callback with the details of the problem, which could throw an appropriate exception to unwind to a suitable recovery point. This is, after all, how conditions are implemented. However, conditions as part of the language mean that every error can have recovery paths registered, not just ones that the developers thought to provide callbacks for. And finally, to your second point. While I originally stole the example from Practical Common Lisp, I've since had exactly this situation come up. I had a ~150GiB file containing, essentially, lines of JSON. My parser validated that the incoming JSON fit a schema and processed it into a more compressed form such that I could fit the aspects of the dataset that I actually cared about into RAM. Now, this dataset had been through several migrations, between a number of different platforms, and not all of the migrations were bug-free. In some cases, it treated UTF-8 as CP-1251 and transcoded that into UTF-8. Others got double-escaped. Still others had parts of some fields duplicated in ways that were easy to detect and undo. Some records were just duplicated outright, and some were different versions of the same record. All of these were recoverable, but it was 150GiB of data. I couldn't manually clean it first; I needed to run the program to see what it barfed on in order to fix it. Worse, being JSON, it compressed easily and this was at a time when 150GiB was more than half the disk space I had available to me. So of course the dataset was compressed on disk, and I was decompressing it as I read it. Now, I'm sure that you can come up with a way that I could have packed the error recovery into the callback in the inner loop of the iterator, but why would I have? I had conditions at my disposal, and the way I actually did write it, I had the happy path in a perfectly clear straight line, and all of the various error cases and how to handle them lined up in a row next to it. The code was easy to read and work with, without any real efficiency cost. The fact that I could have made do with callbacks is no more relevant than that I could have made do using goto instead of loops and functions: we have these abstractions so that we can express what we want our programs to do at a higher level.
- squirtlebonflow 3y ago
  What do you people DO that you are doing tasks like this? I've always loved FP from college, but never ran into problems like this that would actually warrant using one
  - mikelevins 3y ago
    In general, building programs interactively by changing them as they run. That's the default, normal way to write programs in certain languages (notably in Common Lisp and its antecedents and in Smalltalk). In such languages, you want to be able to use the entire language at any arbitrary point, including at a point where execution has halted for the moment because of some error that has occurred in some arbitrary place. The condition system provides a nice set of tools for doing that.
  - thequux 3y ago
    In a small sense, doing yet another data migration in a long line of data migrations. The data I had was a collection of proprietary test results about electronic parts, half hand-entered and half generated by various tools some of which dated back to the 60's. (I strongly suspect that the dataset started out as a stack of punched cards used with an IBM System/360). In other words, about as boring big-business as boring big-business stuff gets. Nothing about the task really required using Lisp, but it it was the language I pulled out of a hat to start playing around with the problem and I got far enough within the first few days of experimentation to promote my prototype to the actual solution. In a larger sense, and probably more relevant to your question, it's not about any particular application space where FP is "warranted", but rather about being familiar enough with enough different languages and architectures that I can look at a problem and see a variety of ways to solve it. Second, I always build a prototype in a "weird" language that I don't intend to put into production, because the prototype is more there to understand the problem than to come up with a production-ready solution. Weird languages add friction to deciding to productize the prototype, and therefore encourage me to really consider what tech stack is appropriate for the actual product. Finally, it's probably worth noting that I rarely work on anything particularly interactive. If you're not touching GUIs or web stuff, you'll often find that you're a lot less constrained on your tech choices, because you don't actually need all that much from libraries.
jjtheblunt 3y ago
> providing replacement data for a malformed file? In what world does that ever happens? that sort of pattern happens in coding theory, error correcting codes for example. > In C++ i don't claim to know the answer, but does it matter that the free variable allocation strategy for a Lisp lambda differs from what lambda means in C++?

thequux 3y ago

Yes, you can do this in ways other than conditions. However, I disagree with basically every objection you raise. First, the decision made by the handler case doesn't necessarily care which file the error was in, what the line number is, etc. All I've ever needed to decide what to do (details below) was the text content of the malformed record. From the perspective of my program, there were only a few possible cases: 1. The record is malformed in a way that I know how to recover from. In that case, I can just do so and invoke the `use-instead` recovery path. 2. The record is damaged in a new and exciting way. I can log it and try to muddle on, in hopes of catching all the new error cases while I'm off doing more interesting things than waiting on a 6-hour job. 3. The record is damaged irrecoverably and future records depend on it. (e.g., the file structure itself is damaged and this can't be recovered from). This is the rarest case I've come across, but also the only one that's convenient to handle with exceptions. Further, if it was just a filename and byte offset that was needed to resume where I left off, you may have a point. However, suppose that there was an additional decompression step involved. You can't, with most decompression libraries, pick up decompression in the middle of a stream, at least not without littering knowledge of the decompression through the entire process. Further, bundling everything necessary to pick up the computation where it left off forces you to structure your code in a certain way. For example, packing the state of the computation into a class with member functions doing each part, so that the file, current list of results, etc, are essentially scoped globals. I estimate that this would have been at least 5x more code than what was essentially wrapping a stream with a couple of transformers and iterating over it. To your third point, I could have structured the parser to call a callback with the details of the problem, which could throw an appropriate exception to unwind to a suitable recovery point. This is, after all, how conditions are implemented. However, conditions as part of the language mean that every error can have recovery paths registered, not just ones that the developers thought to provide callbacks for. And finally, to your second point. While I originally stole the example from Practical Common Lisp, I've since had exactly this situation come up. I had a ~150GiB file containing, essentially, lines of JSON. My parser validated that the incoming JSON fit a schema and processed it into a more compressed form such that I could fit the aspects of the dataset that I actually cared about into RAM. Now, this dataset had been through several migrations, between a number of different platforms, and not all of the migrations were bug-free. In some cases, it treated UTF-8 as CP-1251 and transcoded that into UTF-8. Others got double-escaped. Still others had parts of some fields duplicated in ways that were easy to detect and undo. Some records were just duplicated outright, and some were different versions of the same record. All of these were recoverable, but it was 150GiB of data. I couldn't manually clean it first; I needed to run the program to see what it barfed on in order to fix it. Worse, being JSON, it compressed easily and this was at a time when 150GiB was more than half the disk space I had available to me. So of course the dataset was compressed on disk, and I was decompressing it as I read it. Now, I'm sure that you can come up with a way that I could have packed the error recovery into the callback in the inner loop of the iterator, but why would I have? I had conditions at my disposal, and the way I actually did write it, I had the happy path in a perfectly clear straight line, and all of the various error cases and how to handle them lined up in a row next to it. The code was easy to read and work with, without any real efficiency cost. The fact that I could have made do with callbacks is no more relevant than that I could have made do using goto instead of loops and functions: we have these abstractions so that we can express what we want our programs to do at a higher level.

squirtlebonflow 3y ago
What do you people DO that you are doing tasks like this? I've always loved FP from college, but never ran into problems like this that would actually warrant using one
- mikelevins 3y ago
  In general, building programs interactively by changing them as they run. That's the default, normal way to write programs in certain languages (notably in Common Lisp and its antecedents and in Smalltalk). In such languages, you want to be able to use the entire language at any arbitrary point, including at a point where execution has halted for the moment because of some error that has occurred in some arbitrary place. The condition system provides a nice set of tools for doing that.
- thequux 3y ago
  In a small sense, doing yet another data migration in a long line of data migrations. The data I had was a collection of proprietary test results about electronic parts, half hand-entered and half generated by various tools some of which dated back to the 60's. (I strongly suspect that the dataset started out as a stack of punched cards used with an IBM System/360). In other words, about as boring big-business as boring big-business stuff gets. Nothing about the task really required using Lisp, but it it was the language I pulled out of a hat to start playing around with the problem and I got far enough within the first few days of experimentation to promote my prototype to the actual solution. In a larger sense, and probably more relevant to your question, it's not about any particular application space where FP is "warranted", but rather about being familiar enough with enough different languages and architectures that I can look at a problem and see a variety of ways to solve it. Second, I always build a prototype in a "weird" language that I don't intend to put into production, because the prototype is more there to understand the problem than to come up with a production-ready solution. Weird languages add friction to deciding to productize the prototype, and therefore encourage me to really consider what tech stack is appropriate for the actual product. Finally, it's probably worth noting that I rarely work on anything particularly interactive. If you're not touching GUIs or web stuff, you'll often find that you're a lot less constrained on your tech choices, because you don't actually need all that much from libraries.

jjtheblunt 3y ago

> providing replacement data for a malformed file? In what world does that ever happens? that sort of pattern happens in coding theory, error correcting codes for example. > In C++ i don't claim to know the answer, but does it matter that the free variable allocation strategy for a Lisp lambda differs from what lambda means in C++?