4 ms·

Well this is certainly the coolest thing I've read today. I've been trying to grok closures and this helps a bit. Is there a reason to not use the global hash t

Well this is certainly the coolest thing I've read today. I've been trying to grok closures and this helps a bit. Is there a reason to not use the global hash table itself to store the state instead of a closure? This seems to be trading a database with memory. This also seems to be harder to interrogate, what if I want to go in and see what's currently outstanding, instead of going to Firebase, I'll need to go through the hash table and check the content of the function. I think I may be missing something someone who's actually worked with lisp can see, to me closure, recursion and functional programming is cool but I can do everything its showing off using the standard fare of loops and databases.

lstamour 6y ago
The biggest difference is the “...” assignment to fn. The idea is similar to AWS Lambda — write functions, store those functions, and then call them later when you need them. I’ve minimal Lisp experience, but from my perspective, a closure is a function you can store in a variable that’s defined with a scope, or a set of arguments/variables used in your function, that often (but not always) includes variables from the parent scope it was defined in, (often only) if referenced by the function. Because closures need to have independent scopes, by default values (or variables) that can mutate need to be copied — alternatively, you can use immutable data structures which copy more cheaply. The big differences then between closures and other types of code often comes down to how frequently immutability is used, and whether you call functions that assume state or share state (more OO, or non-FP), or transfer functions with state to other functions (FP, though composability and other properties matter too when defining FP, this is a simplification). This is a bit of a vague answer, perhaps others can chime in with a better one. And if you’re not careful, FP can introduce problems too, though that happens more often with distributed, multi-threaded or recursive programs which can themselves be hard to write using non-FP also.
- throwanem 6y ago
  It's not all that similar to AWS Lambdas in concept or in execution. Those are stateless; to a very good first approximation, they're just a single-route web server with all the boilerplate abstracted away, and that starts up a fresh instance to handle each request and is shut down again immediately after. What 'sillysaurusx describes is much more similar to what, in Scheme and elsewhere but these days mainly there, is called "continuation-passing style". It's a way of pausing a partially completed computation indefinitely by wrapping it up in a function that closes over the state of the computation when you create it, and calling that function later to pick up from where you left off when you're ready to proceed again. I suppose you could maybe do that with an AWS Lambda, but because the technique relies strongly on the runtime instance staying around until the computation finishes, it would probably get expensive. Lambdas aren't priced to stay running, after all. As a side note, it's worth mentioning that the "AWS Lambda" product, which whatever its virtues isn't actually a lambda, derives its name from the lambda calculus, where I believe the concept of anonymous first-class functions originates. I don't recommend reading about the lambda calculus itself unless you're up for a lot of very heavy theory, but it's worth knowing that, especially in the Lisp world and realms adjacent, you'll often see the term 'lambda' used in a sense which has nothing to do with the AWS product, but rather refers to a form of abstraction that relies on defining functions which retain access to the variable ("lexical") scopes in which they were created, even when called from outside those scopes. Javascript functions have this property, which is why they're capable of expressing the technique 'sillysaurusx describes, and it gives them a lot of other useful capabilities as well.
  - lstamour 6y ago
    True. Good distinctions. To re-iterate the above, the approximation to AWS Lambda would require dynamic AWS Lambda functions -- as in code that creates a Lambda with specific state embedded in it -- then tracks each of those by their unique Lambda identifier and ... yeah, that's where this breaks down because it's not all that similar to Lambda if the best use for a Lambda is repeated invocations of the same code. And Lambda IDs presumably aren't based on a hash of their contents and variables the way this is. But dynamic AWS Lambda functions are possible, so there's that. You could write this in Lambda, it just might be expensive if API calls to create and destroy one-time Lambdas are expensive enough. It's a lot cheaper and faster to build functions and store references to them in a hash table in memory. Another similarity to this use of hashing the scope of a function would be in memoization of a function, to cache the output based on the input, such that you hash a function's inputs and assign to that hash a copy of the output of the function when run with those inputs. Then you can hash the inputs and skip re-running the function. You have to be sure the function has no side-effects nor any changes in behaviour or inputs not specified in the memoization hash, though. "Pure" functions are best for this use case.
    - throwanem 6y ago
      Memoization is usually preferable if you can do it, sure. But you can't memoize a continuation, because what it expresses is a computation that has yet to complete and produce the result you'd need in order to memoize. And the use of the g_fnid hash table doesn't qualify as memoization, either, because the keys aren't arguments to the function that produced the values; what it actually is is a jump table, cf. https://en.m.wikipedia.org/wiki/Branch_table#Jump_table_example_in_C https://en.m.wikipedia.org/wiki/Branch_table#Jump_table_exam...
      - lstamour 6y ago
        Thanks for your reply. I ended up looking for a bit more on continuations from the perspective of JS Promises and found https://dev.to/homam/composability-from-callbacks-to-categories-in-es6-17en https://dev.to/homam/composability-from-callbacks-to-categor... which was a pretty easy to follow read on this if you take the time to understand the JS, though there might be better references to continuations elsewhere, this was just one of the first I found.
throwanem 6y ago
It works a lot better in a proper Lisp, where the REPL and debugger are first-class citizens. In Javascript, you can do it, but it's a dancing bear at best; as you note, the observability is poor to nil without heroic effort, and scalability's a problem too.

lstamour 6y ago

The biggest difference is the “...” assignment to fn. The idea is similar to AWS Lambda — write functions, store those functions, and then call them later when you need them. I’ve minimal Lisp experience, but from my perspective, a closure is a function you can store in a variable that’s defined with a scope, or a set of arguments/variables used in your function, that often (but not always) includes variables from the parent scope it was defined in, (often only) if referenced by the function. Because closures need to have independent scopes, by default values (or variables) that can mutate need to be copied — alternatively, you can use immutable data structures which copy more cheaply. The big differences then between closures and other types of code often comes down to how frequently immutability is used, and whether you call functions that assume state or share state (more OO, or non-FP), or transfer functions with state to other functions (FP, though composability and other properties matter too when defining FP, this is a simplification). This is a bit of a vague answer, perhaps others can chime in with a better one. And if you’re not careful, FP can introduce problems too, though that happens more often with distributed, multi-threaded or recursive programs which can themselves be hard to write using non-FP also.

throwanem 6y ago
It's not all that similar to AWS Lambdas in concept or in execution. Those are stateless; to a very good first approximation, they're just a single-route web server with all the boilerplate abstracted away, and that starts up a fresh instance to handle each request and is shut down again immediately after. What 'sillysaurusx describes is much more similar to what, in Scheme and elsewhere but these days mainly there, is called "continuation-passing style". It's a way of pausing a partially completed computation indefinitely by wrapping it up in a function that closes over the state of the computation when you create it, and calling that function later to pick up from where you left off when you're ready to proceed again. I suppose you could maybe do that with an AWS Lambda, but because the technique relies strongly on the runtime instance staying around until the computation finishes, it would probably get expensive. Lambdas aren't priced to stay running, after all. As a side note, it's worth mentioning that the "AWS Lambda" product, which whatever its virtues isn't actually a lambda, derives its name from the lambda calculus, where I believe the concept of anonymous first-class functions originates. I don't recommend reading about the lambda calculus itself unless you're up for a lot of very heavy theory, but it's worth knowing that, especially in the Lisp world and realms adjacent, you'll often see the term 'lambda' used in a sense which has nothing to do with the AWS product, but rather refers to a form of abstraction that relies on defining functions which retain access to the variable ("lexical") scopes in which they were created, even when called from outside those scopes. Javascript functions have this property, which is why they're capable of expressing the technique 'sillysaurusx describes, and it gives them a lot of other useful capabilities as well.
- lstamour 6y ago
  True. Good distinctions. To re-iterate the above, the approximation to AWS Lambda would require dynamic AWS Lambda functions -- as in code that creates a Lambda with specific state embedded in it -- then tracks each of those by their unique Lambda identifier and ... yeah, that's where this breaks down because it's not all that similar to Lambda if the best use for a Lambda is repeated invocations of the same code. And Lambda IDs presumably aren't based on a hash of their contents and variables the way this is. But dynamic AWS Lambda functions are possible, so there's that. You could write this in Lambda, it just might be expensive if API calls to create and destroy one-time Lambdas are expensive enough. It's a lot cheaper and faster to build functions and store references to them in a hash table in memory. Another similarity to this use of hashing the scope of a function would be in memoization of a function, to cache the output based on the input, such that you hash a function's inputs and assign to that hash a copy of the output of the function when run with those inputs. Then you can hash the inputs and skip re-running the function. You have to be sure the function has no side-effects nor any changes in behaviour or inputs not specified in the memoization hash, though. "Pure" functions are best for this use case.
  - throwanem 6y ago
    Memoization is usually preferable if you can do it, sure. But you can't memoize a continuation, because what it expresses is a computation that has yet to complete and produce the result you'd need in order to memoize. And the use of the g_fnid hash table doesn't qualify as memoization, either, because the keys aren't arguments to the function that produced the values; what it actually is is a jump table, cf. https://en.m.wikipedia.org/wiki/Branch_table#Jump_table_example_in_C https://en.m.wikipedia.org/wiki/Branch_table#Jump_table_exam...
    - lstamour 6y ago
      Thanks for your reply. I ended up looking for a bit more on continuations from the perspective of JS Promises and found https://dev.to/homam/composability-from-callbacks-to-categories-in-es6-17en https://dev.to/homam/composability-from-callbacks-to-categor... which was a pretty easy to follow read on this if you take the time to understand the JS, though there might be better references to continuations elsewhere, this was just one of the first I found.

throwanem 6y ago

It works a lot better in a proper Lisp, where the REPL and debugger are first-class citizens. In Javascript, you can do it, but it's a dancing bear at best; as you note, the observability is poor to nil without heroic effort, and scalability's a problem too.