4 ms·

Doesn't say anything about cycles or what datatypes can be serialised. In general lisps _can't_ print arbitrary objects in a form that can be read back in. E.g.

Doesn't say anything about cycles or what datatypes can be serialised. In general lisps _can't_ print arbitrary objects in a form that can be read back in. E.g. Welcome to Racket v7.9 [bc]. > (lambda (x) x) #<procedure> 7.9 doesn't have csexp, was curious what it would do with a function object but not quite curious enough to install a racket from outside the package manager. It seems relatively likely that racket would have some serialisation feature that can round trip arbitrary objects through a bytestring though initial guesses haven't worked out for me > (require racket/serialize) > (serializable? (lambda (x) x)) #f Still, being able to take some arbitrary closure and kick it over the network to another racket instance would be great. It's not immediately obvious that csexp can do that.

nerdponx 3y ago
Common Lisp has a reader macro for defining self-referential data structures: https://stackoverflow.com/a/605729 https://stackoverflow.com/a/605729 Maybe it wouldn't be so hard to extend this format to include a notation like that.
qawwads 3y ago
>Still, being able to take some arbitrary closure and kick it over the network to another racket instance would be great. It's not immediately obvious that csexp can do that. I'm stupid but once your list is reveived by the remote host, isn't executing it only an (eval) away?
- jdkardia 3y ago
  I'm no lisper, but I'm pretty sure it'd take more than an eval because of the context a closure carries with it. if you were sending a pure, anonymous function, that would work fine. But if you're making a closure that makes use of any surrounding variables, libraries, or state, that's a non trivial set of information to transmit across the wire.
  - alexisread 3y ago
    I think this is where Delimited Dynamic binding would come in to play: https://okmij.org/ftp/papers/DDBinding.pdf https://okmij.org/ftp/papers/DDBinding.pdf I think that you would be able to freeze certain variables in the closure, and have others open eg. API_ENDPOINT variable might be configured to change between environments. Something like https://github.com/GiacomoCau/wat-js https://github.com/GiacomoCau/wat-js implements delimited continuations, ddbinding and algebraic effects (registered handlers for I/O etc) - pushing continuations over the network should be easier with these facilities.
- EddieJLSH 3y ago
  Yeah that was my understanding, I've considered using this before (sending objects over the wire and eval'ing them) but never implemented it. Lisp seems very good for this sort of stuff.
convolvatron 3y ago
I used to do this on lispms...just `(lambda (x) ... ,z ...) assuming z is small what you would really do is make a lookup table and keep the closure locally and send a reference and forth. idk why we don't see this more, its a pretty straightforward way to lash together some programs
correnos 3y ago
The docs could do a better job of spelling it out, but csexp's really do only represent "trees of bytestrings". They're not a particularly rich format. If you're looking for "transfer this lambda over the network" semantics that's obviously a lot more complicated, but there's at least some research-project Schemes that've done it. The one I know of is Termite Scheme, which builds an Erlang-style distributed process model.
- nerdponx 3y ago
  If that is indeed the case, then is it any better than an existing format like MessagePack?
  - JonChesterfield 3y ago
    I expect going digging would reveal that csexp is decades older than msgpack. As a format I agree it doesn't look great though. May 1997 for csexp. https://web.archive.org/web/20230219084819/http://people.csail.mit.edu/rivest/Sexp.txt https://web.archive.org/web/20230219084819/http://people.csa... Msgpack less clear, maybe 2008. JSON is 2000ish.
aportnoy 3y ago
> In general lisps _can't_ print arbitrary objects in a form that can be read back in. Hmm, why not?
- JonChesterfield 3y ago
  I haven't worked on any of the classic lisp implementations but can speculate. I suppose the ratio of usefulness to difficulty of implementation doesn't work out. Mutable state with serialise/deserialise either breaks the aliasing or needs some way to keep the referenced variables alive (nasty interaction with garbage collection) and to re-establish aliasing on deserialise. Serialising a DAG to a tree tends to duplicate the previously shared nodes. There's then the question of whether deserialising the tree should re-establish the original sharing or some other sharing. Serialising a graph to a (finite) tree needs some notation to represent the cycles. It looks to me like mutable state is the blocker but I'd be interested in other opinions.
  - aportnoy 3y ago
    Yeah, something like "a" in the below code wouldn't have a "natural" serialization I don't think: (define a (list "a")) (set-cdr! a a)
- cstrahan 3y ago
  How would one print a function with a free variable? Consider this (but imagine there is surrounding code which defines y): (lambda () y) How can we print this in such a way that preserves semantics? You can’t just print the original source definition, as y would then be undefined (or I suppose it could be defined to be whatever y is in scope at the deserialization call site or something thereabouts, but that wouldn’t guarantee that the deserialized function is equivalent to the other with respect to how it behaves, despite the source being identical). You can’t just inline the serialization of y, because the original function returns a particular reference, and there’s no guarantee that the rest of the dependent code will not expect a stable, unique (and possibly mutable/state-full) instance of this value. I suppose you could, on the side, also serialize the entire program state (or rather, at least the stack and heap), but then things get tricky when any object refers to something like an opaque pointer (even if you have an extent you can safely copy, you’d have to handle issues like different virtual addresses being mapped) or file descriptors, etc. But maybe all those challenges can be overcome — so I’ll ask: how would you go about serializing an arbitrary object in such a way that one could deserialize an equivalent object?
zelphirkalt 3y ago
The Racket webserver had serializable-lambda, but iirc it requires you to also make everything you use in it serializable (?). So it is kind of infactious like async in JS. You need to write your code with it in mind. Racketeers please correct me with an example of ad-hoc making a lambda serializable, if I am wrong about this.

nerdponx 3y ago

Common Lisp has a reader macro for defining self-referential data structures: https://stackoverflow.com/a/605729 https://stackoverflow.com/a/605729 Maybe it wouldn't be so hard to extend this format to include a notation like that.

qawwads 3y ago

>Still, being able to take some arbitrary closure and kick it over the network to another racket instance would be great. It's not immediately obvious that csexp can do that. I'm stupid but once your list is reveived by the remote host, isn't executing it only an (eval) away?

jdkardia 3y ago
I'm no lisper, but I'm pretty sure it'd take more than an eval because of the context a closure carries with it. if you were sending a pure, anonymous function, that would work fine. But if you're making a closure that makes use of any surrounding variables, libraries, or state, that's a non trivial set of information to transmit across the wire.
- alexisread 3y ago
  I think this is where Delimited Dynamic binding would come in to play: https://okmij.org/ftp/papers/DDBinding.pdf https://okmij.org/ftp/papers/DDBinding.pdf I think that you would be able to freeze certain variables in the closure, and have others open eg. API_ENDPOINT variable might be configured to change between environments. Something like https://github.com/GiacomoCau/wat-js https://github.com/GiacomoCau/wat-js implements delimited continuations, ddbinding and algebraic effects (registered handlers for I/O etc) - pushing continuations over the network should be easier with these facilities.
EddieJLSH 3y ago
Yeah that was my understanding, I've considered using this before (sending objects over the wire and eval'ing them) but never implemented it. Lisp seems very good for this sort of stuff.

convolvatron 3y ago

I used to do this on lispms...just `(lambda (x) ... ,z ...) assuming z is small what you would really do is make a lookup table and keep the closure locally and send a reference and forth. idk why we don't see this more, its a pretty straightforward way to lash together some programs

correnos 3y ago

The docs could do a better job of spelling it out, but csexp's really do only represent "trees of bytestrings". They're not a particularly rich format. If you're looking for "transfer this lambda over the network" semantics that's obviously a lot more complicated, but there's at least some research-project Schemes that've done it. The one I know of is Termite Scheme, which builds an Erlang-style distributed process model.

nerdponx 3y ago
If that is indeed the case, then is it any better than an existing format like MessagePack?
- JonChesterfield 3y ago
  I expect going digging would reveal that csexp is decades older than msgpack. As a format I agree it doesn't look great though. May 1997 for csexp. https://web.archive.org/web/20230219084819/http://people.csail.mit.edu/rivest/Sexp.txt https://web.archive.org/web/20230219084819/http://people.csa... Msgpack less clear, maybe 2008. JSON is 2000ish.

aportnoy 3y ago

> In general lisps _can't_ print arbitrary objects in a form that can be read back in. Hmm, why not?

JonChesterfield 3y ago
I haven't worked on any of the classic lisp implementations but can speculate. I suppose the ratio of usefulness to difficulty of implementation doesn't work out. Mutable state with serialise/deserialise either breaks the aliasing or needs some way to keep the referenced variables alive (nasty interaction with garbage collection) and to re-establish aliasing on deserialise. Serialising a DAG to a tree tends to duplicate the previously shared nodes. There's then the question of whether deserialising the tree should re-establish the original sharing or some other sharing. Serialising a graph to a (finite) tree needs some notation to represent the cycles. It looks to me like mutable state is the blocker but I'd be interested in other opinions.
- aportnoy 3y ago
  Yeah, something like "a" in the below code wouldn't have a "natural" serialization I don't think: (define a (list "a")) (set-cdr! a a)
cstrahan 3y ago
How would one print a function with a free variable? Consider this (but imagine there is surrounding code which defines y): (lambda () y) How can we print this in such a way that preserves semantics? You can’t just print the original source definition, as y would then be undefined (or I suppose it could be defined to be whatever y is in scope at the deserialization call site or something thereabouts, but that wouldn’t guarantee that the deserialized function is equivalent to the other with respect to how it behaves, despite the source being identical). You can’t just inline the serialization of y, because the original function returns a particular reference, and there’s no guarantee that the rest of the dependent code will not expect a stable, unique (and possibly mutable/state-full) instance of this value. I suppose you could, on the side, also serialize the entire program state (or rather, at least the stack and heap), but then things get tricky when any object refers to something like an opaque pointer (even if you have an extent you can safely copy, you’d have to handle issues like different virtual addresses being mapped) or file descriptors, etc. But maybe all those challenges can be overcome — so I’ll ask: how would you go about serializing an arbitrary object in such a way that one could deserialize an equivalent object?

zelphirkalt 3y ago

The Racket webserver had serializable-lambda, but iirc it requires you to also make everything you use in it serializable (?). So it is kind of infactious like async in JS. You need to write your code with it in mind. Racketeers please correct me with an example of ad-hoc making a lambda serializable, if I am wrong about this.