3 ms·

Paul Graham's startup (way back) did this and he claims it was a competitive advantage. These days it doesn't feel like this approach offers enough.

by sriku 11y ago

derefr 11y ago
Not just way back; HN uses continuations. (Not in many user-visible places any more, but every link on the front page used to be one—expired "next page" continuation links were a frequent source of grumbling.) Note that you don't have to expire continuations, necessarily—you could actually persist them statelessly (i.e. without server-side state), by shipping their encoded representations to the client embedded into HMAC-signed links. You're basically giving the server a raw bytecode-eval endpoint, and then making sure that it only accepts code you yourself wrote. Kind of a crazy strategy compared to the standard predeclared REST API, but interestingly flexible.
- dang 11y ago
  > their encoded representations What's an example of such a representation?
  - derefr 11y ago
    Now that you mention it, I can't think of very many languages that define a language-standard persistent encoding of a continuation on a closure. One that I can think of is Erlang. Here's the relevant section of the Erlang "External Term Format encoding" spec: http://erlang.org/doc/apps/erts/erl_ext_dist.html#id95269 http://erlang.org/doc/apps/erts/erl_ext_dist.html#id95269 Said spec is nominally "about" the wire protocol Erlang nodes use to communicate with one another in a distributed system. But the ETF format defined there is much more general, and is used all over the place (e.g. when persisting values in Erlang's DETS tables to disk.) It's what you get when you call Erlang's global term_to_binary/1 function. Erlang's ETF could probably be most closely compared to Python's "pickle" format. (Though, sadly, you can't pickle a lexical closure in Python.)
    - dang 11y ago
      That's interesting; I didn't know that any language did that. I really like your design idea, and it would be fun to think of an application for which it was the best fit.
      - derefr 11y ago
        Even Erlang's approach is sort of a hack. As can be seen from the format spec, Erlang doesn't actually encode the AST of the closure; instead, they ship a reference to a symbol-table ref in a module. Erlang's closures are all implicitly hoisted out into their own gensym'ed functions and then exported from their module to allow for this. Each node assumes the other nodes in the same distributed system all have access to the same modules (not that they all necessarily have it loaded, but that they can load it Just-In-Time when dereferencing the closure) and so the export-table reference can be turned back into a handle on the closure at the receiving end by looking it up in its module. What this all means is that, although Erlang nodes can rehydrate closures from other nodes that are running the same code, they can't rehydrate arbitrary closures defined and serialized at your personal REPL running different code. (Which defeats half the purpose; you can't then use dehydrated closures as an arbitrary query/callback/RPC syntax between Erlang-based microservices, unless they all share code.) On the other hand, there's nothing explicitly stopping the Erlang Run Time System from supporting the full version, where a closure is dehydrated as its AST (plus free vars, as in ETF encoding.) And so this is exactly what Elixir does. Yet another reason Elixir is an interesting language.
        dang 11y ago
        So what's encoded are the closed variables along with the gensym'ed name of the closure (effectively a pointer to the code)? If so, that's pretty useful. For some cases it's better than serializing the code, because then you don't have to pass a copy of the code around everywhere, only the state that you don't keep a copy of.

Paul Graham's startup (way back) did this and he claims it was a competitive advantage. These days it doesn't feel like this approach offers enough.

by sriku 11y ago

Paul Graham's startup (way back) did this and he claims it was a competitive advantage. These days it doesn't feel like this approach offers enough.

derefr 11y ago

Not just way back; HN uses continuations. (Not in many user-visible places any more, but every link on the front page used to be one—expired "next page" continuation links were a frequent source of grumbling.) Note that you don't have to expire continuations, necessarily—you could actually persist them statelessly (i.e. without server-side state), by shipping their encoded representations to the client embedded into HMAC-signed links. You're basically giving the server a raw bytecode-eval endpoint, and then making sure that it only accepts code you yourself wrote. Kind of a crazy strategy compared to the standard predeclared REST API, but interestingly flexible.

dang 11y ago
> their encoded representations What's an example of such a representation?
- derefr 11y ago
  Now that you mention it, I can't think of very many languages that define a language-standard persistent encoding of a continuation on a closure. One that I can think of is Erlang. Here's the relevant section of the Erlang "External Term Format encoding" spec: http://erlang.org/doc/apps/erts/erl_ext_dist.html#id95269 http://erlang.org/doc/apps/erts/erl_ext_dist.html#id95269 Said spec is nominally "about" the wire protocol Erlang nodes use to communicate with one another in a distributed system. But the ETF format defined there is much more general, and is used all over the place (e.g. when persisting values in Erlang's DETS tables to disk.) It's what you get when you call Erlang's global term_to_binary/1 function. Erlang's ETF could probably be most closely compared to Python's "pickle" format. (Though, sadly, you can't pickle a lexical closure in Python.)
  - dang 11y ago
    That's interesting; I didn't know that any language did that. I really like your design idea, and it would be fun to think of an application for which it was the best fit.
    - derefr 11y ago
      Even Erlang's approach is sort of a hack. As can be seen from the format spec, Erlang doesn't actually encode the AST of the closure; instead, they ship a reference to a symbol-table ref in a module. Erlang's closures are all implicitly hoisted out into their own gensym'ed functions and then exported from their module to allow for this. Each node assumes the other nodes in the same distributed system all have access to the same modules (not that they all necessarily have it loaded, but that they can load it Just-In-Time when dereferencing the closure) and so the export-table reference can be turned back into a handle on the closure at the receiving end by looking it up in its module. What this all means is that, although Erlang nodes can rehydrate closures from other nodes that are running the same code, they can't rehydrate arbitrary closures defined and serialized at your personal REPL running different code. (Which defeats half the purpose; you can't then use dehydrated closures as an arbitrary query/callback/RPC syntax between Erlang-based microservices, unless they all share code.) On the other hand, there's nothing explicitly stopping the Erlang Run Time System from supporting the full version, where a closure is dehydrated as its AST (plus free vars, as in ETF encoding.) And so this is exactly what Elixir does. Yet another reason Elixir is an interesting language.
      - dang 11y ago
        So what's encoded are the closed variables along with the gensym'ed name of the closure (effectively a pointer to the code)? If so, that's pretty useful. For some cases it's better than serializing the code, because then you don't have to pass a copy of the code around everywhere, only the state that you don't keep a copy of.