10 ms·

Do you think that continuations are the right model for web applications (does this model still offer enough advantages if you have to make sure that you don't

by jules 18y ago

Do you think that continuations are the right model for web applications (does this model still offer enough advantages if you have to make sure that you don't have too many closures)?

pg 18y ago
I think they're like using lists to represent stuff. They're good for exploratory programming because they're extremely clean and flexible. When your focus switches to efficiency, you can switch to other representations. It's hard to beat closures for elegance. You say: generate a link, and if the user clicks on it, run the following code. There's so much less to think about. In a closure, state is implicit: if you refer to a variable defined outside its scope, its value stays around. When you switch to putting state in urls, you have to explicitly encode and decode each bit of state you'll need. You also have to worry about bad things someone might do by sending requests with different arguments. It's sort of like the difference between dynamic memory allocation and explicit malloc, times 10.
- icey 18y ago
  Would it be fair to say that the issue wasn't with closures and the web as much as it was with the specific implementation of closures in news.YC? [edited for grammar / clarity]
  - pg 18y ago
    Not so much the implementation of them, but the fact that the site used so many. A crawler hitting some old item page with 20 comments on it would generate at least 40 new closures, 20 for the upvote links and 20 for the reply links. That adds up, now that there are almost 250k potential pages in the site.
    - icey 18y ago
      Thanks for the explanation - This is useful for those folks out there who are working with webservers that utilize continuations. There has been a ton of discussion surrounding stateless and session based web development, but a dearth of information regarding continuation based web development; especially as it relates to sites that get a reasonably high level of traffic.
  - jmatt 18y ago
    I agree one way to fix it is through design.
- benreesman 18y ago
  One thing that I like to do is use opaque representations of a data structure (like base64-encoded JSON) in URLs in situations where I've been compelled to optimize away one of my closures or continuations. The amount of code increases and it is certainly less elegant, but I only write a little more code by hand (the encoded data structure acts like the 'scope') and I keep the script kids from messing with the GET arguments.
  - paulgb 18y ago
    How does that stop script kids from messing with the GET arguments? You are still passing state in the URL, no?
    - benreesman 18y ago
      i can make the encoding of the data arbitrarily secure at a cost of time/space. in the simplest case of base64, it is just security by obscurity which is why i only asserted that it keeps out juvenile delinquents. if you want it to be robust then you must use encryption of course.
      - eru 18y ago
        Signatures.
- jules 18y ago
  Have you thought about handling this at the language level? Instead of allocating the closure on the server you send a string representation of the closure to the client. First make sure that the lambda doesn't have free variables by adding the free variables to the argument list. (lambda (x) (+ n x)) => (lambda (x n) (+ n x)) Because the resulting lambda doesn't have any free variables you can turn it into a global function. (lambda (x n) (+ n x)) => function#345 and somewhere else you (define (function#345 x n) (+ n x)). Next you store the values of the free variables in a hash table (use the memory location of the data as the key for the hash). Now you have a string representation for this closure. The representation contains the function's name (function#345) and the keys of the free variables in the hash table. An example. Assume that users are represented by a cons cell (cons user-name karma). We're writing a page that has a link called "increment karma". When you click this link your karma is incremented. (let ((user the-current-user)) (display-link "increment karma" (lambda () (set-cdr user (+ 1 (cdr user)))))) First transform this to: (define (function#345 user) (set-cdr user (+ 1 (cdr user)))) (let ((user the-current-user)) (display-link "increment karma" (make-closure function#345 user))) make-closure in pseudo code (assuming there is one free variable/argument): (define (make-closure fn arg) (add-to-hash *free-variables* (memory-location arg) arg) (add-to-hash *functions* (memory-location fn) fn) (concat (to-string (memory-location fn)) "/" (to-string (memory-location arg)))) When the server gets a request it parses the string representation of the closure, looks up the free variables and calls the function. (define (invoke-closure closure) (let ((fn <lookup thing before "/" in *functions*>) (arg <lookup thing after "/" in *free-variables*>)) (fn arg))) This solves the problem of allocating closures but the global hash table keeps growing. This may not be a problem in the average case. Consider the usual data in free variables: front page links, users, etc. These things have to be stored on the server anyway. For temporary data that isn't stored permanently you can use a seperate hash table that is cleaned up every x minutes or after it gets too big (just like the closures you're storing now). What do you think? I'm sorry if this explanation isn't clear. I'm not good at explaining these things, especially in english.
  - pg 18y ago
    Making printed representations of closures is a very hard problem. I wouldn't say insoluble, if you were willing make compromises in the language semantics, but one sign of its hardness is that all Lisp implementations I know of punt on it. You could write a whole PhD dissertation on this topic. My gut tells me it probably wouldn't be worth it. Closures don't feel like something one should expect to be representable. But I'm not 100% certain.
- davidmathers 18y ago
  "You say: generate a link, and if the user clicks on it, run the following code." Maybe I'm being dense here, but how is: (user state + resource state) -> link -> code different than: (resource state -> link) + user state -> code other than that in the first instance you're encoding user state in the control layer? In both cases you're mapping (user state + resource state) to a specific block of code. How does the order you do it in affect anything else?
- almkglor 18y ago
  http://arclanguage.com/item?id=7567 http://arclanguage.com/item?id=7567
benreesman 18y ago
In our in-house web framework (which I promise we will open-source at some point) you have access to both straightforward closures and a more sophisticated primitive which is one or more closures backed by a continuation. So I can say: let value = linkToContinuation(blah blah, ..., function(F) { }), and when one of the closures returns I wake up with 'value' populated and go about my business. I find that both are extremely useful.

by jules 18y ago

Do you think that continuations are the right model for web applications (does this model still offer enough advantages if you have to make sure that you don't have too many closures)?

pg 18y ago

I think they're like using lists to represent stuff. They're good for exploratory programming because they're extremely clean and flexible. When your focus switches to efficiency, you can switch to other representations. It's hard to beat closures for elegance. You say: generate a link, and if the user clicks on it, run the following code. There's so much less to think about. In a closure, state is implicit: if you refer to a variable defined outside its scope, its value stays around. When you switch to putting state in urls, you have to explicitly encode and decode each bit of state you'll need. You also have to worry about bad things someone might do by sending requests with different arguments. It's sort of like the difference between dynamic memory allocation and explicit malloc, times 10.

icey 18y ago
Would it be fair to say that the issue wasn't with closures and the web as much as it was with the specific implementation of closures in news.YC? [edited for grammar / clarity]
- pg 18y ago
  Not so much the implementation of them, but the fact that the site used so many. A crawler hitting some old item page with 20 comments on it would generate at least 40 new closures, 20 for the upvote links and 20 for the reply links. That adds up, now that there are almost 250k potential pages in the site.
  - icey 18y ago
    Thanks for the explanation - This is useful for those folks out there who are working with webservers that utilize continuations. There has been a ton of discussion surrounding stateless and session based web development, but a dearth of information regarding continuation based web development; especially as it relates to sites that get a reasonably high level of traffic.
- jmatt 18y ago
  I agree one way to fix it is through design.
benreesman 18y ago
One thing that I like to do is use opaque representations of a data structure (like base64-encoded JSON) in URLs in situations where I've been compelled to optimize away one of my closures or continuations. The amount of code increases and it is certainly less elegant, but I only write a little more code by hand (the encoded data structure acts like the 'scope') and I keep the script kids from messing with the GET arguments.
- paulgb 18y ago
  How does that stop script kids from messing with the GET arguments? You are still passing state in the URL, no?
  - benreesman 18y ago
    i can make the encoding of the data arbitrarily secure at a cost of time/space. in the simplest case of base64, it is just security by obscurity which is why i only asserted that it keeps out juvenile delinquents. if you want it to be robust then you must use encryption of course.
    - eru 18y ago
      Signatures.
jules 18y ago
Have you thought about handling this at the language level? Instead of allocating the closure on the server you send a string representation of the closure to the client. First make sure that the lambda doesn't have free variables by adding the free variables to the argument list. (lambda (x) (+ n x)) => (lambda (x n) (+ n x)) Because the resulting lambda doesn't have any free variables you can turn it into a global function. (lambda (x n) (+ n x)) => function#345 and somewhere else you (define (function#345 x n) (+ n x)). Next you store the values of the free variables in a hash table (use the memory location of the data as the key for the hash). Now you have a string representation for this closure. The representation contains the function's name (function#345) and the keys of the free variables in the hash table. An example. Assume that users are represented by a cons cell (cons user-name karma). We're writing a page that has a link called "increment karma". When you click this link your karma is incremented. (let ((user the-current-user)) (display-link "increment karma" (lambda () (set-cdr user (+ 1 (cdr user)))))) First transform this to: (define (function#345 user) (set-cdr user (+ 1 (cdr user)))) (let ((user the-current-user)) (display-link "increment karma" (make-closure function#345 user))) make-closure in pseudo code (assuming there is one free variable/argument): (define (make-closure fn arg) (add-to-hash *free-variables* (memory-location arg) arg) (add-to-hash *functions* (memory-location fn) fn) (concat (to-string (memory-location fn)) "/" (to-string (memory-location arg)))) When the server gets a request it parses the string representation of the closure, looks up the free variables and calls the function. (define (invoke-closure closure) (let ((fn <lookup thing before "/" in *functions*>) (arg <lookup thing after "/" in *free-variables*>)) (fn arg))) This solves the problem of allocating closures but the global hash table keeps growing. This may not be a problem in the average case. Consider the usual data in free variables: front page links, users, etc. These things have to be stored on the server anyway. For temporary data that isn't stored permanently you can use a seperate hash table that is cleaned up every x minutes or after it gets too big (just like the closures you're storing now). What do you think? I'm sorry if this explanation isn't clear. I'm not good at explaining these things, especially in english.
- pg 18y ago
  Making printed representations of closures is a very hard problem. I wouldn't say insoluble, if you were willing make compromises in the language semantics, but one sign of its hardness is that all Lisp implementations I know of punt on it. You could write a whole PhD dissertation on this topic. My gut tells me it probably wouldn't be worth it. Closures don't feel like something one should expect to be representable. But I'm not 100% certain.
davidmathers 18y ago
"You say: generate a link, and if the user clicks on it, run the following code." Maybe I'm being dense here, but how is: (user state + resource state) -> link -> code different than: (resource state -> link) + user state -> code other than that in the first instance you're encoding user state in the control layer? In both cases you're mapping (user state + resource state) to a specific block of code. How does the order you do it in affect anything else?
almkglor 18y ago
http://arclanguage.com/item?id=7567 http://arclanguage.com/item?id=7567

benreesman 18y ago

In our in-house web framework (which I promise we will open-source at some point) you have access to both straightforward closures and a more sophisticated primitive which is one or more closures backed by a continuation. So I can say: let value = linkToContinuation(blah blah, ..., function(F) { }), and when one of the closures returns I wake up with 'value' populated and go about my business. I find that both are extremely useful.