3 ms·
For calls that don't need to know about more state than their arguments, we just use an ordinary url with arguments. Otherwise we make a closure and store it i
by pg 20y ago
For calls that don't need to know about more state than their arguments, we just use an ordinary url with arguments. Otherwise we make a closure and store it in a hash table in memory, and make the url be x?fnid= where the argument is the hash key. When an http request comes in, we call the corresponding closure. Sometimes what to do next after following a link generated by the closure (e.g. what to do after logging in when you're not logged in and you submit a link through the bookmarklet) is part of the state of the closure; in that case it's a bit like a continuation.
We keep only the most recently generated 20,000 closures. When you click on a link that says it has expired, that means either it has been purged from the closure table, or that we've restarted the server.
- malkia 20y agoIsn't there a risk, that someone can (accidentally or not) enter hash-code of someone else's "closure"?
- malkia 20y agoIsn't there a risk, that someone can (accidentally or not) enter hash-code of someone else's "closure"?
- mdakin 20y agoOne could avoid that problem by keeping track of the owner of each closure and ensuring the user is properly authenticated before calling the closure.
- pg 20y agoThe code explicitly protects against that.
- papersmith 20y agoIf access to the closures are session-specific, you'll have to hijack the session to do that.
- papersmith 20y ago"in that case it's a bit like a continuation." Doesn't MzScheme (assuming it's what Arc is built on) have first-class continuation already?
- pg 20y agoIt does, and so does Arc, but I don't think we've needed them in anything we've written so far. News.YC is a pretty simple application.
- damir 20y agoCan we expect to see Arc go public in the near future?
- shiro 20y agoFrom my experience of continuation-based web app, what you're dealing with is not a full continuation captured by call/cc, but rather a delimited (partial) continuation, which captures the continuation of your application logic but not the state of underlying protocols. You can implement delimited continuations on top of call/cc, and I did based on Gasbichler&Sperber paper (ICFP02), but in most cases explicit CPS wrapped in some macros are just as well worked.
- ralph 20y agoWhat extra data is in the closure for "reply" that means it isn't a simple URL? Is it just what to do after the reply is complete, i.e. what post to return to viewing? Is there a reason this isn't taken from the Referer instead? I guess this technique doesn't scale too well unless the hash table is either shared across servers or a user is kept on one server, which then causes problems of its own. I'm guessing 20,000 closures is plenty and I've been bitten by server restarts. Out of curiosity, do you keep track of the minimum age of the 20,000th hash entry prior to deleting to make room for another? If that's something huge, it would confirm server restarts are the normal reason the "unknown fnid" message is seen.
- avibryant 20y agoWhat particular problems are caused by the user being kept on one server?
- ralph 20y agoThe redundancy of having five servers isn't helping if I'm always served by server #3, whereas an alternate method where any one of the five can answer my current request allows a machine to fail, or be taken down for upgrading, and the load on the other four just increases by 25% each.
- ralph 20y agoHaving thought about this a little more, doesn't this approach lay you open to an easy denial of service attack? I may not have the resources of CPU or bandwidth to hammer a site using this technique, but it seems that the fnids are unique to the page they're generated for, perhaps because the time is part of the key, so by requesting a page enough times I can soon use up the 20,000 closures. This would cause "unknown fnid" errors for other users despite them having only recently generated the page. Here's why I think this. Apologies for the formatting. $ fnid() { wget -qO - "http://news.ycombinator.com/threads?id=${1?} http://news.ycombinator.com/threads?id=${1?}" | tr '?"' '\n\n' | sed -n 's/^fnid=//p'; } $ fnid ralph | sort 1 $ fnid ralph | sort 2 $ wc -l 1 2 | tr \\n , 177 1, 177 2, 354 total, $ comm 1 2 | sed 's/[^\t].//; s/./x/g; s/^/x/' | sort | uniq -c | tr \\n , 177 x, 177 xx, comm(1) is showing the two fnid lists are disjoint. 177 occur only in file 1. 177 occur only in file 2. 0 occur in both (xxx). 20,000 / 177 = 112.99, so 113 "fnid ralph" would flush the hash table. I'm not trying to pick holes, just interested in the techniques that can be used and would welcome opinions.