5 ms·
"In 7 seconds, a hundred or more connections accumulate. So the server ends up with hundreds of threads, most of them probably waiting for input (waiting for th
by mmaunder 16y ago
"In 7 seconds, a hundred or more connections accumulate. So the server ends up with hundreds of threads, most of them probably waiting for input (waiting for the HTTP request). MzScheme can be inefficient when there are 100s of threads waiting for input -- when it wants to find a thread to run, it asks the O/S kernel about each socket in turn to see if any input is ready, and that's a lot of asking per thread switch if there are lots of threads. So the server is able to complete fewer requests per second when there is a big backlog, which lets more backlog accumulate, and perhaps it takes a long time for the server to recover."
I may have misunderstood but it sounds like you have MzScheme facing the open internet? Try putting nginx (or another epoll/kqueue based server) in front of MzScheme. It will handle the thousands of connections you have that are waiting for IO with very little incremental CPU load and with a single thread. Then when nginx reverse proxies to MzScheme each request happens very fast because it's local which means you need much fewer threads for your app server. That means less memory and less of the other overhead that you get with a high thread count.
An additional advantage is that you can enable keepalive again (right now you have it disabled it looks like) which makes things a faster for first-time visitors. It also makes it slightly faster for us regulars because the conditional gets we do for the gif's and css won't have to reestablish connections. Less connections established means you give your OS a break too with fewer syn/syn-ack/ack TCP handshakes.
Someone mentioned below that reverse proxies won't work for HN. They mean that caching won't work - but a reverse proxy like nginx that doesn't cache but handles high concurrency efficiently should give you a huge perf improvement.
PS: I'd love to help implement this free. I run a 600 req/sec site using nginx reverse proxying to apache.
- joshu 16y agoExactly this.
- sedachv 16y agoOr I don't know, use continuations in a place that's actually appropriate? John Fremlin showed that even with horrible CPS rewriting and epoll you can get way better throughput in SBCL (TPD2) than nginx. MzScheme comes with native continuations. It's not hard to call out to epoll. Instead everyone in the Lisp community (pg included) is still enamored with using continuations to produce ugly URLs and unmaintainable web applications.
- pg 16y agoInstead everyone in the Lisp community (pg included) is still enamored with using continuations to produce ugly URLs and unmaintainable web applications. If you read the source of HN, you'll see that it doesn't actually use continuations. I find the source of HN very clear. Have you read it? Is there a specific part you found so complicated as to be unmaintainable?
- axod 16y ago> If you read the source of HN, you'll see that it doesn't actually use continuations. > It had to be some dialect of Lisp with continuations, which meant Scheme, and MzScheme seemed the best. (From further down the page). I'm confused. What needs continuations?
- dauphin 16y agoErrors/exceptions, for one, are implemented using continuations.
- axod 16y agoSounds terribly inefficient to me, but what do I know -shrug-
- dauphin 16y agoYou know, if it weren't really impratical, numbers in Arc would be represented by Church numerals, "hash tables" would be alists, strings would be lists, etc. That's a credo :-)
- jrockway 16y agoAll control flow is a subset of continuations. The stack is a continuation (calling a function is call-with-current-continuation, return is just calling the "current continuation"), loops are continuations (with the non-local control flow, like break/last/redo/etc.), exceptions are continuations (like functions, but returning to the frame with the error handler), etc. Continuations are the general solution to things that are normally treated as different. So continuations are just as efficient (or inefficient) as calling functions or throwing exceptions. In a web app context, though, it's kind of silly to keep a stack around to handler something like clicking a link that returns the contents of database row foo. People do this, call it continuations, and then run into problems. The problem is not continuations, the problem is that you are treating HTTP as a session, not as a series of request/responses. (The opposite of this style is REST.)
- ezalor 16y agoCan't upvote you enough for this well-written post!
- esbcupper 16y agoFilo's BSD hack to buffer the entire HTTP request and then pass it on has the same effect as using nginx here. It doesn't help with keepalive though, but that's probably not needed.
- akkartik 16y ago"it sounds like you have MzScheme facing the open internet?" Yep. The arc webserver runs directly on port 80, for which it needs to run as root. To avoid all sorts of security headaches, it runs: (setuid 2) soon after startup. The whole thing seems hacky.
- sayrer 16y agoThey said caching wouldn't work, but they could be wrong. You can't change the Cache-Control header to public for HN responses, because the same URL can appear different to different users. There may be some ways around this, including giving each logged-in user their own URL to browse with. But that might be a lot of work. You can still set up a proxy that kicks in only for requests that don't contain a session cookie. Then, requests without a cookie can be responded to with a cached copy from Varnish, and Varnish could refresh every 30 seconds or so. That might reduce the number of connections to MzScheme by quite a lot.