4 ms·
C10K is another name for evented I/O, and it's from the 90s. By 2003 thread-per-client was already obsolete and known to be very bad. It's really quite simple
by cryptonector 7y ago
C10K is another name for evented I/O, and it's from the 90s. By 2003 thread-per-client was already obsolete and known to be very bad.
It's really quite simple: threads encourage the use of implicit program state via function calls, with attendant state expansion (stack allocation), whereas evented I/O encourages explicit program state, which means the programmer can make it as small as possible.
Smaller server program state == more concurrent clients for any given amount of memory. Evented I/O wins on this score.
But it gets better too! Smaller program state == less memory, L1/2/3 cache, and TLB pressure, which means the server can take care of each client in less time than an equivalent thread-per-client server.
So evented I/O also wins in terms of performance.
Can you write high-performance thread-per-client code? Probably, but mostly by allocating small stacks and making program state explicit just as in evented I/O, so then you might as well have done that. Indeed, async/await is a mechanism for getting thread-per-client-like sequential programming with less overhead: "context switching" becomes as cheap as a function call, while thread-per-client's context switches can never be that cheap.
The only real questions are:
- async/await, or hand-coded CPS callback hell?
- for non-essential services, do you start with
thread-per-client because it's simpler?
The answer to the first question is utterly dependent on the language ecosystem you choose. The answer to the second should be context-dependent: if you can use async/await, then always use async/await, and if not, it depends on how good you are at hand-coded CPS callback hell, and how well you can predict the future demand for the service in question.
- cryptonector 7y agoI don't normally care about downvotes, but if I give out a nice explanation and you don't like it, it'd be nice to get a reply. Was my response wrong? How? I might learn something from your response.
- zzzcpan 7y agoIt's a trolling dogmatic paper. So of course some people will downvote you if you criticize the dogma.
- afcapel 7y agoA context switch in a modern CPU takes only a few microseconds. A GB of RAM costs less than $10. So those concerns, although valid in theory, are usually irrelevant for most web applications. On the other hand, simplicity in a code base usually matter. Code written with an evented API, littered of callbacks, is usually harder to read and maintain than that written in a sequential way with a blocking I/O API. You can recreate a sync API on top of an evented architecture using async/await, but then you have the same performance characteristics of a blocking API, but with all the evented complexity lurking underneath and leaking here and there. Seems to me a very convoluted way to arrive to the point from where we started.
- cryptonector 7y agoA function call takes less. And using more RAM == thrashing your caches more == slowing down. The price of RAM isn't relevant to that -- this isn't about saving money on RAM but saving cycles. Yes, yes, that's saving money per-client (just not on RAM), but you know, in a commoditized services world, that counts, and it counts for a lot.
- nine_k 7y agoA GB of RAM only costs less than $10 if you are buying for your unpretentious gaming rig. A GB of ECC server RAM costs more. An extra GB of RAM in the cloud can even cost you $10/mo if you have to switch to a beefier instance type.
- projektfu 7y agoThat's true, if you're buying OEM ram for Dell or HP servers, it's more like $10-20/GB. However you can buy Crucial ECC DDR4 ram for $6/GB, so there's a hefty OEM markup.
- Roboprog 7y agoHow much does a MB of L-n cache cost? I don’t have the answer, but you would want to measure dollars to buy it, and nanoseconds to refill it.
- gowld 7y ago
- loeg 7y agoEvented has some difficulties in practice: * On Unix, only socket IO is really evented; this can be solved by using a thread pool (the Flash paper describing this problem and solution for httpds dates to 1999[1]) but is inelegant. This is the approach Golang takes behind the scenes. * How to scale event loop work across cores; this can be solved, but adds complexity. In contrast, threads are quite simple to scale. * How to share accept() workload across cores; this can be solved, too, but not portably. I.e. your 100 core server may very well be accept-limited if you have a single-core accept loop, or highly contend on a single socket object. * Threadpools are definitely inappropriate if you have a ton of relatively idle clients (long-poll server, or even typical webserver), but are less wasteful if you have relatively few, always-busy clients. I don't disagree that the synchronous threadworker design isn't the best choice for high performance services that can afford a lot of engineering time to design and find all the bugs. But thread-per-client is often a completely acceptable place to start. [1]: https://www.usenix.org/events/usenix99/full_papers/pai/pai.pdf https://www.usenix.org/events/usenix99/full_papers/pai/pai.p...
- cryptonector 7y agoThese are minor things, and not really accurate, and at any rate, not relevant to the point that evented I/O == explicit, thus easy-to-minimize program state, while thread-per-client == program state expansion and costlier context switches.