3 ms·

"What makes you think that a thread pool dispatching events would use a lock? No modern efficient thread pool implementation dispatching work items is serialize

by sedachv 16y ago

"What makes you think that a thread pool dispatching events would use a lock? No modern efficient thread pool implementation dispatching work items is serialized by a single lock, to my knowledge." Are you talking about thread-safe queues implemented using atomic instructions? Those aren't free - how do you think they're implemented at the hardware level? The main advantage of atomic instructions over locks is removing the possibility of waiting on a preempted thread holding the lock (lock-freedom). They also have lower overhead than making system calls to provide locking. But the equivalent mutual exclusion logic (and contention penalties) just get moved down to the chipset level - now instead of waiting on other threads, you're waiting on other cores/CPUs. "What you ought to have is multiple async accept calls outstanding, and as they complete they are entered into work queues. Worker threads (i.e. the thread pool) pulling work of those queues should steal work from other queues when their own queue is empty. See e.g. http://www.bluebytesoftware.com/blog/2008/09/17/BuildingACus... http://www.bluebytesoftware.com/blog/2008/09/17/BuildingACus... That article is horrible. Please do not follow the author's advice. Besides the fact that the code deadlocks (see the first comment on the article), it's also easy to see that if one thread starts generating all the work the "solution" is going to become a single global queue (the "local work stealing queue" of that thread), with all the other threads looping through the global queue and then all each other's queues just to reach there! The one good thing about that article is that the throughput gains on the toy benchmark (which doesn't deadlock or spawn tasks non-uniformly) nicely illustrate my point about the expense of contention even if using atomic instructions. What the code in the article attempts to do is alleviate contention by partitioning the tasks among several queues. The problem is that if the work is not distributed uniformly among the queues, some threads will be left idle. The way to overcome that is to fake a global queue by having some way to synchronize the partitioned queues. The optimal solution to how to do this depends not only on the particular system you're running on, but also the pattern of work spawning by the application. And all this depends on being deadlock-free (something not managed by the article)! Why would you ever go through something so horrible for an HTTP server? Multiple node.js processes are much simpler and more efficient. PS - The "GC" you're describing is called CGI.

barrkel 16y ago
I don't think you actually read what I wrote, or if you did, you willfully misunderstood it. "The problem is that if the work is not distributed uniformly among the queues, some threads will be left idle" - this is why you use work-stealing queues! The very nature of work stealing queues is that the worker threads aren't left idle - they steal work from other threads' queues. And CGI is not anything like the GC I talked about - if you have a process per request, where are you going to put your shared state? But much of this discussion is besides the point. Don't forget, the OS scheduler is at its heart an event dispatcher when there are more runnable threads than CPU cores. The thread stack is little different than context provided to a triggered event. You want to have the same number of runnable threads as CPU cores in order to avoid the kernel cost of a context switch. You can do that by having multiple single-threaded processes, or multiple threads in a single process. While neither choice of partitioning affects the degree to which you can use an eventing style to serve requests, one - the separate process model - makes it much harder to share state. And therein lies the reason why I believe that optimal performance lies in threads, rather than processes. There are other good reasons for using processes instead - but it will be at some cost to efficiency.

by sedachv 16y ago

barrkel 16y ago

I don't think you actually read what I wrote, or if you did, you willfully misunderstood it. "The problem is that if the work is not distributed uniformly among the queues, some threads will be left idle" - this is why you use work-stealing queues! The very nature of work stealing queues is that the worker threads aren't left idle - they steal work from other threads' queues. And CGI is not anything like the GC I talked about - if you have a process per request, where are you going to put your shared state? But much of this discussion is besides the point. Don't forget, the OS scheduler is at its heart an event dispatcher when there are more runnable threads than CPU cores. The thread stack is little different than context provided to a triggered event. You want to have the same number of runnable threads as CPU cores in order to avoid the kernel cost of a context switch. You can do that by having multiple single-threaded processes, or multiple threads in a single process. While neither choice of partitioning affects the degree to which you can use an eventing style to serve requests, one - the separate process model - makes it much harder to share state. And therein lies the reason why I believe that optimal performance lies in threads, rather than processes. There are other good reasons for using processes instead - but it will be at some cost to efficiency.