4 ms·
There is nothing fundamentally new described in the blog post, although pinning threads to core is almost never the default operation mode (though many popular
by markpapadakis 11y ago
There is nothing fundamentally new described in the blog post, although pinning threads to core is almost never the default operation mode (though many popular servers expose an option for turning this one).
As someone else stated in another post, SO_REUSEPORT and accept4() is best, all things considered, way to accept connections across multiple threads. Again, most modern servers support this by default, if supported by the underlying operating system (e.g nginx).
By the way, Aerospike accepts new connections in a single thread and then directly adds the socket FD to one of the i/o threads (round-robbing selection scheme) directly using e.g epoll_ctl(fd, EPOLL_CTL_ADD, ..).
See http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-context.html http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-ma... for costs of context switching, and performance improvement when pinning threads to cores (it's quite impressive). Also, note that according to the author, on average, a context switch is 2.5-3x more expensive when using virtualization.
You may also want to read https://medium.com/software-development-2/high-performance-services-using-coroutines-ac8e9f54d727#.ho0s7q28b https://medium.com/software-development-2/high-performance-s... -- it's been a long time since I wrote this, but it describes how one can deal with asynchronous I/O and other operations that may block a thread.
- geocar 11y agoI'm glad Aerospike uses this technique. Yes, it is not new. I've been using this method in one form or another (e.g. SIGIO/F_SETOWN) for more than a decade, however nginx (relatively modern?) doesn't use this technique and could really benefit from it on some workloads. I wrote about it after I casually mentioned it a roomful of quant programmers and found out they didn't know about it; I do not think most servers use this technique. I've added a note about accept4(). When I have some time I intend to repeat my benchmarks with it to see how it does. Context switching is much cheaper than memory so I do not expect much. You should publish benchmarks that are easy for other people to reproduce. I tried running Aerospike on my laptop and was unable to get above 25,000 writes/sec. I'm certain I'm doing something wrong, because the same hardware easily gets above 60,000 writes/sec on KDB per CPU.
- markpapadakis 11y agoI am not using AeroSpike nor am I affiliated with them. I just studied their code base (I study codebases as a hobby ). There is a _lot_ of room for improvement there IMO but that's true for the majority of codebases I went through :)