5 ms·

This article seems a bit over the top. I implemented the MySQL interface to libevent for mysql 6.0, and we got really good performance numbers vs. thread per co

This article seems a bit over the top. I implemented the MySQL interface to libevent for mysql 6.0, and we got really good performance numbers vs. thread per connection: http://krow.livejournal.com/572937.html http://krow.livejournal.com/572937.html I've spent a fair amount of time inside the libevent library source, and I'm pretty sure adding new connections is constant time as far as the library goes. There is however the underlying cost of the event struct memory allocation and the cost of the low level apis, but otherwise, IIRC, they keep it down to a single memalloc and link it into the head of internal linked lists for fixed cost insertion. The observed log(N) insertion cost comes likely from the low level poll apis, and there is pretty much nothing libevent can do to make that faster. However you can choose the low level api used on some platforms via build options, so theoretically you can use a poll api better suited to your workload.

tptacek 18y ago
My impression was that Marek assumed every read event would always have a specified timeout, and his insertion time referred to inserting into the timer queue.
- damienkatz 18y ago
  I see, but if it's a timer problem then with libevent you can roll your own timer queue optimized for your workload. The libevent core is small and well designed (but not well documented), integrating custom timer mechanisms is actually pretty straightforward. However the claim is made that connections are added and removed more often than polling is conducted (definitely the opposite of the average mysql workload), so I'd guess most low level high performance poll apis aren't optimized for that sort of workload.
  - tptacek 18y ago
    The only place where I'd intuit that the event queue is modified as often as raw I/O is performed is for async writes (you're potentially adding/removing them every other timeslice). Obviously I agree with you on the timer queue thing; I got a performance win simply by shoplifting the timer queue implementation out of MIT Athena; it took maybe a couple hours to integrate and test, and I didn't need to touch the libevent internals to do it.

tptacek 18y ago

My impression was that Marek assumed every read event would always have a specified timeout, and his insertion time referred to inserting into the timer queue.

damienkatz 18y ago
I see, but if it's a timer problem then with libevent you can roll your own timer queue optimized for your workload. The libevent core is small and well designed (but not well documented), integrating custom timer mechanisms is actually pretty straightforward. However the claim is made that connections are added and removed more often than polling is conducted (definitely the opposite of the average mysql workload), so I'd guess most low level high performance poll apis aren't optimized for that sort of workload.
- tptacek 18y ago
  The only place where I'd intuit that the event queue is modified as often as raw I/O is performed is for async writes (you're potentially adding/removing them every other timeslice). Obviously I agree with you on the timer queue thing; I got a performance win simply by shoplifting the timer queue implementation out of MIT Athena; it took maybe a couple hours to integrate and test, and I didn't need to touch the libevent internals to do it.