4 ms·

My experience is across several years writing high performance http clients, writing in pure C.

by benwills 8y ago

kaendfinger 8y ago
This changed recently so both of you are right in different ways.
- benwills 8y ago
  Do you have a link to either an announcement of this change, or C code that demonstrates it? If it has changed, I might start doing some things differently...
  - caf 8y ago
    (2008) https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c7004482e8d https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
    - benwills 8y ago
      I dealt with all of this in 2015, and spent three months trying to get epoll to perform similarly to kqueue; namely, to avoid returning on any and every packet received and, preferably, to abide by some sort of minimum bytes received (or connection closing) before returning. Unless I'm missing something (and I could very well be missing something), I'm not seeing how that is any different than anything I tried...and I went through every socket option I could find, even if it seemed irrelevant. Have you written code that actually performs this way? Or are you speculating? I really do wish I could do this with epoll, but I (and several others) just never were able to figure it out. And kqueue simply performed at least an order of magnitude better of any variation I could hack together with epoll.
      - caf 8y ago
        I had neither written code to test it nor was speculating - I was reading the kernel source, which is pretty clear (if you know in general how file polling is implemented in the Linux kernel). However, because of what you have observed I wrote up a quick test, which confirms to my satisfaction that epoll_wait() respects SO_RCVLOWAT on TCP sockets: https://gist.github.com/keaston/d2473b8b996a34a5860b0744684f2017 https://gist.github.com/keaston/d2473b8b996a34a5860b0744684f... It seems likely that you must have been dealing with an old "enterprise" kernel in 2015 that hadn't had this fix backported to it?
        benwills 8y ago
        Wow. When I saw your code, I thought to myself "yeah, but does it work when the socket is for a client?" Turns out, it does. My only guess is that I was botching something else when I was digging into this a few years ago. I can't tell you how much I appreciate you putting that together. This seriously changes a lot of what I work on. Thank you. The client code I wrote: https://gist.github.com/benwills/95ee844853d3b18588fb3df7d5660a9f https://gist.github.com/benwills/95ee844853d3b18588fb3df7d56...
        caf 8y ago
        Good example. The errors are just because you need to check for EAGAIN or EWOULDBLOCK when read() returns -1, and not bail out when that happens.

My experience is across several years writing high performance http clients, writing in pure C.

kaendfinger 8y ago

This changed recently so both of you are right in different ways.

benwills 8y ago
Do you have a link to either an announcement of this change, or C code that demonstrates it? If it has changed, I might start doing some things differently...
- caf 8y ago
  (2008) https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c7004482e8d https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
  - benwills 8y ago
    I dealt with all of this in 2015, and spent three months trying to get epoll to perform similarly to kqueue; namely, to avoid returning on any and every packet received and, preferably, to abide by some sort of minimum bytes received (or connection closing) before returning. Unless I'm missing something (and I could very well be missing something), I'm not seeing how that is any different than anything I tried...and I went through every socket option I could find, even if it seemed irrelevant. Have you written code that actually performs this way? Or are you speculating? I really do wish I could do this with epoll, but I (and several others) just never were able to figure it out. And kqueue simply performed at least an order of magnitude better of any variation I could hack together with epoll.
    - caf 8y ago
      I had neither written code to test it nor was speculating - I was reading the kernel source, which is pretty clear (if you know in general how file polling is implemented in the Linux kernel). However, because of what you have observed I wrote up a quick test, which confirms to my satisfaction that epoll_wait() respects SO_RCVLOWAT on TCP sockets: https://gist.github.com/keaston/d2473b8b996a34a5860b0744684f2017 https://gist.github.com/keaston/d2473b8b996a34a5860b0744684f... It seems likely that you must have been dealing with an old "enterprise" kernel in 2015 that hadn't had this fix backported to it?
      - benwills 8y ago
        Wow. When I saw your code, I thought to myself "yeah, but does it work when the socket is for a client?" Turns out, it does. My only guess is that I was botching something else when I was digging into this a few years ago. I can't tell you how much I appreciate you putting that together. This seriously changes a lot of what I work on. Thank you. The client code I wrote: https://gist.github.com/benwills/95ee844853d3b18588fb3df7d5660a9f https://gist.github.com/benwills/95ee844853d3b18588fb3df7d56...
        caf 8y ago
        Good example. The errors are just because you need to check for EAGAIN or EWOULDBLOCK when read() returns -1, and not bail out when that happens.