5 ms·
I might be missing something, but Linux caters to this already: SO_RCVLOWAT and SO_SNDLOWAT Specify the minimum number of bytes in the buffer unti
by _wmd 8y ago
I might be missing something, but Linux caters to this already:
SO_RCVLOWAT and SO_SNDLOWAT
Specify the minimum number of bytes in the buffer until the socket
layer will pass the data to the protocol (SO_SNDLOWAT) or the user
on receiving (SO_RCVLOWAT). These two values are initialized to 1.
On the transmit side:
EPOLLET
Sets the Edge Triggered behavior for the associated file descriptor.
The default behavior for epoll is Level Triggered. See epoll(7) for
more detailed informa‐ tion about Edge and Level Triggered event
distribution architectures.
- benwills 8y agoCorrect. You can set those socket options and change the triggering behavior of epoll. But those socket options do not actually affect how epoll handles the data when it comes in. They are completely ignored by epoll.
- _wmd 8y agoNot sure how old your experience is, but I tested from Python via epoll a few minutes after posting (never used that option before!) and it worked as described.
- benwills 8y agoMy experience is across several years writing high performance http clients, writing in pure C.
- kaendfinger 8y agoThis changed recently so both of you are right in different ways.
- benwills 8y agoDo you have a link to either an announcement of this change, or C code that demonstrates it? If it has changed, I might start doing some things differently...
- caf 8y ago(2008) https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c7004482e8d https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
- benwills 8y agoI dealt with all of this in 2015, and spent three months trying to get epoll to perform similarly to kqueue; namely, to avoid returning on any and every packet received and, preferably, to abide by some sort of minimum bytes received (or connection closing) before returning. Unless I'm missing something (and I could very well be missing something), I'm not seeing how that is any different than anything I tried...and I went through every socket option I could find, even if it seemed irrelevant. Have you written code that actually performs this way? Or are you speculating? I really do wish I could do this with epoll, but I (and several others) just never were able to figure it out. And kqueue simply performed at least an order of magnitude better of any variation I could hack together with epoll.
- caf 8y agoI had neither written code to test it nor was speculating - I was reading the kernel source, which is pretty clear (if you know in general how file polling is implemented in the Linux kernel). However, because of what you have observed I wrote up a quick test, which confirms to my satisfaction that epoll_wait() respects SO_RCVLOWAT on TCP sockets: https://gist.github.com/keaston/d2473b8b996a34a5860b0744684f2017 https://gist.github.com/keaston/d2473b8b996a34a5860b0744684f... It seems likely that you must have been dealing with an old "enterprise" kernel in 2015 that hadn't had this fix backported to it?
- 8y ago
- benwills 8y agoBehind the scenes, Python may have another layer in there to check the socket options, etc, before returning. ie: Just because it looks like it's working on the Python level doesn't mean it's working the same way underneath.
- deleted 8y ago[deleted]
- benwills 8y agoIt's also worth noting that kqueue() also doesn't use the socket options for this. Instead, it uses a flag called EVFILT_READ. More here: https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2 https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2