4 ms·

(I wrote the Serverfault question) Thanks for the suggestions, here's some clarifications: * I clarified what hardware I've tested in a comment (see below).

(I wrote the Serverfault question) Thanks for the suggestions, here's some clarifications: * I clarified what hardware I've tested in a comment (see below). * I've run tests with my server ranging from 10 sec up to 10 minutes. They're consistently bad unfortunately. * Interesting what you say about dom0/domU. I'm no Xen guru, but the culprit is probably something like that. I've been using a backlog of 1024 for the server tests (set both in Java land and sysctl.conf). The netperf are all defaults, both in terms of run time and backlog. Was actually trying to monitor the backlog somehow, but I'm not sure that's even possible in Linux? * The server isn't doing anything disk IO-bound so this shouldn't be the case. * syncookies seems like a good idea, I will definitely try that along with a lot bigger backlog and see if it makes any difference. I'll also try and see if netperf can be tweaked as well to provide a better, isolated test case. Writing this off as Xen overhead would be such a shame, virtualization should not cause this much overhead. I'll continue investigating!

ay 15y ago
A nit re. syncoookies: TCP three-way handshake always occurs in the kernel. You can grab the kernel source, the relevant stuff is in net/ipv4/tcp_input.c; What syncookie may somewhat help you with is if you have super-large number of half-open (SYNRCVD state) connections - and even then, the data structures for those are supposed to be efficient enough for this not to be a problem. OTOH, what you will be trading off with syncookies, is that they subtly violate the TCP standard, and especially this will be a problem with the "server talks first" connections (like [E]SMTP, SSH): if your third ACK of the three way handshake gets lost on the way from client to the server, the canonical TCP implementation would have retransmitted the SYNACK from the server side. Except that the whole point of the syncookies is not to keep the state on the server side, i.e. there is nothing that can at all retransmit the SYNACK. The HTTP, being by nature "client talks first" protocol, hides this problem (the ACK for the SYNACK will be effectively retransmitted because it will be part of the data segment), but I thought it might be useful to remind that turning syncookies on by default is not the standard modus operandi.

ay 15y ago

A nit re. syncoookies: TCP three-way handshake always occurs in the kernel. You can grab the kernel source, the relevant stuff is in net/ipv4/tcp_input.c; What syncookie may somewhat help you with is if you have super-large number of half-open (SYNRCVD state) connections - and even then, the data structures for those are supposed to be efficient enough for this not to be a problem. OTOH, what you will be trading off with syncookies, is that they subtly violate the TCP standard, and especially this will be a problem with the "server talks first" connections (like [E]SMTP, SSH): if your third ACK of the three way handshake gets lost on the way from client to the server, the canonical TCP implementation would have retransmitted the SYNACK from the server side. Except that the whole point of the syncookies is not to keep the state on the server side, i.e. there is nothing that can at all retransmit the SYNACK. The HTTP, being by nature "client talks first" protocol, hides this problem (the ACK for the SYNACK will be effectively retransmitted because it will be part of the data segment), but I thought it might be useful to remind that turning syncookies on by default is not the standard modus operandi.