4 ms·

I'm not entirely convinced that increasing the size of LHTABLE solves anything. True, it may remove some collisions in the hash table but note that 63925 % 64 =

by ecma 11y ago

I'm not entirely convinced that increasing the size of LHTABLE solves anything. True, it may remove some collisions in the hash table but note that 63925 % 64 = 53. Given that the two slow ports listed seem to be arbitrary and assigned to customers, they're probably just a symptom of the overload on 53/UDP. I'm not suggesting they chose 64, that's unclear, but whatever they chose probably just shifts problem the elsewhere. Increasing it /would/ inherently reduce the frequency of the events though so you can call that a win. A naïve solution would be to choose a bucket based on the destination port as well as the source port if one is available (e.g. TCP). This might help balance load affecting particular local ports since we can assume the source port for TCP will be random enough. However, it doesn't solve the problem - it'll just hide it. Random spikes in latency for connections to random customers? Sounds undesirable. A reasonable solution might be to work out a way to map gateway 53/UDP to a diverse set of ports which are bound to rrdns processes on the boxes which currently have 16K IP addresses. For UDP packets, this would be possible by doing on-wire modifications to the transport header and recalculating any checksums. Perhaps that just shifts the burden though.

eridius 11y ago
You can't include the source port in the hash, because this is a table of listening sockets, i.e. no connection has been established yet and the socket needs to see packets from ANY source as long as they go to the right destination. You could suggest including the bound destination IP in the hash, but then you'd also need a separate hashtable for sockets that are bound to any IP (instead of being bound to a specific IP).
- caf 11y ago
  You wouldn't need a separate hash table, you could first look up Hash(destip, destport) then if that fails to find a listener, look up Hash(0, destport).
- ecma 11y ago
  Good catch. I did mean that mixing in the source would apply to the established table if it's constructed in the same way but doesn't do so already (which would surprise me now that I think about it properly). I don't think that you'd need a separate table for star bound listeners if the IP is mixed in since you could just hash in 0.0.0.0 but you'd need to check both the real IP and the special value too which is a potentially damaging performance hit. It's probably done with just the destination port for a good reason.

by ecma 11y ago

eridius 11y ago

You can't include the source port in the hash, because this is a table of listening sockets, i.e. no connection has been established yet and the socket needs to see packets from ANY source as long as they go to the right destination. You could suggest including the bound destination IP in the hash, but then you'd also need a separate hashtable for sockets that are bound to any IP (instead of being bound to a specific IP).

caf 11y ago
You wouldn't need a separate hash table, you could first look up Hash(destip, destport) then if that fails to find a listener, look up Hash(0, destport).
ecma 11y ago
Good catch. I did mean that mixing in the source would apply to the established table if it's constructed in the same way but doesn't do so already (which would surprise me now that I think about it properly). I don't think that you'd need a separate table for star bound listeners if the IP is mixed in since you could just hash in 0.0.0.0 but you'd need to check both the real IP and the special value too which is a potentially damaging performance hit. It's probably done with just the destination port for a good reason.