4 ms·

You're using ConnectX-6 Dx here. Any technical reason for that particular NIC, or just haven't gotten around to ConnectX-7s yet? Have you examined other NIC ve

• drewg123 4y ago
  This talk was given roughly 4 months ago. CX7 was not available yet. I'm looking forward to testing on them when we get some. We looked at Chelsio (as T6 was available well before CX6-DX). However, the CX6-DX offers a killer feature not available on T6. The CX6-DX can remember the crypto state of any in-order stream, while the T6 cannot. That means that the TCP stack can send, say, 4K of a TLS record, wait for acks, and come back 40ms later and send the next 4K and DMA just the requested 4K from the host. The T6 cannot remember the state, and would need to DMA the first 4K (which was already sent) in order to re-establish the crypto state, and then DMA the requested 4K. This could run the PCIe bus out of bandwidth. The alternative is to make TCP always chunk sends at the TLS record size, but this was horrible for streaming quality.