4 ms·
One thing I've run into when building my own 10gbps router that I didn't see mentioned - you probably need to disable power saving / frequency scaling. The hand
by r1ch 5y ago
One thing I've run into when building my own 10gbps router that I didn't see mentioned - you probably need to disable power saving / frequency scaling. The handling of packets is done in software, so when you start a download and the flood of initial data comes in at 10gbps, your CPU will still be running at 600 MHz and unable to keep up. It ramps up within 5-50 milliseconds depending on CPU, but for a TCP download the sender will have already interpreted the dropped packets as congestion and reduced their upload rate.
You can monitor this with ethtool stats to see how many packets the NIC dropped due to host buffers being full.
- secure 5y agoIn none of my tests did I need to change power saving settings. My iperf3 transfers stayed at 10 Gbps throughout.
- r1ch 5y agoYou need to check your dropped packet statistics to know for sure. iperf3 TCP over LAN will not see throughput affected by a few dropped packets, a stream 100ms away may take longer to recover. High bandwidth UDP testing with iperf3 should also expose this, the first measurement will usually see dropped packets.
- AdamJacobMuller 5y agoYou won't see the impact because you're testing over something with very low latency.
- bluedino 5y agoHe was only using 15% of the CPU at max speed
- r1ch 5y agoYou can't measure the CPU at 1 second granularity to see this problem, in fact measuring the CPU at the sampling rate required may be enough to raise the CPU frequency :).