I'm afraid that doesn't solve the problem I cite either. Using Mode 2 or Mode 4 with xmit_hash_policy=layer3+4 (so you're load balancing on the IP/port tuple)

I'm afraid that doesn't solve the problem I cite either. Using Mode 2 or Mode 4 with xmit_hash_policy=layer3+4 (so you're load balancing on the IP/port tuple) still only balances one TCP stream out one slave. An out-of-order might be common over the internet, but shouldn't happen on a good high-speed LAN connection. As we approach tens of gigabits, too much out-of-order processing will kill wirespeed bandwidths. Considering people suggest disabling TCP SACK because the CPU overhead is (allegedly) too high on good working connections, out-of-order processing cost would be much higher. All bonding modes except Mode 4 (LACP) and Mode 6 (ALB) are transmit-only. The receive throughput on a Mode 0/1/2/3/5 bond is still the max of one slave.