3 ms·

Let's find out with a sample size of 1: Server is an HP Microserver with an Intel Xeon E3-1265L V2 @ 2.50GHz running FreeBSD 12.1. Client is a custom build wit

Let's find out with a sample size of 1: Server is an HP Microserver with an Intel Xeon E3-1265L V2 @ 2.50GHz running FreeBSD 12.1. Client is a custom build with an Intel Core i7-4790K @ 4.00GHz running NixOS 20.03. $ ip route show default via 192.168.0.1 dev eno1 proto dhcp src 192.168.0.4 metric 203 192.168.0.0/24 dev eno1 proto dhcp scope link src 192.168.0.4 metric 203 192.168.1.0/24 dev wg0 scope link $ iperf3 -c 192.168.0.2 # no vpn Connecting to host 192.168.0.2, port 5201 [ 5] local 192.168.0.4 port 37382 connected to 192.168.0.2 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 115 MBytes 961 Mbits/sec 0 571 KBytes [ 5] 1.00-2.00 sec 111 MBytes 929 Mbits/sec 0 571 KBytes [ 5] 2.00-3.00 sec 112 MBytes 939 Mbits/sec 0 571 KBytes [ 5] 3.00-4.00 sec 111 MBytes 929 Mbits/sec 0 571 KBytes [ 5] 4.00-5.00 sec 112 MBytes 938 Mbits/sec 0 571 KBytes [ 5] 5.00-6.00 sec 111 MBytes 929 Mbits/sec 0 571 KBytes [ 5] 6.00-7.00 sec 112 MBytes 938 Mbits/sec 0 571 KBytes [ 5] 7.00-8.00 sec 112 MBytes 938 Mbits/sec 0 571 KBytes [ 5] 8.00-9.00 sec 111 MBytes 929 Mbits/sec 0 571 KBytes [ 5] 9.00-10.00 sec 112 MBytes 938 Mbits/sec 0 571 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.09 GBytes 937 Mbits/sec 0 sender [ 5] 0.00-10.00 sec 1.09 GBytes 934 Mbits/sec receiver iperf Done. $ iperf3 -c 192.168.1.1 # vpn Connecting to host 192.168.1.1, port 5201 [ 5] local 192.168.1.5 port 60358 connected to 192.168.1.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 108 MBytes 905 Mbits/sec 2 274 KBytes [ 5] 1.00-2.00 sec 106 MBytes 890 Mbits/sec 0 274 KBytes [ 5] 2.00-3.00 sec 107 MBytes 895 Mbits/sec 0 274 KBytes [ 5] 3.00-4.00 sec 107 MBytes 895 Mbits/sec 0 289 KBytes [ 5] 4.00-5.00 sec 107 MBytes 895 Mbits/sec 0 289 KBytes [ 5] 5.00-6.00 sec 107 MBytes 896 Mbits/sec 0 290 KBytes [ 5] 6.00-7.00 sec 104 MBytes 874 Mbits/sec 0 290 KBytes [ 5] 7.00-8.00 sec 106 MBytes 885 Mbits/sec 0 290 KBytes [ 5] 8.00-9.00 sec 105 MBytes 885 Mbits/sec 0 290 KBytes [ 5] 9.00-10.00 sec 107 MBytes 896 Mbits/sec 0 290 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.04 GBytes 892 Mbits/sec 2 sender [ 5] 0.00-10.00 sec 1.04 GBytes 891 Mbits/sec receiver iperf Done.

tachion 6y ago
I would assume you're testing that on the stock kernel settings that aren't really prepared for the highest network throughtput. There's a lot that can be done in the kernel sysctl's tuning for saturating the NIC and I'd expect you to see a bit better results when doing so. This is a very nice starting point for those interested: https://calomel.org/freebsd_network_tuning.html https://calomel.org/freebsd_network_tuning.html
- sigwinch28 6y ago
  I am running stock kernel settings. I would naively expect that the default kernel settings for both Linux and FreeBSD would allow me to saturate a 1Gbit link in a LAN. Anyway, this looks like one of those things I could go down the rabbit hole of tuning (so I'm not just copy-pasting swathes of configuration without understanding it), but this was just a quick demo which shows that: "basically, the userspace implementation isn't too slow".
  - tachion 6y ago
    At least in case of FreeBSD, the network saturation isn't an active goal of default kernel settings, hence the link I've pasted. It's especially nice, as it explains a lot of things it proposes so that the blind copy&paste wouldn't be so blind. It's really a good read. And I do get a point of your test and I agree with the anecdotal conclusion :)
- justin66 6y ago
  Judging from the "no vpn" set of numbers, the stock kernel is admirably prepared for the highest network throughput.
- pacija 6y ago
  Some quite knowledgeable people in the field of BSD networking, including Henning Brauer, maintainer of OpenBSD's PF, have little love for instruction given on site you are linking to: https://marc.info/?l=openbsd-misc&m=130105013025396&w=2 https://marc.info/?l=openbsd-misc&m=130105013025396&w=2

tachion 6y ago

I would assume you're testing that on the stock kernel settings that aren't really prepared for the highest network throughtput. There's a lot that can be done in the kernel sysctl's tuning for saturating the NIC and I'd expect you to see a bit better results when doing so. This is a very nice starting point for those interested: https://calomel.org/freebsd_network_tuning.html https://calomel.org/freebsd_network_tuning.html

sigwinch28 6y ago
I am running stock kernel settings. I would naively expect that the default kernel settings for both Linux and FreeBSD would allow me to saturate a 1Gbit link in a LAN. Anyway, this looks like one of those things I could go down the rabbit hole of tuning (so I'm not just copy-pasting swathes of configuration without understanding it), but this was just a quick demo which shows that: "basically, the userspace implementation isn't too slow".
- tachion 6y ago
  At least in case of FreeBSD, the network saturation isn't an active goal of default kernel settings, hence the link I've pasted. It's especially nice, as it explains a lot of things it proposes so that the blind copy&paste wouldn't be so blind. It's really a good read. And I do get a point of your test and I agree with the anecdotal conclusion :)
justin66 6y ago
Judging from the "no vpn" set of numbers, the stock kernel is admirably prepared for the highest network throughput.
pacija 6y ago
Some quite knowledgeable people in the field of BSD networking, including Henning Brauer, maintainer of OpenBSD's PF, have little love for instruction given on site you are linking to: https://marc.info/?l=openbsd-misc&m=130105013025396&w=2 https://marc.info/?l=openbsd-misc&m=130105013025396&w=2