4 ms·

> The average [of iperf3] across multiple tests was about 7.5 Gbps. > Ntttcp averaged about 12.75 Gbps […] Ntttcp does something called pre-posting receives, w

by klabb3 3y ago

> The average [of iperf3] across multiple tests was about 7.5 Gbps. > Ntttcp averaged about 12.75 Gbps […] Ntttcp does something called pre-posting receives, which is unique to this tool. This is TCP we’re talking about, and with large enough buffer—per-syscall (128k on my iperf3 by default - that’s good) you should be able to saturate even a thick line, provided auto-tuning is enabled correctly. But just add -P 16 to parallelize to be sure it’s not the TCP stack, instead? Did they even try a workaround, before suggesting an extremely limited and proprietary tool that nobody is familiar with? Or better yet, help the project out by contributing native code if the open source community is so bad at windows? At the very least, make a wire-compatible version. Iperf is a very boring and wonderful tool that just works. Importantly, it’s a networking tool, which means you need interop across different systems. Any networking tool that’s single-platform is dead in the water, as far as I’m concerned. Edit: To be fair, I see they have a Linux version, how generous. Last updated in 2022 and many open issues though.

distortedsignal 3y ago
After reading the article, it sounds like iPerf3 on Windows is going through an emulation layer (Cygwin) which - I kinda get why it's going slower if that's the case. Isn't syscall translation always going to add overhead?
- deleted 3y ago
  [deleted]
- deleted 3y ago
  [deleted]
raggi 3y ago
It’s a shame that they don’t talk about it in the article but a big part of the drive toward next generation tools is to study the interfaces needed and used by quic implementations which can’t “just be solved by” the flags you’re talking about, and iperf has no support for the relevant features (offloads).
- klabb3 3y ago
  Yeah, to be fair I’m assuming tcp both on the iperf side (which I always use it for) and the alternative ntttcp tool does have TCP in the name although it appears they have UDP support as well(?) I know UDP and QUIC comes with a ton of novel bottlenecks throughout the stack (even down to the NICs, no?), and at the very least tons of knobs, which warrants more custom tooling and keeping up with the latest in terms of OS/platform support. I can totally see how iperf isn’t suited for that. Still, it’s a pretty strong and different message that Microsoft is sending on an official channel, which is unfortunate if it indeed is the case that iperf works just fine on windows with pretty standard flags (note that even on typical Linux recv buffers are constrained to ~6MiB so -P is standard for WAN saturation tests). (Also good to see your relentless attention to engineering details as always. We used to work together some years back)
  - raggi 3y ago
    It largely depends what you mean by "works just fine", if all you want is "high throughput on a low latency link" then sure, whack up the buffer and throw it at the wall, it'll stick - but you can just as well use curl for that. To study higher end network stacks or more modern protocol dynamics, or cross network flows, the other tools are providing much better insights. The need for -P for basic tests is also telling and part of the problem: if the tool can't outpace a browser then even for these "basic cases" it's showing its inadequacy for the kinds of things the bulk of users care about even today, even with old protocols.

by klabb3 3y ago

distortedsignal 3y ago

After reading the article, it sounds like iPerf3 on Windows is going through an emulation layer (Cygwin) which - I kinda get why it's going slower if that's the case. Isn't syscall translation always going to add overhead?

deleted 3y ago
[deleted]
deleted 3y ago
[deleted]

raggi 3y ago

It’s a shame that they don’t talk about it in the article but a big part of the drive toward next generation tools is to study the interfaces needed and used by quic implementations which can’t “just be solved by” the flags you’re talking about, and iperf has no support for the relevant features (offloads).

klabb3 3y ago
Yeah, to be fair I’m assuming tcp both on the iperf side (which I always use it for) and the alternative ntttcp tool does have TCP in the name although it appears they have UDP support as well(?) I know UDP and QUIC comes with a ton of novel bottlenecks throughout the stack (even down to the NICs, no?), and at the very least tons of knobs, which warrants more custom tooling and keeping up with the latest in terms of OS/platform support. I can totally see how iperf isn’t suited for that. Still, it’s a pretty strong and different message that Microsoft is sending on an official channel, which is unfortunate if it indeed is the case that iperf works just fine on windows with pretty standard flags (note that even on typical Linux recv buffers are constrained to ~6MiB so -P is standard for WAN saturation tests). (Also good to see your relentless attention to engineering details as always. We used to work together some years back)
- raggi 3y ago
  It largely depends what you mean by "works just fine", if all you want is "high throughput on a low latency link" then sure, whack up the buffer and throw it at the wall, it'll stick - but you can just as well use curl for that. To study higher end network stacks or more modern protocol dynamics, or cross network flows, the other tools are providing much better insights. The need for -P for basic tests is also telling and part of the problem: if the tool can't outpace a browser then even for these "basic cases" it's showing its inadequacy for the kinds of things the bulk of users care about even today, even with old protocols.