4 ms·

I would love to see an IXIA or Spirent line rate RFC2544 test with an imix packet sizes. Most servers and OS stacks cannot do line rate 100G/bits today without

I would love to see an IXIA or Spirent line rate RFC2544 test with an imix packet sizes. Most servers and OS stacks cannot do line rate 100G/bits today without a ton of tweaking. Netflix has a few blogs on it. In my current company we moved our NAS to 2x100G but the build servers at 10G and 25g (Ubuntu) cannot max a link in their normal operation (which includes pulling 4g VMs). There is some low hanging fruit here tweeting wise we can do but.. The NAS is TrueNAS with SSD and all the ZFS goodness, ZIL, etc. I can built hand crafted iperf streams between it and another 100g server that hit line rate (including overhead). The reason for the 100g is there are racks and racks of BS that pull down their images so aggregate we can hit 100g with a fully non-blocking Clos fabric.

wmf 6y ago
You're kind of mixing different things here. Endpoints (like Netflix or your NAS and build server) have completely different characteristics than switches/routers. Likewise, interrupt-based kernel-based networking is very different and ~10x slower than polling-based kernel-bypass. Doing 10G routing per core using polling was demonstrated years ago so 100G really isn't an extraordinary performance claim.
sarosh 6y ago
Is there a hardware write up anywhere? Would be very interested to read up on it.
- idorosen 6y ago
  As an alternative, back in the mid-2000s there was a multicast system image distribution tool called flamethrowerd. It was used to netboot clusters of Linux & Mac machines. Probably abandoned, but still available for download.[1] I wonder what happened to that tool, or if there are any good modern replacements in common use. (It was a single purpose UDP multicast file distribution tool.) [1]: https://sourceforge.net/projects/systemimager/files/flamethrower/0.1.8/ https://sourceforge.net/projects/systemimager/files/flamethr...
yjftsjthsd-h 6y ago
> The reason for the 100g is there are racks and racks of BS that pull down their images so aggregate we can hit 100g with a fully non-blocking Clos fabric. It may not pan out in practice, but in theory isn't that the perfect usecase for p2p image distribution?
- dicknuckle 6y ago
  could probably do it with BTsync if you don't want to build your own. Or there's also Syncthing which is more open source
  - yjftsjthsd-h 6y ago
    I was thinking ipfs, but yeah there's lots of ways to skin this cat.

wmf 6y ago

You're kind of mixing different things here. Endpoints (like Netflix or your NAS and build server) have completely different characteristics than switches/routers. Likewise, interrupt-based kernel-based networking is very different and ~10x slower than polling-based kernel-bypass. Doing 10G routing per core using polling was demonstrated years ago so 100G really isn't an extraordinary performance claim.

sarosh 6y ago

Is there a hardware write up anywhere? Would be very interested to read up on it.

idorosen 6y ago
As an alternative, back in the mid-2000s there was a multicast system image distribution tool called flamethrowerd. It was used to netboot clusters of Linux & Mac machines. Probably abandoned, but still available for download.[1] I wonder what happened to that tool, or if there are any good modern replacements in common use. (It was a single purpose UDP multicast file distribution tool.) [1]: https://sourceforge.net/projects/systemimager/files/flamethrower/0.1.8/ https://sourceforge.net/projects/systemimager/files/flamethr...

yjftsjthsd-h 6y ago

> The reason for the 100g is there are racks and racks of BS that pull down their images so aggregate we can hit 100g with a fully non-blocking Clos fabric. It may not pan out in practice, but in theory isn't that the perfect usecase for p2p image distribution?

dicknuckle 6y ago
could probably do it with BTsync if you don't want to build your own. Or there's also Syncthing which is more open source
- yjftsjthsd-h 6y ago
  I was thinking ipfs, but yeah there's lots of ways to skin this cat.