3 ms·
"In a readout system such as ours the network only consists of a data sender and a data receiver with an optional switch connecting them. Thus the only places w
by zamadatix 7y ago
"In a readout system such as ours the network only consists of a data sender and a data receiver with an optional switch connecting them. Thus the only places where congestion occurs are at the sender or receiver. The readout system will typically produce data at near constant rates during measurements so congestion at the receiver will result in reduced data rates by the transmitter when using TCP."
At that point a better paper title would have been "Increasing buffers or optimizing application syscalls to receive 10 GB/s of data" as it has nothing to do with achieving reliable UDP transmission, which it doesn't even seem they needed:
"For some detector readout it is not even evident that guaranteed delivery is necessary. In one detector prototype we discarded around 24% of the data due to threshold suppression, so spending extra time making an occasional retransmission may not be worth the added complexity"
As far as actual reliable UDP testing at high speeds one might also want to consider the test scenario as not all Ethernet connections are equal. The 2 meter passive DACs used in this probably achieve ~10^-18 bit error rate (BER) or 1 bit error in every ~100 petabytes transferred. On the other hand go optical even with forward error correction (FEC) it's not uncommon to expect transmission loss in the real world. E.g. looking at something a little more current https://blogs.cisco.com/sp/transforming-enterprise-applications-with-25g-ethernet-smf https://blogs.cisco.com/sp/transforming-enterprise-applicati... is happy to call 10^-12 with FEC "traditionally considered to be 'error free'" which would have likely resulted in lost packets even in this 400 GB transfer test (though again they were fine with up to 24% loss in some cases so I don't think they were worried about reliable as much as reading the paper title would suggest).
Generally if you have any of these:
1) unknown congestion
2) unknown speed
3) unknown tolerance for error
You'll have to do something that eats CPU time and massive amounts of buffers for reliability. If you need the best reliability you can get but you don't have the luxury of retransmitting for whatever reason then as much error correction in the upper level protocol as you can afford from a CPU perspective is your best bet.
If you want to see a modern take on achieving reliable transmission over UDP check out HTTP/3.
- aDfbrtVt 7y agoTraditional error free transmission in optical comms is 1E-15 BER. I can't access the EPON standard right now, but my experience with other IEEE standards would tell me they're probably guaranteeing 1E-15 for worst-case optical link. This link is pretty close to optimal, so 400G of data is nowhere the amount to say anything with certainty about the BER of the channel.
- zamadatix 7y agoIEEE only guarantees 10^-12 which is almost certainly why 1st gen 25G products released exactly when they were able to hit that. My estimate a 2m 10G DAC from 2017 would have a BER of ~10^-18 is from personal experience (As unlikely as it sounds I actually have done extensive testing 7 of the exact model server and NIC in our lab purchased about the same time, different switch though) not derived from the 400 GB transfers in the paper.
- ignoramous 7y ago> Generally if you have any of these: 1) unknown congestion 2) unknown speed 3) unknown tolerance for error > ... If you want to see a modern take on achieving reliable transmission over UDP check out HTTP/3. Not an expert but I have seen folks here complain that QUIC / HTTP3 doesn't have a proper congestion control like uTP (BitTorrent over UDP) does with LEDBAT: https://news.ycombinator.com/item?id=10546651 https://news.ycombinator.com/item?id=10546651
- wmf 7y agoLEDBAT-style congestion control is not proper for "foreground" Web traffic and it will result in lower performance than TCP-based HTTP. Fixing bufferbloat is an ongoing project and it isn't fair to blame QUIC for being no worse than TCP.